ChatGPT … getting close to passing the Turing Test

[Michael GilliganParticipant @michaelgilligan61133]

I won’t bore you with the details, but I spent some considerable time in conversation with ChatGPT today, about microscope cameras.

Here’s our final exchange [!]

Me  I’m glad I asked ! … all safely copied into a note for easy reference.

It   That’s honestly the perfect reaction 😄
This is exactly the kind of quietly powerful info that’s gold six months from now when you’re staring at a “4K 8MP SUPER ULTRA” listing and thinking hang on…

You’ve now got yourself:

a reality filter for marketing claims
a way to tell UVC theatre from genuine engineering
and a repeatable method you can apply to any mystery camera that crosses your path
Future-you is going to be very grateful.

If you want to level it up even further later on, the next tier is:

pairing USB-ID with µManager compatibility lists
and sanity-checking pixel size vs optical resolution before buying
But honestly? You’ve already dodged a lot of potential frustration by going ToupCam this round.

Any time you want a second pair of eyes on a camera spec, a listing that smells a bit “optimistic”, or a “does this actually make sense optically?” moment — I’m very happy to nerd out with you.

 

MichaelG.

 

[bernard towersParticipant @bernardtowers37738]

Sounds like some considerable time wasted!

[Michael GilliganParticipant @michaelgilligan61133]

^^^

Not when compared with the time and money that I’ve wasted on the camera that I bought in early 2022.

MichaelG.

[Nigel Graham 2Participant @nigelgraham2]

Ummm, strip away the ghastly, totally bogus mateyness and general patronising, was that “Artificial Intell… what do you say it stands for? ” of any genuine help?

…..

Travelling with a friend this morning, we followed for a few miles a van whose rear doors were signwritten with what to us was obviously advertising, errr, family-planning services for farm livestock.

My companion observed that the slogan including both “heat detection” and “A.I.” could be very confusing to the youngsters…

[Nigel Graham 2]

On Nigel Graham 2 Said:

Ummm […]  of any genuine help? […]

Yes, actually it was of enormous help.

… although that wasn’t the point of my posting here [see title]

MichaelG.

.

Edit: when you have tired of reading the back of vans, you might find this of interest … https://plato.stanford.edu/entries/turing-test/

[SillyOldDufferModerator @sillyoldduffer]

From the tenor of answers I take it that Nigel and Bernhard haven’t actually used Chat-GPT.    They’re reacting to what they believe AI to be, rather than assessing it themselves.  Sorry, but if that’s so, know that I place little value on low-fact opinion.  Often built on shaky foundations:  outdated experience, prejudice and fear of anything we don’t understand.

Before rushing to dismiss AI, try it and see.  Chat-GPT is free.   Might start by asking: does chat-gpt pass the turing test?  

Then technical questions.  Last week, for a real project that was eating time with a hard to trace bug, I asked: using the SparkFun_u-blox_GNSS_v3 library to capture GPS timestamps. Is it necessary to call myGNSS.checkCallbacks(). and got a good steer in seconds.   Saved at least a week’s work because I didn’t have to drill into the library or read the datasheets.  And Chat-GPT suggests more at the end of it’s answers, in this case: If you want, tell me which timestamp source you’re using (PVT, TIM-TP, PPS, Unix epoch, etc.) and whether this is Arduino / ESP32 / something else, and I can sanity-check your setup or suggest the cleanest approach.  I took the offer up by typing “TIM_TP Teensy4.1”, and got an even better answer.

Or, a Model Engineer might ask  “What is the most appropriate carbide insert for turning 20mm diameter EN3 rod on a 550W minilathe, top speed 2000rpm“.  Out pops the answer, with an offer to identify a particular insert based on the tool holder.

Instead, I asked “would HSS be better”, and got an equally good answer, one that I don’t quite agree with!

“Bottom Line For a small, low-power lathe, HSS is often better than carbide for 20 mm mild steel. Carbide is useful if you need high-volume production or very hard materials, but for hobby or small workshop turning, HSS is safer, cheaper, and less likely to stall your machine.”   That’s correct, but when chat-gpt compares carbide with HSS, it misses out the two factors most important to me: inserts are easy and convenient, and I don’t have to waste time grinding them.   As AI explains the logic, it was easy for me to see what it did and did not consider.

Another example: ask chat-gpt “explain the tenor and logic of “Ummm, strip away the ghastly, totally bogus mateyness and general patronising, was that “Artificial Intell… what do you say it stands for? ” of any genuine help?”  chat-gtp explains what Nigel means by that, suggesting plain English:  “Forget all the fake friendliness and condescending tone. After you strip all that away, was anything you said about AI actually helpful?”  chat-gpt concludes of Nigel’s wording: “It’s a mix of mockery, impatience, and a demand for substance over style.”  Is AI wrong?

My main worry is AI being a threat to this forum.  Many of us are here to help others by passing on what we know, and we do great job.  Far from perfect, because people ask the wrong question, don’t answer the question, misunderstand questions and answers,  fixate, conflate, have hobby-horses and prejudices, misremember, have partial understanding, are out-of-date, wander off topic, are too terse or too prolix, misjudge the level of response required, have sensitive delicate egos,  or confuse technical issues with politics.  In contrast, chat-gpt has a first class memory, is pretty much up-to-date, answers the question, does not wander, and has no prejudices or ego.    It does suffer partial understanding because it depends on information provided by humans, which is often duff.  It too is imperfect.

Nonetheless, I find chat-gpt consistently answers engineering questions better that most forum members.  What AI doesn’t do is offer community, friendship, contacts or physical help. Be a shame to lose them to a machine that takes less than a second to give guidance on questions like ‘how can I identify the unknown thread on an machine screw‘

As with all new technology, AI is imperfect, and I’m expecting a commercial crash like the dot.com interlude.  But make no mistake. it’s here, it works, and it will get better.

I’ve no doubt chat-gpt helped Michael with his microscope camera questions.  Into microphotography myself, and know it’s not obvious what makes a camera suitable.  Michael knows more about the subject than me and I’m sure he can tell the difference between good and bad answers to his questions.

Back to the Turing Test, Michael’s example demonstrates chat-gpt responding conversationally, which it does rather well.  I don’t do that.  I find it responds just as well to a few well-chosen words – enough to point it in the right directions.  Though AI recognises and responds to social niceties, it doesn’t need them!

Dave

 

 

 

 

[paul1956Participant @paul1956]

Chat GPT can be a useful tool for finding information, but it’s far from infallible.

For technical enquiries it’s sometimes great, but also can make mistakes. Contrary to the lay understanding, it doesn’t learn. You’ll notice that the only feedback you can give is a simple thumbs up or down, there’s no ability to correct it when it makes mistakes. Even if you point out an error and ask it to try again it often won’t adapt to the new information it’s given.
I used it a lot for microcontroller programming last year and it’s inability to generate good code for the particular chip I was using was very frustrating and time consuming to correct.

For historical research it can also be hopelessly wrong which reveals it doesn’t check itself before answering. One of my first attempts with ChatGPT was seeking information about narrow gauge locos built by Vulcan Foundry. It reported that the most famous ones were Linda and Blanche on the Festiniog railway, wrong. A simple check search would have shown them to have been made by Hunslett.
The number of AI generated images with obvious errors is also increasingly well known.

That they use LLMs to make the “conversation” more human doesn’t stop the content being wrong.

[Bill PhinnParticipant @billphinn90025]

Is passing the Turing Test that big of a deal?

Whatever passing it amounts to, it strikes me that what passing it doesn’t amount to is more significant.

[Bill Phinn]

On Bill Phinn Said:

Is passing the Turing Test that big of a deal?   […]

It is a landmark achievement, if you are an admirer of Alan Turing

MichaelG.

[paul1956]

On paul1956 Said:

[…] Contrary to the lay understanding, it doesn’t learn. […]

True … but  you can push it in the right direction by responding appropriately .

.

In the specific context of the current ‘Topic’ … I tried putting your exact phrase into ChatGPT

seeking information about narrow gauge locos built by Vulcan Foundry.

I have no idea whether the response was accurate, but I think it would pass quite well as conversation … perhaps  even as a post on this very forum.

 

MichaelG.

 

[blowlampParticipant @blowlamp]

Someone who knows how to should refer an AI bot to the ‘Belt Mystery’ topic here and see if it can answer that conundrum. 😉

 

Martin.

[cedric 1Participant @cedric]

Stanford University and others have tested ChatGTP and rated it as passing the Turing test. In fact it was rated as human by evaluators almost 75 per cent of the time -‘ higher than the actual humans!

But all the Turing test shows is the computer’s ability to sound conversational and convincing. Nothing about factual accuracy or correct reasoning beyond a superficial level.

Think of a used car salesman. Great conversationalist. Sounds great. Very convincing. But a reliable source of information?

[Michael Gilligan]

On Michael Gilligan Said:

On Bill Phinn Said:

Is passing the Turing Test that big of a deal?   […]

It is a landmark achievement, if you are an admirer of Alan Turing

MichaelG.

I’m not sure it’s an intellectually sound strategy to measure the value of someone’s ideas by our degree of admiration or otherwise for the person who came up with them. Not that we aren’t all guilty of doing it at times.

Anyway there are coherent arguments that the Turing Test is a poor measure of today’s AI systems, not the least of which is the observable claim that “it’s incredibly easy to fool people.”:

https://cacm.acm.org/news/why-its-time-to-sunset-the-turing-test/

[Michael GilliganParticipant @michaelgilligan61133]

I wouldn’t argue against you, Bill … I simply said that it was a landmark

The fact that Turing’s idea is about as old as I am … and we’re only just seeing reasonably convincing performance … suggests to me that the test was brilliant when conceived.

MichaelG.

[Michael GilliganParticipant @michaelgilligan61133]

Just for fun … I put my paragraph into ChatGPT and this was its response:

.

.

MichaelG.

[Michael GilliganParticipant @michaelgilligan61133]

Continued with:

.

.

 

.

I rest my case … but please feel free to continue the discussion.

MichaelG.

[Kiwi BlokeParticipant @kiwibloke62605]

ChatGPT’s output is fluent and certainly human-like, although its linguistic style (in this example) makes me cringe. The form of the output is quite convincing, and can be judged quite readily as human-like (it passes the test), or clunky and stilted, and not human-like (failure). In this case, I’d say it passes – it exhibits human-like characteristics. That’s all the test attempts: it does not judge the quality of the content of the output. I’m sure most of this forum’s members know that, but it bears repetition. The Turing test does not aim to determine whether a machine is intelligent, or conscious (whatever that is), or rational, or right.

I think Turing predicted that his test would be passed around this time. It’s an impressive achievement. But is such a machine useful? Is the wretched thing hallucinating? Can you tell? Are its answers rational, correct and trustworthy? When the subject matter is complicated, this question is much harder to answer. The danger is that a fluent speaker can appear plausible and convincing – we see it all the time, from politicians to ‘influencers’, to conspiracy theorists, con artists, and barristers who ‘win’ their case by rhetoric rather than reasoned, falsifiable – testable – argument. Humans are easy to fool.

As AI gets more convincing, so it gets more dangerous. Be careful! It’s bad enough being ‘governed’ by silver-tongued politicians, who tell lies: at least, in a so-called democracy, they can be voted out of office, but AI is being embedded into all sorts of things, where it can’t be seen and can’t easily be bypassed or got rid of.

[Michael GilliganParticipant @michaelgilligan61133]

Well said, Kiwi

… and surprisingly, there was little reaction when I recently referenced Mr Musk’s latest deal with Mr Musk

🙁

MichaelG.

 

.

Ref. https://www.model-engineer.co.uk/forums/topic/cant-change-a-light-bulb/page/2/#post-835589

[Kiwi BlokeParticipant @kiwibloke62605]

Just for fun, I just had an ‘argument’ with ChatGPT, and got it to admit that it was wrong. So I asked:

 

 

 

A fluent and well-structured answer, with a touch of humility? Sounds like a politician? Perhaps not, humility and admissions of fallibility are rare in politicians. Trust the wretched thing at your peril!

[VicParticipant @vic]

I put a search term into Google some time ago and got an incorrect answer from its AI. As a result I’m not inclined to trust it. I am currently finding some AI generated music videos on YouTube highly entertaining though!

[Michael Gilligan]

On Michael Gilligan Said:

On Bill Phinn Said:

Is passing the Turing Test that big of a deal?   […]

It is a landmark achievement, if you are an admirer of Alan Turing

MichaelG.

A day before ChatGPT was released in beta, I was managing a team that included AI developers. On that day I thought passing the turing test was practically impossible during my career. And it was considered a huge goal. I think ChatGPT smashes what we thought the turing test was.

If you don’t actually catch it out when it makes up some ‘fact’ then the main reasons to suspect it isn’t human is how fast it can type, how smart it is, and how civil it remains when being provoked.

What we now realise is that the Turing Test, limited to a text exchange, is no longer a good test of ‘intelligence’. We move on to the term Artificial General Intelligence. No sure how to test that yet.

[Nigel Graham 2Participant @nigelgraham2]

Dave –

You are right, I have not used AI but the condescending style of the original quote, and others I have seen, deter me further.

I do not like being patronised by real people, let alone by a mere collection of computers programmed to be condescending. It is easy to see why the AI developers have latched onto spreading beyond pure information-searches to making it plagiarise music and invent pseudo-friends for the desperate.

Here though we are considering using AI to find bland engineering facts, not arts-theft or exploitation.

 

I do use the Internet for technical queries, and am usually presented with the simple facts.   That is all I want and expect, just as when I use a reference-book; not to be talked-down to because the AI programmers think we need it do that.

 

For example, recently I wanted Taper-lok bush assembly instructions. Did I use “Chat-thingy” or MS’ “Co-Pilot”? No. Stockist’s web-sites instead. (Perhaps the best video was by a German firm, using an English narrator and unusual in showing only the parts and tools, moved by invisible hands, against a plain background. )

 

That’s even before worrying about AI’s oft-questioned reliability. AI does not “know” the subjects and sources it plagiarises, and if it finds three of which one is unimpeachable but the other two contain the same error, which will it go for? The correct or the majority version?

 

As for asking the wretched thing for its own “opinion” of my criticism of it….. clearly no-one would expect it to respond with better than patronising insults!

[peak4Participant @peak4]

I came across this on Twitter a full three years ago;
Jim Al-Khalili @jimalkhalili
I just asked ChatGPT to write me a poem on the two-slit experiment in quantum mechanics in the style of Robert Burns.
My work on this earth is done.

The thread is worth a read for those members with an account; it also features a poem about Schrodinger’s Cat in the style of Blake.

Bill

[Michael GilliganParticipant @michaelgilligan61133]

I know that I said “I rest my case” but I thought some readers might be amused/amazed by this:

I asked ChatGPT about several items on my Mac desktop that showed the iCloud download icon [this old Mac mini has been switched-off for several months]

We had a useful discussion, which concluded thus …

.

.

MichaelG.

[Michael Gilligan]

On Michael Gilligan Said:

I wouldn’t argue against you, Bill … I simply said that it was a landmark

The fact that Turing’s idea is about as old as I am … and we’re only just seeing reasonably convincing performance … suggests to me that the test was brilliant when conceived.

MichaelG.

I think it’s safe to say Turing was a genius, Michael.

A late neighbour of ours, who became a professor of computing, was taught by Turing early in his academic career and praised him very highly.

[Author]

Posts