Friday, September 19, 2008

What's wrong with the Turing Test?


Though many would argue that it should be killed, because it does nothing to further the science of understanding human consciousness and intelligence.

What is the Turing Test? When a machine, unseen and unheard deceives a human into believing that they are in conversation with another human, then that machine must necessarily be deemed intelligent. Why? Because it has mastered the art of deception, and has convinced that it is human through dialogue.

I contend that the Turing Test is the bottom rung on a long ladder to 'True AI'. It provides a platform to enable conversation between human and machine, thanks to Weizenbaum's paradigm seen in the first, pre-Internet natural language understanding system Eliza.

Turing's idea is simple yet brilliant. Turing even pointed the way to proceed with design of an artificial conversational entity (ACE - Shah, 2006). By dividing the problem into two parts: child programme and education process:

Instead of trying to produce a programme to simulate the adult mind,

why not rather try to produce one which simulates the child’s? If this

were then subjected to an appropriate course of education one would

obtain the adult brain (Turing, 1950 - see link to right).

Turing further added:

We cannot expect to find a good child-machine at the first attempt. One must experiment with teaching some such machine and see how well it learns.

Structure of the child-machine = Hereditary material

Changes in the child-machine = Mutations

Natural Selection = Judgment of the Experimenter

Turing added that a simple child-machine might be constructed on the principle of associating punishments and rewards with the teaching process.

Current ACE, based on Turing’s Test, can be seen in two annual competitions: Chatterbox Challenge (CBC) and the Loebner Prize.

The Loebner Prize is sponsored by American scientist and philanthropist Dr. H.G. Loebner. Transcripts of conversations from the Prize provide a feast of information about how human conversation works, an insight into human consciousness and the problem with defining conversational intelligence. It is so subjective.

© Huma Shah (first posted in blog in 2006)


Political Umpire said...

Hi Huma,

I've just added you to my blogroll, and am working through the blog to do a short introduction to the new link. I won't pretend to understand all the scientific things on your site, but reading this post I'm sure of one thing - you must LOVE Bladerunner!

All the best


Political Umpire said...

Hello Huma, hope this finds you well. Just wanted to say I hope you weren't too despondent about the result the other day - remember how they started in 1992, this is just another attempt to generate the 'cornered tigers' mentality ... !

Political Umpire said...

hope you're ok :-)

Political Umpire said...

Hi Huma

Isn't it just getting worse and worse about Woolmer? No real clue as to what happened, but if it transpires that it was bookmakers irate about the Ireland game, cricket's credibility will be in deep trouble.

Political Umpire said...

Hello Huma,

Hope you're enjoying the game. I'm off to Lords today ...

Anonymous said...

I don't think it's the Turing Test which is bad, but perhaps it just should be properly implemented. A computer that could truly carry on a conversation with a human, without the responses that characterize chatterbots, would convince me of the presence of an artificial intelligence. It's just that contests like the Loebner Prize tend to encourage chatterbot development rather than true AI. The only organization I know of that is actually striving to create a true AI would be Cycorp, and no doubt stands the best chance of passing a properly administered Turing Test.

So, if it's done right, let's do it...but let's do it right : ) Darren

Huma said...

Darren: would you contact me? If you leave your email I won't publish it. I'd like to let you know about Loebner 2008 and wonder if you'd consider participating?

huoyangao said...

Turing Test Two:
In Turing Test Two, two players A and B are again being questioned by a human interrogator C. Before A gave out his answer (labeled as aa) to a question, he would also be required to guess how the other player B will answer the same question and this guess is labeled as ab. Similarly B will give her answer (labeled as bb) and her guess of A's answer, ba. The answers aa and ba will be grouped together as group a and similarly bb and ab will be grouped together as group b. The interrogator will be given first the answers as two separate groups and with only the group label (a and b) and without the individual labels (aa, ab, ba and bb). If C cannot tell correctly which of the aa and ba is from player A and which is from player B, B will get a score of one. If C cannot tell which of the bb and ab is from player B and which is from player A, A will get a score of one. All answers (with the individual labels) are then made available to all parties (A, B and C) and then the game continues. At the end of the game, the player who scored more is considered had won the game and is more "intelligent".

Huma said...

That's not the Turing Test that's moving the goal post!

Presumably, in TT2, ABC are unacquanted with each other? If C asks "how many siblings do you have?" and both A & B answer along the lines of "don't know" re the other's response (your 'ab' and 'ba' answers), and C continued with personal questions, then there'd be no point to your variation of the TIG.

Any judgment on the part of C is subjective; who ever wins is not necessarily the more 'intelligent' - a better liar perhaps.

Thank you for your post, interesting idea.