Our objective for running this year's contest was three fold:
1) Bring awareness of the Turing Test to people who had never heard of Alan Turing and his imitation game, especially to children
2) Invite people who had heard of Turing, including non-experts who had read academic discussions of his writings, to experience a live imitation game
3) To test, through five minutes, unrestricted topic of conversation, parallel-paired against a human, whether any of the 2008 Loebner entries could achieve this:
"I believe that in about fifty years' time it will be possible to programme computers, with a storage capacity of about 10 to the power 9, to make them play the imitation game so well that an average interrogator will not have more than a 70% chance of making the right identification after five minutes of questioning"
(Computing Machinery & Intelligence, Mind, Vol. LIX, No 236, 1950)
We succeeded with the first two; we believe that it won't be long before a system succeeds with the third. In an email from the UK Government Department responsible for Children Schools and Families, this comment was received:
"Government very much welcomes the continued support you and other industries have given to our work on Science, Technology, Engineering and Mathematics (STEM) and commends your efforts to raise young people's awareness of science through the Loebner Prize event. The Minister for schools Jim Knight MP has asked me to pass on his thanks"
I am delighted to announce that Elbot, by Fred Roberts/ Artificial Solutions won the 2008 Loebner Prize for Artificial Intelligence bronze award for 'most-human-like' machine, after deceiving three human judges that it was a human that they were chatting with.
Winner, Fred Roberts wrote:
"...haven’t said it clearly enough, Mark [MATT protocol developer] did a fantastic job .... It was a fun contest. I’d say that even if Elbot hadn’t have won."
You can read more about Elbot's win here, and chat to Elbot here.
Eugene, developed by Vladimir Veselov, Eugene Demchenko & Sergey Ulasen, came second in the contest, convincing one judge, a Times newspaper journalist, that it was a human. Here is that journalist's piece on their experience.
Vladimir Veselov, who attended the event had this to say:
"I want to thank Huma, Mark, and all organizers of the Loebner Prize 2008. They did a tremendous work preparing the event."
Included below some pictures taken with my PDA, before testing began at 8.30am, in the judges/spectators room:
Left: judges terminals
Below: graphic of parallel set up
BBC news video of the contest can be viewed here.
BBC pictures and more information can be found here.
The MATT message-by-message communications protocol - using ICE (Internet communication engine), facilitated the five minutes, unrestricted conversation, ACE parallel-paired with hidden-humans imitation games, was developed by Marc Allan. As Loebner Prize Sponsor, Hugh Loebner wrote: "I'm sure that anyone who can develop an artificial intellect should be able to interface with ICE". [message #10132, Robitron forum, Thu Mar 20, 2008 3:46 pm]
University of Reading, School of Systems Engineering staff research posters in spectators area
Judges wrote of their experience:
"It was fascinating."
"Just a quick email to thank you for a wonderful day. The event itself was a great success and it definitely passed the test to qualify as a great meeting. So congratulations and many thanks again!"
"My experience of being a judge was very interesting. For the first two conversations I had, it was quite easy to tell which was the bot and which was the hidden-human. BUT, for the third one I really couldn't tell which was which!!"
One of the hidden-humans wrote:
"Thank you so much for giving me the opportunity to be part of the Loebner Prize. I had great fun!"
And finally, a couple of quotes from posters on the New Scientist piece about Elbot's win:
"It's of course still nowhere near passing the "imitation game", which is philosophically valid. Asking any question is a better test of true intelligence than a limited (but hard) domain like visual recognition. I think the design should be fully open to psychologists trying to understand human cognition." [Future In Politics, By Cedric Knight, Mon Oct 13 21:48:09 BST 2008]
"There's already chat bots out there that can do a much better job. But they're not competing for this prize, they're hooking people on IM into clicking affiliate links to download porn or links to malware. If this contest wants to see the best, they're going to have to offer a prize that beats the payoff people are finding in the commercial market." [Need A Larger Prize, By Jeremy Tue Oct 14 15:38:56 BST 2008]
In all, a hectic few months culminated in a great contest, thank you to all participants: contestants, judges and hidden-humans.
Right: Cyborg Head sculpture on display between judges terminals (artist: Luqman).Links to other articles:
New Scientist
Reading Chronicle
The Herald
Telegraph
Good luck to the organisers and participants of Loebner 2009 :-)
(first posted 15/10/08)
