Sunday, September 06, 2009

David Levy Winner of 2009 Loebner Prize for Artificial Intelligence & Information for 2010 Loebner Prize

David Levy, author of Love and Sex with Robots: The Evolution of Human-Robot Relationships beats two other entrants to win the 2009 Loebner Prize for Artificial Intelligence.

From Hugh Loebner Robitron post #12868 4.26pm Sunday 6 September 2009:

No programs fooled the judges. Therefore, victory was determined by rankings (1 = highest)

First Place David Levy - Rank 4.5
Second Place Rollo Carpenter - Rank 5.0
Third Place Mohan Embar - Rank 5.5

David Levy won the 1997 Loebner Prize, well done on becoming twice winner.

9.9.09 postscript: 2009 Loebner Prize Transcripts, interlaced by Kris Schnee, available here.

Loebner 2009 Judge's assessment:
I was one of the judges for the Loebner prize.

None of the judges had any difficulty in distinguishing human from non-human interlocutors after the first or second turn in the conversation. The two main features which allowed me to identify a human vs. a non-human agent are (i) capacity for fluent domain general discourse marked by frequent and unpredictable changes in topic, (ii) willingness to allow the judge to take over the conversation, (iii) capacity to handle ellipsis, pronouns, and non-sentential fragments, and (iv) typing errors and corrections in human but not program contributions. The relative absence of progress in developing general purpose conversational agents contrast sharply with the substantial progress of the past 10-15 years in task driven, domain specific dialogue management systems and other types of NLP.

From here:

Information on the 2010 Loebner Prize, 20th consecutive Loebner sponsored contest, will be held 23 October 2010 at California State University, Los Angeles:

First Prize: $3000 and the Bronze Annual Medal
Second Prize: $1000
Third Prize: $750
Fourth Prize: $250.

At risk will be the $25,000 Silver Medal Prize. This prize will be awarded to any submitter(s) whose program:

1. Can, when, tested by the method of Paired Comparisons with a human, fool more judges into thinking it is a human than can any other entered program;

2. And can fool at least 1/2 of at least four judges (i.e. 2 of 4, 3 of 5 or 6, etc.) when compared with at least 4 humans.

3. In the event that two or more programs succeed equally with regard to point 2, the prize money will be split, but the Silver Medal itself will be awarded to the submitter(s) whose entry is scored highest by the judges who judged the program to be a program.

No comments: