Wednesday, November 04, 2009

Practical Turing Tests

Couple of new papers on Practical Turing tests:

1. Testing Turing’s five minutes, parallel-paired imitation game accepted for publication in (forthcoming) Kybernetes Turing Test Special Issue. Snippets from the abstract and conclusion -

Abstract: The authors consider Turing’s two tests for machine intelligence: the parallel-paired, three-participants’ game presented in his 1950 paper, and the ‘jury-service’ one-to-one measure described two years later in a radio broadcast. Both versions were instantiated in practical Turing tests during the 18th Loebner Prize for Artificial Intelligence hosted at the University of Reading, UK, in October 2008....

Conclusion: The Turing test supposes all humans are ‘packed’ with conversational intelligence, that interrogators could preclude their subjective notion of intelligence (Warwick, 2001), and what constitutes a machine-like response. .... What we have seen, and can conclude from that competition, is that .... ACE dialogue has improved since Eliza. Modern Elizas are able to recall, share information and disclose personal interests. The progress may appear slow, but it is present. [© Shah & Warwick 2009a]

2. Hidden Interlocutor Misidentification in Practical Turing Tests submitted for journal publication - snippet below:

Abstract: Based on insufficient evidence, and inadequate research, Floridi and his students report inaccuracies and draw false conclusions in their Minds and Machines evaluation, which this paper aims to clarify....

Conclusion: We suggest the interrogation strategy of 'power' .... resulted in a low correct identification rate. .... As Turing himself reminded: "[the] popular view that scientists proceed inexorably from well-established fact to well-established fact, never being influenced by any unproved conjecture, is quite mistaken" (1950: p.442). [© Shah & Warwick 2009c]

See also Myths and Misconceptions, re practical Turing tests in the 2008 Loebner Prize.

No comments: