Can a machine think? Report on the Turing tests in Loebner Prize 2008

How Loebner 2008 Prize judges determined that the artificial agents were not thinking (in the human sense) included:

1. ...the ACE did not make spelling mistakes in contrast to the hidden-humans ... this is a "fairly key signifier of human-ness"

2. ... 'speed of response' and 'length of utterance': If an entity replied very quickly with a long sentence, or asked a question in response to a question, it was more likely to be a machine; whereas if a discussion developed with a hidden entity answering ... questions, then they were likely to be human.

Another judge correctly identified the gender of hidden-humans, against whom the artificial conversational agents were compared:

.. male as "less conversational (short answers, to the point)" and female hidden-humans as "quite happy to tell me a lot about themselves and their life".

From the organisers' report on Loebner Prize 2008, with ACE finalist's average scores here.