Sunday, November 15, 2009

Problem with BCS's Machine Intelligence Competition

BCS 2009 Machine Intelligence Contest announce A.L.I.C.E* involvement:

"The British Computer Society Specialist Group on Artificial Intelligence (SGAI) has announced the finalists in the BCS Machine Intelligence competition, the eighth in a regular series held at its annual conferences at Peterhouse College, Cambridge. The prize is awarded on the basis of delegate voting to the system ‘which best demonstrates progress towards machine intelligence’.

One of this year’s finalists is ALICE, a chatbot developed by Dr. Richard Wallace and his colleagues at the ALICE AI Foundation. It aims to demonstrate that simple behaviourist stimulus-response methods, when applied on a large scale, lead to a believable illusion of intelligence in the context of Turing's Imitation Game. It has three times won the prestigious Loebner Prize, an annual international artificial intelligence competition for the most human-like chatbot." (From BCS_SIGAI email)

* Alice last won a Loebner Prize back in 2004. In that competition, in points awarded it was actually runner up to John Precedo's system, disqualifed on a technicality. Alice came sixth in Loebner Prize 2008.

"Will it win the BCS Machine Intelligence prize to add to this, in the face of opposition from

Dora the Explorer, a mobile robot which is able to inspect its own world knowledge for gaps and then perform behaviours to fill these gaps (e.g. by searching the room for objects which support inference about room category, such as a kettle in a kitchen).

Taaable, a system that illustrates how humans and machines can collaborate to reason and can share common knowledge on a particular domain, in this case creating recipes.

Fly by Ear, an intelligent autonomous helicopter which has been evolved to react optimally to its environment by listening to the sound emitted by the helicopter and interpreting the information transferred.

The competition is on Wednesday December 16th and is open to all delegates to the SGAI annual conference AI-2009 or the SGAI Real AI Day (the following day) " (From BCS_SIGAI email)

Problems with BCS's Machine Intelligence contest include that it is exclusive. Only delegates to BCS SIGAI conference have access. Secondly, the contest does not gather like-technology; contestants differ in what type of intelligence they attempt to demonstrate. For example, in 2009 Alice's dialogue will compete alongside Fly by Ear's helicopter autonomy.

As delegates decide, presumably not all can be experts in the dissimilar competing technologies, hence, judgment will be based on subjective opinion of "progress towards machine intelligence". Will the visceral concede to the abstract this year? Chatbot systems, Carpenter's Jabberwacky, and David Burden's HALO are past BCS Machine Intelligence contest entrants. Neither won it, good luck to Alice designer, Richard Wallace.