Conversation, Deception and Intelligence: Hugh Loebner did a David Bowie: Died Surprisingly Quietly.

Hugh Loebner from 2006 Prize, UCL

One week ago, December 4, 2016 a tweet from @eloebner announced “Hugh LOEBNER died peacefully in his sleep” [1].

Hugh Loebner’s ex-wife’s message didn’t sink in. That day, like many, I was in anticipation of Westworld TV show’s Season 1 finale [2] promised by Sky Atlantic to air at the same ‘time’ (2am UK 5 Dec) as for viewers in the US (9pm Dec 4), the mesmerising “dawning of consciousness” [3] robot epic based on the 1973 movie starring Yul Brynner as the refusing-to-die killer robot [4].

One week on, I’m reminded of John Sundman’s 2003 article in Salon Artificial Stupidity [5]:

“All Hugh Loebner wanted to do was become world famous, eliminate all human toil, and get laid a lot. And he was willing to put up lots of good money to do so. He’s a generous, fun-loving soul who likes to laugh, especially at himself. So why does everybody dislike him so much? Why does everybody give him such a hard time?”

I don’t think this is the time to go over his human failings, because we all have them, what I do want is to remind the academic world and beyond how approachable Hugh Loebner was. And yes he did have a sense of humour, read his cheeky retort claiming that the (now late) Marvin Minsky supported the Loebner Prize [6]. But more than that, Hugh had a profound understanding of Turing’s idea about how to build and measure whether a machine could think [7] - see his response to Stuart Shieber’s article Lessons from a Restricted Turing test [8].

I’m loathe to divulge personal experiences of interactions with Hugh Loebner, and there were many. I first saw him in 2003 at the 13^th Loebner Prize contest at the University of Surrey. Then from 2005 discussions began for me to organise his 16^th contest. I co-organised two Loebner Prizes: at UCL in 2006 [9], and at Reading University in 2008 [10]. Hugh allowed me to design his 18^th contest as an original Turing test experiment different from all his previous Prizes– see Chapter 5 in my PhD thesis [11]. I can say as late as 2014, long after I’d gone my own way designing my own Turing test events, at Bletchley Park [12] on the 100^th anniversary of Alan Turing’s birth June 23^rd, 2012, and another on the 60^th anniversary of Turing's passing away, at The Royal Society London [13], 6-7 June, 2014, Hugh remained approachable. He responded swiftly to my queries on the pass-rate in his 2014 Loebner Prize (I was concerned and have many emails making a plea not to use the ‘unsure’ score in my own 2014 Turing tests as part of a RoboLaw project dissemination event, and the media backlash following prove me right, but that’s another story).

Anyway, Hugh clarified that in his 2014 contest “The program must fool half the judges. That automatically means it will be compared to two different humans. Each judge meets each human once.” (Personal email to me: 18 May 2014 at 16:26).

Back to Hugh’s own words, here’s Hugh’s analysis of his own scholarship. It conveys his ability to self-mock. In early 2005 I had posed a question on Robby Garner’s Yahoo Robitron group message board: Knowledge and a Doctorate: to be a doctor or not, that is the question.

This is how Hugh Loebner replied:

16 March 2005: "I pose the following question: Am I a fraudulent Ph.D.?

Affirmative: I submitted the dissertation for my Ph.D. knowing full well that the results were garbage. I published (jointly with my chairman) the results of my dissertation in the Journal Demography (top journal in the field) knowing full well that the results were garbage.

Negative: I did not alter the data. I performed my statistical calculations as I described. I explicitly explained and described why the results were garbage.

Why were the results garbage?

I reanalyzed some data on fertility in Central India which the chairman of my committee had collected and previously analyzed using cross tabulations and published as a book.

Rather than perform cross tabulations as my chairman had done, I used the method of 'path analysis', the 'hot' new statistical analysis in sociology/demography at the time.

Path analysis proceeds in three steps.

1. Take a piece of paper and mark dots on the paper representing variables of interest.

2. Based upon 'theory,' draw arrows from some dots to some other dots
representing causation

3. After the arrows are drawn (based upon 'theory') perform a multiple regression analysis to put numbers next to the arrows indicating relative strengths of the 'causes.'

Note Well: The choice of paths (arrows) is based upon "theory." The data do not reveal the causal network. The data reveal the strengths of the causes *given whatever particular path diagram is drawn.*

I asked the question (which apparently no one else has ever considered): given N variables, how many different possible path diagrams can be drawn?

Note that with N variable 1, 2, 3, 4 ... n, there are n*(n-1) = M ( 1 -> 2; 2 -> 1; 1 -> 3; 3 ->1; ... etc) possible arrows (causes), only some of which, presumably, are correct. We can have every path
diagram from the null case (nothing directly causes anything else eg no arrows on the page) to the complete case, (every variable causes every other variable eg every possible arrow on the page).

Given M objects, how many different subsets can be drawn. The answer is "the Power Set" = 2^m .

The answer to the question "How many path diagrams are possible given M arrows?" is 2^M. M, remember, is the number of arrows, not variables.

My analysis had 22 variables(dots). This means that the number of possible causes (arrows) was 22*21=462 possible arrows. The number of possible path diagrams for my set of variables is 2^462, which is a very large number. Only one path diagram can be correct, therefore, as a first approximation, the p that my path diagram was correct is 1/(2^462) I pointed this out on page 3 of my dissertation and column 3 of the article, and then said "with this limitation in mind, lets analyze the results."

Ergo, the results were garbage.

Am I a fraud? My results were garbage, but I said it first. If anyone critiques my dissertation (or article) I can reply: "Yes, of course, but I already said that." ;-) Hugh

And that’s how I will remember Hugh-the-original :)

References

[1] Elaine Loebner on Twitter: https://twitter.com/eloebner

[2] Westworld 2016: http://www.hbo.com/westworld

[3] JJ Abrams at Westworld premiere: https://www.youtube.com/watch?v=jJQOXPbmjd8

[4] Westworld 1973: http://www.imdb.com/title/tt0070909/?ref_=nv_sr_2

[5] Artificial Stupidity, Salon, February 23, 2003: http://www.salon.com/2003/02/26/loebner_part_one/

[6] Home of the Loebner Prize: http://www.loebner.net/Prizef/loebner-prize.html and

1995 Loebner Prize Announcement: http://loebner.net/Prizef/minsky.txt

[7] From the Buzzing in Turing’s Head to Machine Intelligence Contests: https://www.academia.edu/226311/From_the_Buzzing_in_Turings_Head_to_Machine_Intelligence_Contests

[8] Hugh Loebner: In Response to Stuart Shieber: http://loebner.net/Prizef/In-response.html

[9] 2006 Loebner Prize: http://loebner.net/Prizef/2006_Contest/loebner-prize-2006.html

Blog with pictures: http://humashah.blogspot.co.uk/2006/09/loebner-2006-contest-day.html

[10] 2008 Loebner Prize:

http://loebner.net/Prizef/2008_Contest/loebner-prize-2008.html and

Reading University page:

https://www.reading.ac.uk/15/research/ResearchReviewonline/featuresnews/res-featureloebner.aspx

[11]. Deception-detection and machine intelligence in Practical Turing tests. https://www.academia.edu/415888/Deception-detection_and_machine_intelligence_in_practical_Turing_tests

[12] Turing100in2012 at Bletchley Park:

http://www.reading.ac.uk/news-and-events/releases/PR445524.aspx

[13] Turing2014 at The Royal Society, London: http://turingtestsin2014.blogspot.co.uk/

[14] RoboLaw: http://www.robolaw.eu/