© Huma Shah, December 11, 2016

Hugh Loebner from 2006 Prize, UCL |

Hugh Loebner’s ex-wife’s message didn’t sink in. That day,
like many, I was in anticipation of

**Westworld TV**show’s Season 1 finale [2] promised by Sky Atlantic to air at the same ‘time’ (2am UK 5 Dec) as for viewers in the US (9pm Dec 4), the mesmerising “*” [3] robot epic based on the 1973 movie starring***dawning of consciousness****Yul Brynner**as the refusing-to-die killer robot [4].
“All
Hugh Loebner wanted to do was become world famous, eliminate all human toil,
and get laid a lot. And he was willing to put up lots of good money to do so.
He’s a generous, fun-loving soul who likes to laugh, especially at himself. So
why does everybody dislike him so much? Why does everybody give him such a hard
time?”

I don’t think this is the time to go over his human
failings, because we all have them, what I do want is to remind the academic
world and beyond how approachable Hugh Loebner was. And yes he did have a sense
of humour, read his cheeky retort claiming that the (now late)

**Marvin Minsky**supported the**Loebner Prize**[6]. But more than that, Hugh had a profound understanding of Turing’s idea about how to build and measure whether a machine could think [7] - see his**response to Stuart Shieber**’s article*Lessons from a Restricted Turing test*[8].
I’m loathe to divulge personal experiences of interactions
with Hugh Loebner, and there were many. I first saw him in

**2003 at the 13**Then from 2005 discussions began for me to organise his 16^{th}Loebner Prize contest at the University of Surrey.^{th}contest. I co-organised two Loebner Prizes: at UCL in 2006 [9], and at Reading University in 2008 [10]. Hugh allowed me to design his 18^{th}contest as an original Turing test experiment different from all his previous Prizes– see Chapter 5 in my PhD thesis [11]. I can say as late as 2014, long after I’d gone my own way designing my own Turing test events, at**Bletchley Park**[12] on the 100^{th}anniversary of Alan Turing’s birth June 23^{rd}, 2012, and another on the 60^{th}anniversary of Turing's passing away, at**The Royal Society London**[13], 6-7 June, 2014, Hugh remained approachable. He responded swiftly to my queries on the pass-rate in his 2014 Loebner Prize (I was concerned and have many emails making a plea not to use the ‘unsure’ score in my own 2014 Turing tests as part of a**RoboLaw**project dissemination event, and the media backlash following prove me right, but that’s another story).
Anyway, Hugh clarified that in his 2014 contest “The program must fool half the judges.
That automatically means it will be compared to two different humans. Each
judge meets each human once.” (Personal email to me: 18 May 2014 at 16:26).

Back to Hugh’s own words, here’s Hugh’s analysis of
his own scholarship. It conveys his ability to self-mock. In early 2005 I had posed a question on Robby Garner’s Yahoo Robitron group message
board:

*Knowledge and a**Doctorate: to be a doctor or not, that is the question*.
This is how Hugh Loebner replied:

16
March 2005: "

Affirmative: I submitted the dissertation for my Ph.D. knowing full well that the results were garbage. I published (jointly with my chairman) the results of my dissertation in the Journal Demography (top journal in the field) knowing full well that the results were garbage.

Negative: I did not alter the data. I performed my statistical calculations as I described. I explicitly explained and described why the results were garbage.

Why were the results garbage?

I reanalyzed some data on fertility in Central India which the chairman of my committee had collected and previously analyzed using cross tabulations and published as a book.

Rather than perform cross tabulations as my chairman had done, I used the method of 'path analysis', the 'hot' new statistical analysis in sociology/demography at the time.

Path analysis proceeds in three steps.

**I pose the following question: Am I a fraudulent Ph.D**.?Affirmative: I submitted the dissertation for my Ph.D. knowing full well that the results were garbage. I published (jointly with my chairman) the results of my dissertation in the Journal Demography (top journal in the field) knowing full well that the results were garbage.

Negative: I did not alter the data. I performed my statistical calculations as I described. I explicitly explained and described why the results were garbage.

Why were the results garbage?

I reanalyzed some data on fertility in Central India which the chairman of my committee had collected and previously analyzed using cross tabulations and published as a book.

Rather than perform cross tabulations as my chairman had done, I used the method of 'path analysis', the 'hot' new statistical analysis in sociology/demography at the time.

Path analysis proceeds in three steps.

1.
Take a piece of paper and mark dots on the paper representing variables of
interest.

2.
Based upon 'theory,' draw arrows from
some dots to some other dots

representing causation

representing causation

3.
After the arrows are drawn (based upon 'theory')
perform a multiple regression analysis to put numbers next to the arrows indicating relative strengths of the
'causes.'

Note Well: The choice of paths (arrows) is based upon "theory." The data do not reveal the causal network. The data reveal the strengths of the causes *given whatever particular path diagram is drawn.*

I asked the question (which apparently no one else has ever considered): given N variables, how many different possible path diagrams can be drawn?

Note Well: The choice of paths (arrows) is based upon "theory." The data do not reveal the causal network. The data reveal the strengths of the causes *given whatever particular path diagram is drawn.*

I asked the question (which apparently no one else has ever considered): given N variables, how many different possible path diagrams can be drawn?

Note
that with N variable 1, 2, 3, 4 ... n, there are n*(n-1) = M ( 1 -> 2; 2 -> 1; 1 -> 3; 3 ->1; ...
etc) possible arrows (causes), only some of which, presumably, are
correct. We can have every path

diagram from the null case (nothing directly causes anything else eg no arrows on the page) to the complete case, (every variable causes every other variable eg every possible arrow on the page).

Given M objects, how many different subsets can be drawn. The answer is "the Power Set" = 2^m .

The answer to the question "How many path diagrams are possible given M arrows?" is 2^M. M, remember, is the number of arrows, not variables.

My analysis had 22 variables(dots). This means that the number of possible causes (arrows) was 22*21=462 possible arrows. The number of possible path diagrams for my set of variables is 2^462, which is a very large number. Only one path diagram can be correct, therefore, as a first approximation, the p that my path diagram was correct is 1/(2^462) I pointed this out on page 3 of my dissertation and column 3 of the article, and then said "with this limitation in mind, lets analyze the results."

Ergo, the results were garbage.

Am I a fraud? My results were garbage, but I said it first. If anyone critiques my dissertation (or article) I can reply: "Yes, of course, but I already said that." ;-) Hugh

diagram from the null case (nothing directly causes anything else eg no arrows on the page) to the complete case, (every variable causes every other variable eg every possible arrow on the page).

Given M objects, how many different subsets can be drawn. The answer is "the Power Set" = 2^m .

The answer to the question "How many path diagrams are possible given M arrows?" is 2^M. M, remember, is the number of arrows, not variables.

My analysis had 22 variables(dots). This means that the number of possible causes (arrows) was 22*21=462 possible arrows. The number of possible path diagrams for my set of variables is 2^462, which is a very large number. Only one path diagram can be correct, therefore, as a first approximation, the p that my path diagram was correct is 1/(2^462) I pointed this out on page 3 of my dissertation and column 3 of the article, and then said "with this limitation in mind, lets analyze the results."

Ergo, the results were garbage.

Am I a fraud? My results were garbage, but I said it first. If anyone critiques my dissertation (or article) I can reply: "Yes, of course, but I already said that." ;-) Hugh

And that’s how I will remember

*Hugh-the-original*:)
© Huma Shah, December 11, 2016

**References**

[1] Elaine Loebner on Twitter: https://twitter.com/eloebner

[2] Westworld 2016: http://www.hbo.com/westworld

[3] JJ Abrams at Westworld premiere: https://www.youtube.com/watch?v=jJQOXPbmjd8

[4] Westworld 1973: http://www.imdb.com/title/tt0070909/?ref_=nv_sr_2

[5] Artificial Stupidity, Salon, February 23, 2003: http://www.salon.com/2003/02/26/loebner_part_one/

[6] Home of the Loebner Prize: http://www.loebner.net/Prizef/loebner-prize.html and

1995 Loebner Prize Announcement: http://loebner.net/Prizef/minsky.txt

[7]

*From the Buzzing in Turing’s Head to Machine Intelligence Contests*: https://www.academia.edu/226311/From_the_Buzzing_in_Turings_Head_to_Machine_Intelligence_Contests
[8] Hugh Loebner: In Response to Stuart Shieber: http://loebner.net/Prizef/In-response.html

[9] 2006 Loebner Prize: http://loebner.net/Prizef/2006_Contest/loebner-prize-2006.html

Blog with pictures: http://humashah.blogspot.co.uk/2006/09/loebner-2006-contest-day.html

[10] 2008 Loebner Prize:

Reading University page:

https://www.reading.ac.uk/15/research/ResearchReviewonline/featuresnews/res-featureloebner.aspx

[11]. Deception-detection and machine intelligence in
Practical Turing tests. https://www.academia.edu/415888/Deception-detection_and_machine_intelligence_in_practical_Turing_tests

[12] Turing100in2012 at Bletchley Park:

[13] Turing2014 at The Royal Society, London: http://turingtestsin2014.blogspot.co.uk/

[14] RoboLaw: http://www.robolaw.eu/

## No comments:

Post a Comment