Home Page of The Loebner Prize in Artificial Intelligence

"The First Turing Test"

 

Loebner Prize Gold Medal
(Solid 18 carat, not gold-plated like the Olympic "Gold" medals)

What is the Loebner Prize?

The Loebner Prize for artificial intelligence ( AI ) is the first formal instantiation of a Turing Test. The test is named after Alan Turing the brilliant British mathematician. Among his many accomplishments was basic research in computing science. In 1950, in the article Computing Machinery and Intelligence which appeared in the philosophy journal Mind, Alan Turing asked the question "Can a Machine Think?" He answered in the affirmative, but a central question was: "If a computer could think, how could we tell?" Turing's suggestion was, that if the responses from the computer were indistinguishable from that of a human,the computer could be said to be thinking. This field is generally known as natural language processing.

In 1990 Hugh Loebner agreed with The Cambridge Center for Behavioral Studies to underwrite a contest designed to implement the Turing Test. Dr. Loebner pledged a Grand Prize of $100,000 and a Gold Medal (pictured above) for the first computer whose responses were indistinguishable from a human's. Such a computer can be said "to think." Each year an annual cash prize and a bronze medal is awarded to the most human-like computer. The winner of the annual contest is the best entry relative to other entries that year, irrespective of how good it is in an absolute sense.

Further information on the development of the Loebner Prize and the reasons for its existence is available in Loebner's article In Response to the article Lessons from a Restricted Turing Test by Stuart Shieber.

The Loebner Prize was originally made possible by funding from Crown Industries, Inc., of East Orange NJ.

For a comprehensive overview of chatbots in general, check chatbots.org


Information on 2013 Contest

    The Imagineering Quarter is hosting the international Loebner Prize 2013 Contest in Artificial Intelligence (AI) to find the world's best conversational chatbot computer program in collaboration with Universities in the USA (New Mexico), Denmark (Aalborg), France (LIMSI-CNRS), England (Exeter, Sheffield) & Ireland (The Queen's University of Belfast, UCD, DCU) and technically co-sponsored by the IEEE Computational Intelligence Society UKRI chapter. 
    The contest is being held during the CultureTECH 2013 digital media and creativity festival & City of Culture 2013 celebration.

    Each year an Annual Prize & Bronze Medal is awarded to the most human-like computer. 
    Loebner Prize 2013 Annual First Prize is: US$ 4000 + Annual Bronze Medal, Second Prize: US$ 1000, Third Prize: US$ 750 and Fourth Prize: US$ 250.

    The Silver Medal Prize of $25,000 + Silver Medal will be awarded if any program fools two or more judges when compared to two or more humans. At that point the contest will progress to the MultiModal stage in which entries in subsequent years will necessitate processing of MultiModal input (e.g. music, speech, pictures, videos). During the MultiModal stage, if any entry fools half the judges compared to half of the humans, the program's creator(s) will receive the Grand Prize of $100,000 + 18kt Gold Medal, and the competition will be discontinued.

    All prizes are sponsored by New York Philanthropist, Dr. Hugh Loebner.

    A Call for Entries has been announced (see below) and the 4 entries with the highest scores will be selected as finalists.


    Note to entrants:  Judges will interact with your program via the Judge Program, which is a Perl program running under ActiveState Perl 5.16.  In order to run this program:

      • Download the program
      • Save the program with the extension changed from .txt to .pl
      • Download and install ActiveState Perl 5.16 from this link
      • Create a "Communications" folder
        • Create Four sub-folders within the "Communications" folder named
          • "Webcast Left"
          • "Webcast Right"
          • "Human"
          • "Your Program's Name" eg "Mybot" or however you chose to name it.
      • Click on the Judge Program to start it
      • In the window which appears enter a "1" in the boxes at the top of the window which are labeled "Judge Number" and "Round Number"
      • Click on the green button at the right of the window which is labeled "New Round"
      • The Judge Program will request, via a browse function, folders for Webcast Left, Webcast Right, Human, and Your Program. Select them accordingly.
      • The Judge Program will indicate which side, left or right, interaction with your program will take place.  The judge will enter, character by character, his or her questions to your program in the entry boxes at the bottom of the window labeled either "My Left Entry" or "My Right Entry."

    Your program will interact with the Judge Program using the Loebner Prize Protocol "LPP" via the sub folder with your program's name which is nested within the Communications folder.  The other three folders are used during the competition, but are not necessary for testing. 


    Loebner Prize 2013, in collaboration with Foyle Learning Community and TeenTech CIC, will also include a separate prize known as the Junior Loebner Prize in which the judging panel will consist of school pupils between 12 and 14 years old. This prize first arose after a paper (see below) and discussion at AISB 2010 where Loebner 2011 (Exeter) contest organiser, Dr. Ed Keedwell, proposed a Turing Test based on child development. An additional prize fund is available for the machines in this Junior Loebner Prize Contest.

    The University of Exeter's Computer Science Department (Dr. Ed Keedwell, Kent McClymont) will run a live webcast of Loebner Prize 2013 and interested people from around the world will be able to follow the conversations the judges have as they happen.

    Loebner Prize 2013 is directed by Prof. Paul Mc Kevitt of The University of Ulster and produced by Dr. Hugh Loebner, together with Production and Location (drawn from The Imagineering Quarter) Crews and casting of Contest Entrants and an International Judging Panel. A cameo appearance by a BBC Media Celebrity who is an expert on natural language is anticipated.

    Loebner Prize 2013 will be filmed by WGBH (Boston, USA) & 360 Production (Derry/London) as part of a documentary on Artificial Intelligence (AI) for the Nova Primetime Science TV Series produced by Chad Cohen for screening on the USA Public Broadcasting Service (PBS) Television Network.

    The Loebner Prize is the first formal instantiation of `The Turing Test'. The test is named after Alan Turing, the brilliant British mathematician who also developed basic research on the theoretical foundation of computing science.

    Without wishing to complicate things, a not altogether unrelated spanner in the works is the simple matter of, ``The Chinese Room Problem'' (Prof. John Searle, Berkeley, USA).

    The Loebner Prize contest, first inaugurated in 1991 at The Computer Museum (Boston, USA), has been hosted internationally at locations such as: Carnegie Hall (NY, USA), The Science Museum (London), The Powerhouse Museum (Sydney, Australia), Bletchley Park (England), Dartmouth College (NH, USA), California State University (LA, USA), University College London, Surrey, Reading & Exeter Universities (England), Flinders University (Adelaide, Australia) and even Hugh Loebner's appartment in New York City (USA).


 


Winners of Previous Contests

1991 Joseph Weintraub , Thinking Systems Software

1992 Joseph Weintraub, Thinking Systems Software

1993 Joseph Weintraub, Thinking Systems Software

1994 Thomas Whalen

1995 Joseph Weintraub, Thinking Systems Software

1996 Jason Hutchens,  Agworld Pty Ltd

1997 David Levy, Intelligent Research Ltd.

1998 Robby Garner

1999 Robby Garner

2000 Richard Wallace (another link)

2001 Richard Wallace

2002 Kevin Copple

2003 Juergen Pirner

2004 Richard Wallace

2005 Rollo Carpenter

2006 Rollo Carpenter

2007 Robert Medeksza

2008 Fred Roberts and Artificial Solutions

 2009 David Levy

2010 Bruce Wilcox

2011 Bruce Wilcox

2012 Mohan Embar

2013 Stephen Worswick


1992 Contest Information and transcripts
(thanks to Ms. Jamilah Ogburn who scanned the pdf file )

Thom Whalen's Account of winning the 1994 contest
1994 Contest Information and transcripts are currently unavailable

1995 Contest Information and transcripts
Thom Whalen's Account of his experience at the 1995 contest  

1996 Contest Information and transcripts

1997 Contest Information and transcripts

1998 Contest Information and transcripts

1999 Contest Information and transcripts

2000 Contest Information and transcripts are currently unavailable

2001 Contest Information and transcripts

2002 Contest Information and transcripts

2003 Contest Information and transcripts

2004 Contest Information and transcripts

2005 Contest Information and transcripts

2006 Contest Information and transcripts

2007 Contest Information and transcripts

2008 Contest Information and transcripts

2009 Contest Information and transcripts

2010 Contest Information and transcripts

2011 Contest Information and transcripts

2012 Contest Information and transcripts


Marvin Minsky Co-sponsors the Contest! Read all about it!

(An amusing thread on the comp.ai news groups)

1 April 2013