Rules for Loebner Prize 2007
1. Important Dates:
The 2007 Competition is scheduled for Sunday, 21 October 2007 in New York City.
The date and venue are subject to change but NO changes to the
date and venue will be made after
The winner will receive USD 2250 and the Annual Medal.
The three runners up will receive USD 250.
2. Submission of Entries
Entrants should submit their entries on CD, DVD, portable hard drives or USB Flash via a shipment medium (a) requiring a receipt signature and (b) having a time/date stamp (E.g. Certified, Registered, FedEx, UPS, etc)
To: Loebner Prize Contest
c/o Crown Industries, Inc.
3. Installation of programs for selection of Final Four:
a. It is the intention of these rules to simplify contest management’s task in installing programs for testing.
b. Programs will be installed either by contest management or by entrants themselves.
c. Contest management will load and test
programs which meet the following installation criteria.
i.
Contest management will provide Intel based hardware running
Windows XP service pack 2.
ii.
Contest management will not install any program which requires
that management key in path names. All directories, files, etc, must be
indicated via normal OS “Browse” functions.
i.
Contest management will not install any program which requires
that management modify system variables (although these may be modified by a
supplied installer)
ii. Contest management will not install any program which must be run from a command line.
iii. No entry will be considered which does not provide all necessary program components, interpreters, etc (e.g. Perl, MySQL, etc).
iv. Entries must be accompanied by a “read me” file having the name, affiliation (if any), telephone number and postal and email addresses of the submitters as well as installation instructions.
d. Contestants who wish to submit a program whose installation procedure is at variance with the above are free to do so. In order to do so they must adhere to the following procedures. They may also provide hardware running alternative operating systems.
i. Submit their entry as described under rule 2.
ii. Accompany their submission with a printed hard copy letter clearly stating their desire to install their own program on site. They should also include possible schedules dates and communications channels..
iii. Note that the contestant will be installing an entry that was previously sent in accordance to rule 2.
e. Entrants under 18 years of age must have a letter of permission signed by at least one parent or guardian.
f. No person shall be affiliated with more than one entry.
g. Only the first 16 compliant entries will be evaluated in depth. All entries will be test for compliance with the rules. After non-complying entries are eliminated, the 16 compliant entries having the earliest mailing time stamps will be screened according to the criteria in point 4, below.
h. Notwithstanding the method of installation of the entry (installed by contest management or by the contestant himself or herself) the program and components (such as data files, etc) MUST run within a movable “Program Home Directory” (which will be created by contest management) having a name indicating submitter and program E.g. John_Smith_Program_Whosis. Contestants may suggest or supply the name for the folder.
4. The communications protocol used in the 2006 will remain unchanged.
a. Each Entry Program must communicate with a perl "Judge Communications" program via the creation, recognition, and deletion of sub-directories within a communications folder within the “Program Home Directory.”
b. The current version of the judge program is available at: http://loebner.net/Prizef/JudgeCommunications.txt
i. The name of the file should be changed to have a .pl extension so that the perl interpreter can recognize this as a perl script.
ii. Perl can be downloaded from activestate.com
c. Extensive discussions were held in the "robitron" group of groups.yahoo.com on the communications protocol. Parties interested in the Loebner Prize are advised to join the group. Many questions can be answered by perusing past messages, or posting questions.
5. In 2007 the Judges will be required to start the conversations. Entries will be expected to respond to the judges' initial comment or question. There will be no restrictions on what names etc the entries, humans, or judges can use, nor any other restrictions on the content of the conversations. Other rules remain substantially the same as in the 2006 competition:
6. Final Four entrants will each be awarded a USD 250 stipend. The winner of the Final Four round will receive the Competition Cash prize of USD 2000, making a total cash award of USD 2250 to the winner (unless there is only one compliant entry, in which case the cash award will be USD 2000.
7. Final Four entrants or their representatives must be present at the competition.
8. Judges will have 25 minutes to interact with each pair of entities.
9. Before scoring the entries, judges will have 20 minutes after completion of their interactions with the programs to review the interactions.
10. If there is no compliant Entry for the 2007 Competition, the $2000 prize will be added to the 2008 Competition prize making the 2008 prize $4000, and the 2008 Competition will be held under these rules.
11. If there is only one Entry compliant with these rules the submitter(s) of that Entry will receive $2000 and the "Annual Medal." If there are two or more Entries compliant with these rules a Competition will be held and the submitter(s) of the winning Computer Entry will receive $2000 and the "Annual Medal."
12. Contest management reserves the right to enter one or more publicly available open source programs.
13. SELECTION OF FINAL FOUR: Contest management herewith announces 4 sets of screening questions, upon which next year's finalists will be selected.
a. It is NOT a condition of entry to the contest that a program be able to respond "intelligently" to any of these questions.
b. However, the finalists chosen will be those best able to respond "intelligently." The 4 entries with the highest scores will be selected as finalists.
c. If no entries can respond "intelligently" to these questions selection of the finalists will be on the general quality of responses.
d. Contest management will not ask about rare or unusual things. All nouns, adjectives and verbs will come from a dictionary suitable for children or adolescents under the age of 12.
Set 1 - Questions relating to time:
Background facts: For testing purposes, contest management will consider these to be correct whether or not the time and venue of the contest have been changed.
a. The system clock will be accurate to within a minute or two.
b. The competition is scheduled to start at
c. There will be 7 rounds of 30 minutes each.
Sample Questions
Etc.
Set 2 - General questions about things.
Sample Questions:
Etc.
Set 3 Questions relating to comparisons
Sample Questions
Etc:
Set 4 - Questions demonstrating "memory" or persistence.
Sample Questions
Etc.