Loebner Prize Contest 2003 'Can
machines think?' - Alan Turing, 1950
Home : : The Contest
: : Rules : : Times & Places
: : Links : : Contact Us
Results
Methodology
Each of the judges (numbered 1 to 9) conversed with each of the ten
terminals (labelled A to J) in a randomised order. They gave each
terminal a "humanness" score, on the following scale:
Score
Was your conversational partner a human or a machine?
0
Partner not accessible, or severe system malfunction
1
Definitely a machine
2
Probably a machine
3
Could be a machine or a human; undecided
4
Probably a human
5
Definitely a human
Fractional values were permitted. After every judge had conversed with
every terminal, their score sheets were collected and their scores were
entered into a spreadsheet, which calculated the mean humanness score
for each terminal.
Final Results
The final results, ranked in decreasing mean humanness, are as follows:
Rank
Entity
Contestant
T
J1
J2
J3
J4
J5
J6
J7
J8
J9
Mean
1
Confederate 2
H
4.50
5.00
4.80
3.50
5.00
5.00
1.00
5.00
1.00
3.867
2
Confederate 1
D
5.00
2.00
4.10
1.00
5.00
4.90
5.00
5.00
1.00
3.667
3
Jabberwock
Juergen Pirner
C
1.00
1.00
1.25
4.00
1.20
2.00
3.00
2.90
1.00
1.928
4
Elbot
Fred Roberts
J
1.00
1.00
3.50
1.00
1.50
1.50
2.00
2.60
1.00
1.678
5 =
Eugene Goostmann
Vladimir Veselov and
Eugene Demchenko
A
1.10
1.00
2.20
1.00
1.00
2.50
2.00
3.00
1.00
1.644
5 =
Jabberwacky
Rollo Carpenter
F
1.00
2.00
1.45
1.00
1.30
2.25
2.00
2.80
1.00
1.644
7
Lucy
Saskia van der Elst
I
1.00
1.00
1.10
1.00
1.10
1.50
3.00
1.70
1.00
1.378
8
Markbot
Mark Connell
B
1.00
1.00
1.50
1.00
1.00
1.70
1.00
1.60
2.00
1.311
9
ALICE
Richard Wallace
E
1.00
1.00
1.70
1.00
1.00
2.00
1.00
1.90
1.00
1.289
10
Gabber
Peter Neuendorffer
G
1.00
2.00
1.10
1.00
1.10
1.00
1.00
1.50
1.00
1.189
As both human confederates outranked the computer entries, the Silver
Award was not presented.
The Bronze Award went to Jabberwock, the "most human bot" according to
the judges.
Home : : The Contest
: : Rules : : Times & Places
: : Links : : Contact Us