Confederate effect

The confederate effect is the phenomenon of people falsely classifying human intelligence as machine (or artificial) intelligence during Turing tests. For example, in the Loebner Prize during which a tester conducts a text exchange with one human and one artificial-intelligence chatbot and is tasked to identify which is which, the confederate effect describes the tester inaccurately identifying the human as the machine.^[1]

The confederate effect is the reverse of the ELIZA effect, which Sherry Turkle states is humans' "more general tendency to treat responsive computer programs as more intelligent than they really are":^[2] that is, anthropomorphizing.

The phenomenon was seen in the University of Surrey 2003 Loebner Prize for Artificial Intelligence, when both confederate (tested) humans, one male and one female, were each ranked as machine by at least one judge. More precisely, Judge 7 and Judge 9 ranked the female 'Confederate 2' as "1.00=definitely a machine"; the male 'Confederate 1' was ranked "1.00=definitely a machine" by Judge 4 and Judge 9.^[3] Also, the gender of these two hidden-humans were incorrectly identified (male considered female; woman considered man) in independent transcript analysis ('gender-blurring' phenomenon, see Shah & Henry, 2005).^[1]