Ashok Goel, a computer science professor at Georgie Institute of Technology, trained a system using IBM Watson technology to behave as a teaching assistant in an artificial intelligence course. The system, named Jill Watson, answered questions, reminded students of deadlines and, generally, provided feedback to the students by email. It was, so to speak, a robotic teaching assistant.
Jill was trained using nearly 40,000 postings available on a discussion forum, and was configured to answer only when the level of confidence was very high, thus avoiding weak answers that would give “her” away. In March, she went online, began posting responses live.
As the Wall Street journal reports, none of the students seemed to notice, and some of them were “flabbergasted” when they were told about the experiment. Some, however, may have harboured doubts, since Jill replied so quickly to the questions posed by the students.
Even though this falls way short of a full-fledged Turing test, it raises significant questions about how effective can AI agents be in replacing professors and teaching assistants, in the task of providing feedback to students. Next year, Ashok Goel plans to tell his students one of the TAs is a computer, but not which one. Like with the Cylons, you know. What could possibly go wrong?