Nel: A Spoken Language Physics Tutor
My thesis consisted of designing and coding an Intelligent Tutoring System (ITS) that included a conversational speech interface and animated agent (“Nel”) to better emulate human interaction (that is, “Nel” spoke the problems to the students and they in turn spoke their answers). Nel was designed after working closely with Dr. Nels Madsen of Auburn University’s Physics Department (the inspiration and catalyst for the project). Ultimately the ITS attempted to provide an alternative means of gaining assistance with homework assignments when the professor or tutors were not available.
Dr. Madsen explained that problems are often solved by conquering a series of smaller steps. Thus, in the database (and throughout the interface) each problem was broken down into individual steps then each step had a question, answer, and optional hint.
Data Entry Form
The system was built to be subject independent (physics was just the primary area of Dr. Madsen and, thus, the initial focus). The system included a generic Web interface for the instructor to enter the problems and accompanying steps, following the database’s organization.
Tutoring System User Interface
The system components included the following:
- Haptek Player Agent – 3-D character created by Haptek, Inc.
- AT&T Natural Voices – natural-sounding text-to-speech
- Speech Application Language Tags – captured spoken input for Web interpretation
- XML Grammars – format for the “speech grammars” — the application’s vocabulary
Based on the system design of problems and sub-steps, the system can be looked at in 2 components: finding and retrieving the problem and traversing the steps.
The words entered in the Problem Statement of the data entry interface are used to create the main speech grammar for recognizing the student. The words from this text are used to create a “bag of words” and the strategy of “closest match” (between what the system heard vs. the statements in the database) is used to recognize and retrieve the problem the student wants to work on.
Once the student confirms the problem, they simply repeat the same interaction of answering the questions from each step (as entered by the instructor). This process continues until they have reached the final step/answer for the problem.