Note to Editors: To observe sixth graders using the software, contact teacher Cindy Matthews at Platt Middle School in Boulder, (303) 499-6800.
Computer software invented at the University of Colorado at Boulder that grades the content of essay exams has also improved the writing of local sixth graders and helped launch a new company.
The Intelligent Essay Assessor allowed sixth grade students at Platt Middle School in Boulder to improve their reading and writing as part of a computer tutor called Summary Street, teachers and researchers said.
Writing and revising summaries is an effective way to learn and would be used by teachers more often if it weren't so time-consuming to comment on the writing. So a computer program was designed to help.
In a two-year study at the school, more than 100 students used Summary Street to tell them how they had done in writing summaries, to point out omitted concepts, redundancies and irrelevancies, and to let them try again. Teachers then rated summaries written by the students without knowing who had used Summary Street and who hadn't.
The scores of students who used the program were significantly better than those of students who did not, researchers said. Â鶹ÒùÔº using Summary Street also spent much more time working to improve their summaries.
"We're as happy with these outcomes as we are with the excellent results of the Essay Assessor for testing," said psychology Professor Thomas Landauer, one of three co-inventors.
The Intelligent Essay Assessor received national attention when it was announced in April 1998. The system was developed by Landauer, CU-Boulder doctoral student Darrell Laham and Peter W. Foltz, now an assistant professor of psychology at New Mexico State University.
The three researchers received a patent on their software in October and have launched a new Boulder-based company, Knowledge Analysis Technologies, with Landauer serving as president. The company holds an exclusive license from CU, and the university will receive a portion of any profits.
In addition to the co-founders, the company employs the equivalent of six full-time workers, all of whom are CU-Boulder graduates.
"Knowledge Analysis Technologies is a delightful example of a University of Colorado spin-off company based on CU technologies that embody all three components of our mission: research, teaching and service," said Stephen ONeil, director of the CU-Boulder Office of Technology Transfer and Industry Outreach. "As the company matures and expands its market, the benefits of those technologies will be made increasingly available to the local and national public -- an excellent example of technology transfer."
Publisher Prentice-Hall uses the Intelligent Essay Assessor in a Web site companion to its popular freshman orientation book, "Keys to Success." Â鶹ÒùÔº visiting the site enter answers in a Web page box. The computer then tells them if their answers were good or not in terms of understanding and summarizing what the book had to say on the topic.
The software also tells them which pages in the book contain information that would make their answers more complete. When the students are satisfied, they can email their essays and the Assessor's comments to their teachers.
The Internet-based company has contracted with several schools and companies, including the University of South Africa, to provide essay-grading and commenting services. All student writing assignments are sent via the Internet to the company's offices in the CU Research Park. The company then uses its own computers to evaluate the texts and returns the results over the Internet.
The Intelligent Essay Assessor works by "reading" an on-line textbook to learn the meaning of words and concepts used in a subject. Then it simulates the way teachers compare the conceptual content of one essay with another, or with the source from which the student learned.
The software has been tested on more than 3,000 essays on more than 15 diverse topics, from biology to history to business. For each essay, researchers had both the Assessor and at least two independent teachers or other experts give scores. On average, the Assessor agreed with the human experts as much as the experts agreed with each other.
"The kids wrote significantly better essays with the help of Summary Street," said Platt Middle School teacher Cindy Matthews, who uses the program with fellow sixth grade teacher Ron Lamb.
"They definitely enjoy it," Matthews said. "They look forward to working on it and will spend longer on their writing because of it."
"It is easy to use," said sixth grader Evan Sharp, 11. "If (the summary) has missing parts it will ask you to work on that."
The Summary Street software was developed by David Steinhart as part of his doctoral thesis. The project is a cooperative effort between CU-Boulder researchers and the teachers at Platt, said Eileen Kintsch, CU research associate and project manager. The results of the school studies will be published in a special issue of the journal Interactive Learning Environments.
Other applications of the basic method for machine understanding of text are under research and development at CU-Boulder and by the company. They include computer-based leadership training, adult reading tutorship and non-educational uses for personnel selection and assignment. The CU-Boulder researchers are part of an interdisciplinary team of psychologists and computer scientists in the Institute of Cognitive Science.
To try a demonstration of the Intelligent Essay Assessor visit the Knowledge Analysis Technologies Web site at .