Essay-Grading Software Seen as Time-Saving Tool

Teachers are turning to software that is essay-grading critique student writing, but critics point to serious flaws into the technology

Jeff Pence knows the easiest way for his 7th grade English students to enhance their writing would be to do a lot more of it. But with 140 students, it can take him at the very least two weeks to grade a batch of their essays.

Therefore the Canton, Ga., middle school teacher uses an internet, automated essay-scoring program which allows students to have feedback to their writing before handing in their work.

"It doesn't tell them what to do, but it points out where issues may exist," said Mr. Pence, who says the a Pearson WriteToLearn program engages the students almost like a game title.

A week and individualize instruction efficiently with the technology, he has been able to assign an essay. "I feel it's pretty accurate," Mr. Pence said. "will it be perfect? No. However when I reach that 67th essay, i am not accurate that is real either. As a united team, we have been very good."

Aided by the push for students to become better writers and meet with the new Common Core State Standards, teachers are eager for new tools to aid out. Pearson, which is located in London and New York City, is regarded as several companies upgrading its technology in this space, also referred to as artificial intelligence, AI, or machine-reading. New assessments to evaluate deeper learning and move beyond multiple-choice answers are also fueling the demand for software to help automate the scoring of open-ended questions.

Critics contend the application doesn't do alot more than count words and therefore can not replace readers that are human so researchers will work difficult to improve the software algorithms and counter the naysayers.

Although the technology has been developed primarily by companies in proprietary settings, there's been a focus that is new improving it through open-source platforms. New players on the market, such as the startup venture LightSide and edX, the enterprise that is nonprofit by Harvard University while the Massachusetts Institute of Technology, are openly sharing their research. Last year, the William and Flora Hewlett Foundation sponsored an open-source competition to spur innovation in automated writing assessments that attracted commercial vendors and teams of scientists from around the planet. (The Hewlett Foundation supports coverage of "deeper learning" issues in Education Week.)

"we have been seeing plenty of collaboration among competitors and individuals," said Michelle Barrett, the director of research systems and analysis for CTB/McGraw-Hill, which produces the Roadmap that is writing for in grades 3-12. "this collaboration that is unprecedented encouraging a lot of discussion and transparency."

Mark D. Shermis, an education professor at the University of Akron, in Ohio, who supervised the Hewlett contest, said the meeting of top public and commercial researchers, along side input from many different fields, may help boost performance for the technology. The recommendation through the Hewlett trials is that the automated software be used as a "second reader" to monitor the human readers' performance or provide additional information about writing, Mr. Shermis said.

"The technology can't do everything, and nobody is claiming it can," he said. "But it really is a technology that features a promising future."

The initial essay-scoring that is automated get back to the early 1970s, but there isn't much progress made through to the 1990s with the advent regarding the Internet as well as the ability to store data on hard-disk drives, Mr. Shermis said. More recently, improvements have been made in the technology's ability to evaluate language, grammar, mechanics, and style; detect plagiarism; and supply quantitative and qualitative feedback.

The computer programs assign grades to writing samples, sometimes on a scale of 1 to 6, in a number of areas, from word choice to organization. The merchandise give feedback to help students improve their writing. Others can grade answers that are short content. The technology can be used in paper writer various ways on formative exercises or summative tests to save time and money.

The Educational Testing Service first used its e-rater automated-scoring engine for a high-stakes exam in 1999 when it comes to Graduate Management Admission Test, or GMAT, relating to David Williamson, a senior research director for assessment innovation for the Princeton, N.J.-based company. It uses the technology with its Criterion Online Writing Evaluation Service for grades 4-12.

The capabilities changed substantially, evolving from simple rule-based coding to more sophisticated software systems over the years. And statistical techniques from computational linguists, natural language processing, and machine learning have helped develop better methods of identifying certain patterns in writing.

But challenges remain in picking out a definition that is universal of writing, as well as in training a computer to understand nuances such as for example "voice."

Over time, with larger sets of data, more experts can identify nuanced aspects of writing and increase the technology, said Mr. Williamson, who is encouraged by the era that is new of concerning the research.

"It is a hot topic," he said. "there is a large number of researchers and academia and industry looking into this, and that's a good thing."

High-Stakes Testing

As well as utilising the technology to improve writing in the classroom, West Virginia employs software that is automated its statewide annual reading language arts assessments for grades 3-11. Their state has worked with CTB/McGraw-Hill to customize its product and train the engine, using a huge number of papers it has collected, to score the students' writing based on a prompt that is specific.

"we have been confident the scoring is very accurate," said Sandra Foster, the lead coordinator of assessment and accountability into the West Virginia education office, who acknowledged facing skepticism initially from teachers. But the majority of were won over, she said, after a comparability study indicated that the accuracy of a trained teacher and the scoring engine performed much better than two trained teachers. Training involved a hours that are few simple tips to measure the writing rubric. Plus, writing scores have gone up since implementing the technology.

Automated essay scoring is also applied to the ACT Compass exams for community college placement, the brand new Pearson General Educational Development tests for a school that is high diploma, and other summative tests. But it has not yet been embraced by the College Board for the SAT or perhaps the ACT that is rival college-entrance.

The 2 consortia delivering the assessments that are new the typical Core State Standards are reviewing machine-grading but never have devoted to it.

Jeffrey Nellhaus, the director of policy, research, and design when it comes to Partnership for Assessment of Readiness for College and Careers, or PARCC, wants to know if the technology is likely to be a fit that is good its assessment, and the consortium should be conducting a research predicated on writing from its first field test to see how the scoring engine performs.

Likewise, Tony Alpert, the chief operating officer for the Smarter Balanced Assessment Consortium, said his consortium will evaluate the technology carefully.

Together with his new company LightSide, in Pittsburgh, owner Elijah Mayfield said his data-driven method of writing that is automated sets itself apart from other products in the marketplace.

"that which we are attempting to do is build a system that instead of correcting errors, finds the strongest and weakest sections of the writing and the best place to improve," he said. "It is acting more as a revisionist than a textbook."

The software that is new which is available on an open-source platform, has been piloted this spring in districts in Pennsylvania and New York.

In higher education, edX has just introduced software that is automated grade open-response questions for usage by teachers and professors through its free online courses. "One of this challenges in past times was that the code and algorithms were not public. They certainly were viewed as black magic," said company President Anant Argawal, noting the technology is within an experimental stage. "With edX, we place the code into open source where you could observe how it is done to aid us improve it."

Still, critics of essay-grading software, such as for instance Les Perelman, want academic researchers to have broader usage of vendors' products to judge their merit. Now retired, the previous director regarding the MIT Writing over the Curriculum program has studied some of the devices and was able to get a score that is high one with an essay of gibberish.

"My principal interest is he said that it doesn't work. Whilst the technology has some limited use with grading short answers for content, it relies an excessive amount of on counting words and reading an essay requires a deeper level of analysis best done by a person, contended Mr. Perelman.