Students who enlist the help of ChatGPT to write essays can be outed by a new analytical method developed in the UK to help educators identify cheats.
Since ChatGPT emerged publicly in November 2022, the concern of many educators has been that students will use it to write their essays, and that this will lead to reduced literary and critical-thinking skills among students.
Professor Ken Hyland, based at the University of East Anglia’s School of Education and Lifelong Learning in Norwich, set up a study to address such concerns, working alongside Professor Kevin Jiang of Jilin University in China.
Up to now, Prof Hyland said, no one has developed an effective way to identify essays that have been written by AI.
“Although tools such as GPTZero and AICheatCheck have been developed to detect AI involvement in writing, these seem currently unable to make a reliable distinction,” he said.
The researchers compared essays written by 145 students with those written by ChatGPT and the findings are published this week in the journal Written Communication.
“We found that the essays written by the real students consistently featured a rich array of engagement strategies, making them more interactive and persuasive,” Prof Hyland said. “They were full of rhetorical questions, personal asides and direct appeals to the reader – all techniques that enhance clarity, connection and produce a strong argument.”
The essays produced by ChatGPT, while grammatically correct and coherent, were impersonal, Prof Hyland said.
The AI essays attempted to reproduce the academic writing style, but were sorely lacking in the personal touch.
“They tended to avoid questions and limited personal commentary,” Prof Hyland said. “Overall, they were less engaging, less persuasive and there was no strong perspective on a topic.”
The use of ChatGPT and other AI algorithms by students at second and third level has become a challenge for educators. There is recognition that AI tools can enhance student learning, but not when used to write their essays.
Commenting on the new research methods to detect the use of ChatGPT in essay writing, a Department of Education spokesperson said: “AI is a rapidly evolving technology, and while relevant research is useful and informative, the nature of the AI is such that the technology will continue to develop and adapt, so there can be no real certainty that such detection mechanisms can future-proof the potential challenges posed by generative AI.”
Moira Leydon, assistant general secretary of the Association of Secondary Teachers in Ireland (ASTI), said it is up to the Government to outline the way forward for the use of AI in education.
“Leadership in this field must come from the Department of Education, who have already committed to several research and policy-formation exercises around AI in education,” Ms Leydon said.
I think it is imperative to build collaborative writing skills with students where AI tools are used effectively and responsibly
“This work must proceed expeditiously to assure teachers and school leaders of how best to harness evolving technology to support authentic learning and skill development.”
At third level, Prof Jim O’Mahony of Munster Technological University (MTU) is part of an MTU group assessing the use of generative AI like ChatGPT.
He said the “applicability” of the new East Anglia research was “questionable”, given it was focused on a group of “niche essays”. In addition, he said, the study only looked at ChatGPT, which is just one of the many AI language models that students might choose.
Further, Prof O’Mahony said an approach that emphasised the detection of cheating was “a flawed strategy”.
“I think it is imperative to build collaborative writing skills with students where AI tools are used effectively and responsibly rather than overdeveloping strategies that detect cheating,” he added.
Universities in Ireland and around the world are assessing how they might respond to the emergence of ChatGPT and other AI tools.
Certainly, it appears a distinction is being made between where a student uses AI to simply copy material or as part of a creative process.
“There are moral issues concerning plagiarism and universities should perhaps reconsider their current assessment procedures,” Prof Hyland said.
“Are essays written outside of class still the best way to assess students’ academic and writing skills, for example?”
Dr Christina O’Connor, of University of Limerick’s Department of Management and Marketing, has written on the topic of student ChatGPT use.
Dr O’Connor was the winner of a university-wide award for teaching two years ago, and teaches student classes of up to 500 at a time.
She said it would be useful to have a tool that could reliably detect if essays were generated by AI, and to what extent.
However, she added, the focus should be on embedding AI in new ways of learning and assessing rather than trying to catch the cheats.
“We are doing a disservice if we say to our students you can’t use ChatGPT here, and then we expect that they go into the workforce and that they are savvy with all tools,” she said.
Science Spinning 