In an era where artificial intelligence (AI) is becoming increasingly sophisticated, it has found its way into various aspects of our lives. One such area is education, particularly in essay writing. With advancements in AI technologies, there’s a growing concern about students potentially leveraging these tools to generate their essays.

While AI can be a powerful tool for learning and development, using it to produce original work raises significant ethical questions. It is crucial for educators to be able to distinguish between student-written and AI-generated essays to uphold academic integrity.  As educators each week we will see more students using Generative AI and attributing it as their own work. 

Detecting AI-Generated Text

AI has become surprisingly good at generating human-like text. However, there are still tell-tale signs that can help discern if an essay was composed by a student or an AI. Here are some methods educators can utilize:

1. Unusual Phrasing or Structure

AI language models have become highly advanced, but they still have a tendency to generate phrases that deviate from natural human language. These peculiarities can manifest as unusual expressions or sentences that are technically correct but sound awkward in their meaning. When reading an essay, if you come across sections that feel “off” or significantly diverge from the student’s typical writing style, it may indicate the use of AI assistance. Apart from unusual phrasing, AI-generated content often exhibits a consistent structural pattern. For instance, paragraphs tend to have a similar length, typically consisting of around four to five lines. This consistent paragraph length can serve as another indicator of AI involvement in the writing process. By paying attention to both the phrasing and structural aspects of the text, you can better assess whether AI tools were employed to compose the document.

2. Lack of Deep Contextual Understanding

AI models generate text based on input and patterns learned during their training, but they lack the human capacity for contextual understanding. If an essay appears superficial, misinterprets key details, or fails to demonstrate a nuanced understanding of its subject matter, this could indicate the use of an AI.  That is best represented by its confusing similar things where the understanding of the context is required to differentiate them.  For example, an American Historian and an Australian Philosopher share the same name.   Most AI models see them as one and it will introduce him as both or it mixes them up in the generated text.

3. Overuse of Certain Phrases or Ideas

AI models can sometimes overuse certain phrases or ideas if they are strongly represented in the model’s training data. Excessive repetition without a clear reason can raise a red flag for AI-generated content. 

4. Inconsistency in Style and Tone

Despite their advanced capabilities, AI models may subtly shift in style and tone throughout a text – an inconsistency that most human writers would avoid. Monitoring for these variations can help detect an AI-generated essay.

5. Outdated or Incorrect Information

As of my knowledge cutoff in September 2021, AI models didn’t have real-time access to new information. If an essay includes outdated or proven incorrect information after the AI’s last training cut-off, it strongly suggests that the essay was generated by AI. For instance, if students wrote an essay about the overturning of Roe v. Wade, ChatGPT wouldn’t have been aware of the recent development regarding the court case being overturned.

6. Utilizing AI Detection Tools

Interestingly, one of the best weapons against AI may be AI itself. Various detection tools are being developed to identify AI-generated text by looking for patterns and anomalies that are characteristic of machine-generated content.  Ultimately AI detection tools like GPTZero and others are being circumvented by AI tools.  For example, ChatSonic will let you rewrite something to no longer be identified as AI generated.   With that said, AI detection tools are very inaccurate, frequently identifying historical writings like Fredrick Douglass and Mark Twain as highly likely generated by AI.    

7. Direct Questioning

Finally, sometimes the most straightforward approach is to ask students about their writing process. Encouraging honesty and promoting a culture of academic integrity can often be the most effective strategy.

The Foolproof Method

After all the other methods have caused you to suspect something was written with a generative AI tool, the surefire method to confirm is to look at the document history. Examining the document’s history can provide valuable insights into the writing process and any potential use of AI tools. By analyzing the document history, you can track the evolution of the text, identifying any unusual patterns or sudden changes that may indicate the intervention of an AI system. Document history serves as a reliable source of evidence, allowing you to verify your suspicions and gain a better understanding of how the text was composed.

An excellent tool for exploring document history is Draftback (link), an add-in for Google Docs. Draftback presents the writing process as a captivating movie. By utilizing this add-in, you can visualize the sequence of edits, copy/pastes, revisions, and collaborations that took place during the creation of a document. By observing the document’s journey from its initial draft to the final version, Backdraft allows you to make informed judgments about the authenticity of the text.

Microsoft Word also provides a way to view document history. Microsoft Word’s history feature allows you to access a list of revisions and edits made to a document over time, providing a means to review the sequence of changes made to a document.

Conclusion

These methods are not foolproof, and some might produce false positives or negatives. As educators, it’s crucial to use our judgment and consider multiple factors when evaluating student work. However, staying informed about AI capabilities and maintaining open dialogues with students about academic integrity are significant first steps. With diligence and careful consideration, we can ensure that AI is used as a tool for learning and inspiration, rather than a shortcut to academic success.