Turnitin is one of the most well-known and trusted plagiarism checker models on the market and is widely used by academic institutions. In April 2023, the system evolved to include an AI detection model and expanded its features to keep up with the ever-changing world of AI.
But how does this new AI detection model work? And can you trust it? In this guide, we’ll explore Turnitin’s AI detector from A to Z, including how it works, how to interpret its results, and how accurate the tool is.
So, if you’re ready to delve into the ins and outs of Turnitin, let’s get started.
How Does Turnitin Work For AI Detection?
Turnitin has a unique approach to detecting AI-generated content. It involves a combination of advanced algorithms, machine learning, and language analysis.
This system is designed to identify pieces of text that have patterns that are consistent with being generated by AI or Large Language Models (LLMs). These may include ChatGPT-3, ChatGPT-3.5, and similar AI models.
As simple as this explanation may sound, there’s a lot more that goes into this process than meets the eye. There are a series of steps that Turnitin’s AI detection tool uses to flag content generated by AI writing tools, with each step playing a crucial role in the process. These steps include:
- Submission processing: When you submit an assignment through Turnitin, it is first broken up into smaller portions of text. This is usually a few hundred words per portion to help analyze the text in context and improve Turnitin’s accuracy.
- Segment scoring: Then, each segment of text is scored by the AI detection model on a scale from 0 to 1. A score of 0 suggests that the text is human and a score of 1 indicates that the segment is AI-generated text. The scores go in increments as well, with scores from 0.5 to 1 being flagged as AI depending on the context within the overall content.
- Aggregation and prediction: Once all of the segments have been scored, these scores are combined or aggregated to generate a prediction of how much of the text is likely to be AI-generated. This overall score is then presented in Turnitin’s AI detection indicator.
It’s important to remember that Turnitin’s AI detection tool is trained on various language models, which helps it pick up on text generated by multiple different AI tools. Plus, unlike some other AI checkers, Turnitin is designed to recognize language characteristics that are generated by LLMs.
For example, LLMs usually produce text that has a high probability of selecting the next word in a sentence based on patterns that the AI writer has learned from its training data. Turnitin’s AI detector classifiers are then trained to pick up on these patterns and can differentiate them from human writing.
How Accurate Is Turnitin’s AI Writing Detection?
One of the most interesting claims that Turnitin makes is that its AI writing detection model is up to 98% accurate in identifying text generated by AI. While there are plenty of academic institutions that use Turnitin’s AI detection tool, it’s crucial to understand that there is no AI detection tool that is 100% foolproof.
As with any AI detector, Turnitin is still at risk of false positives. False positives refer to instances where AI detectors incorrectly classify human writing as content generated by AI. Of course, this can cause students to be falsely accused of using AI tools to generate their assignments.
To avoid this unnecessary complication of using an AI detector, Turnitin emphasizes the importance of reducing false positives and regularly undergoes testing. This means that it may occasionally miss AI-generated text. However, Turnitin aims to keep its false positive rate below 1% for assignments with over 20% of AI writing.
Although Turnitin is a way to streamline the vetting process for submissions, knowing how accurate this AI detector is can make your grading process more accurate. For even more accurate grading results, you can also use Smodin’s AI Grader to help you process and grade more submissions in even less time.
What AI Writing Models Does Turnitin Detect?
Turnitin’s AI writing detection capabilities are specifically designed to identify content generated by various AI models. When it was initially launched, Turnitin was trained to detect models like ChatGPT-3 and ChatGPT-3.5, as well as all of their variants. This includes recognizing content generated by AI language models that are based on or similar to the ChatGPT-3 model.
Turnitin has been tested for its compatibility with ChatGPT-4 (particularly ChatGPT Plus) and claims to be able to detect AI writing produced by the model in most cases. Still, the AI detector needs to be tweaked and re-evaluated for better accuracy with this AI generator.
As AI continues to evolve, Turnitin has acknowledged the need to expand its AI writing detection capabilities for new or updated AI models. This means that educators should be aware of potential false reports when students use more advanced or different AI language models.
If you want to use an AI detector that can accurately pick up on text that has been generated by other models, then be sure to check out Smodin’s AI Content Detector.
Understanding Turnitin’s AI Results
Part of using Turnitin’s AI writing detection tool is knowing how to interpret its results. The percentage that is displayed in the AI writing detection indicator usually represents how much of the submitted text was determined to be generated by AI writing tools.
Usually, this percentage is based on ‘qualifying’ text. This involves prose sentences that are written in a standard grammatical form. However, it doesn’t include text that may be written in the form of lists or bullet points. So, when you’re checking submissions, it’s important to remember that not all of the text may have been scanned – only the qualifying text.
Another important factor to consider when working with an AI detection tool is that it should always be used as a supplementary aid and shouldn’t be used as a definitive way to prove or disprove the use of AI writing. Since there aren’t any AI detectors that have 100% accuracy, you’ll need to use your own judgment in analyzing the AI scores as well.
Turnitin’s Similarity Report
Turnitin’s AI detection capabilities have made it a powerhouse for checking that a submitted assignment within an academic context follows the rules and regulations of the educational institution. However, Turnitin also includes an integrated plagiarism checker.
For this reason, understanding the difference between the AI score and the Similarity score is crucial for educators. Otherwise, you may unintentionally accuse students of using AI-created text when, in fact, their assignment has come back with a higher plagiarism score – not a high AI score.
The Similarity Report provides educators with a detailed analysis of how similar a submitted assignment is to existing sources (within Turnitin’s database). The plagiarism checker, alongside the similarity score (represented as a percentage), highlights the similarities between the two texts.
FAQs
Does Turnitin’s AI writing detector flag paraphrased content?
In some cases, submissions may include paraphrased content that has been spun using an AI Detection Remover.
Turnitin’s AI detector can usually pick up on instances where these tools have been used, although there is still the possibility of false positives or negatives depending on the AI writing model.
This is particularly true if the text contains a combination of AI-generated content and human-written content.
How is Turnitin’s AI writing detection different from plagiarism detection?
Turnitin’s AI detector and plagiarism checker features are two distinct processes on the platform. The plagiarism checker is designed to look for cases where the text shares similarities with other published articles, blogs, academic papers, and general content. This is represented in the similarity report. The higher the similarity score, the more similarities the tool has found.
On the other hand, Turnitin’s AI detection capabilities are limited to finding instances of AI writing, where the content is found to be generated by language models like ChatGPT. This is represented by an AI percentage score.
Can Turnitin detect AI content in languages other than English?
Turnitin’s AI detector is limited to analyzing submissions in English-based texts. This means that institutions that primarily receive essays and papers in English can use Turnitin to analyze these documents, but the checker will not work for submissions in other languages.
Final Thoughts
Turnitin is a fantastic tool to maintain academic integrity in the midst of the evolution of AI content. With the tool claiming up to 98% accuracy, it has become an invaluable aid for educators and students alike and seems to be improving and expanding to keep up with the evolutions in AI writing tools.
In addition to its AI detection feature, it also has an integrated plagiarism checker, creating a comprehensive tool that reduces the need for multiple tools. In turn, this makes checking and assessing academic documents quicker and easier than ever.