Do Writing Assistants Like Grammarly Get Flagged As AI?

Key Takeaways

• Copyleaks analyzed whether certain writing assistant tools and their key features get flagged as AI content.
• The study found that content altered by writing assistant tools that do not use genAI features to correct grammar, tone, mechanics, etc., does not get flagged as AI-generated content.
• The Copyleaks data revealed that copy rewritten by writing assistant tools using genAI-powered features like GrammarlyGo will likely be flagged as AI.
• A smaller analysis from another AI detector flagged 20% of Grammarly’s non-AI features, which include spell check and grammar, as AI content at an astounding 20% false positive rate. Such results highlight the importance of utilizing more sophisticated, enterprise-level detection platforms like Copyleaks that can decipher what functions are truly AI to avoid false positives.

How Have Writing Assistant Tools Changed In the Age of AI?

Writing assistant tools are nothing new. 

MIT created the first spell-check tool in 1961, and by the early 1980s, spell-check had become part of PCs for individual users. In 2009, Grammarly arrived as a ‘writing partner’ that checked your spelling and helped you fix grammar mistakes, adjust tone, and more. In short, writing assistants have been a part of our daily lives for a while now.

Then, generative AI crashed the party.

Many of these writing assistant tools have been utilizing AI to help with essential functions for a while, so what’s different? 

Yes, writing assistant tools have been using AI, but now, some have evolved to use large language models (LLMs) to rewrite content, fill in sentences, and more. 

What does that mean? Well, depending on what writing assistant features you use, your content could get flagged as AI-generated, posing problems for students, marketers, bloggers, and more. 

Testing Whether Writing Assistants Get Flagged As AI

To determine what exactly makes content edited by writing assistant tools get flagged as AI, we tested using two AI-powered writing tools: Grammarly and our tool, the Copyleaks Writing Assistant.

We collected 1,000 random files from a public essay dataset of English-language text. The dataset is designed to be English only and does not contain AI. We then ran that content through the Copyleaks Writing Assistant and Grammarly. 

Here is what we found.

• One thousand human-created files were run through the Copyleaks Writing Assistant, with each one averaging around 35% of changes made. These updated files were scanned through the Copyleaks AI Detector. All 1000 came back as human content.

• The same original files were run through Grammarly for correctness and clarity (Note: We used only the more commonly used corrections portion of Grammarly; this does not include their GenAI platform, GrammarlyGo).  These updated files were also processed through the AI Detector. All 1000 returned as human content.

• We then took 500 of the same original files and used Grammarly’s GenAI  ‘Improve’ feature.  These updated files were processed through the AI Detector. 31.6% returned as AI Content and 68.3% as human.  

Things to Note

1. In the last experiment, the content coming back as human-written had fewer overall changes than those coming back as AI.

2. A high cosine similarity between the original text and the ‘Improved’ means they are very similar. “1” is nearly identical, and the texts are less similar as the value gets lower. The average cosine for the files detected as AI was .76, while the files identified as human-written was .88. This means the text detected as AI was significantly changed, resulting in a lower cosine score than the source. In contrast, texts with minimal changes had a higher cosine score than the source and were not detected as AI.

3. A smaller analysis from another AI detector flagged 20% of Grammarly’s non-AI features, which include spell check and grammar, as AI content, an astounding 20% false positive rate. We took the exact text performed in their analysis and ran it through the AI Detector, and it came back with 0% AI content detected. These results highlight the importance of utilizing more sophisticated, enterprise-level detection platforms like Copyleaks that can decipher what functions are truly AI. 

Explore the Copyleaks Writing Assistant 

Explore Now

The Takeaway

Only certain features of writing assistants can cause your content to be flagged by AI detectors as AI-generated.

When it comes to tools such as Grammarly, which offers a genAI option to help re-write your content entirely, whether you’re asking it to improve or shorten the text, it may very well get flagged as AI because, simply put, it is AI-generated. 

On the other hand, the Writing Assistant did not get flagged as AI, nor did any of the content that Grammarly changed to fix grammatical errors, tone issues, etc., because they do not use genAI to power these features or functionalities.

Bottom line: AI changed the writing assistant landscape, as it has with many others. So, while using writing assistants in the traditional sense does not get flagged as AI, to be on the safe side, you’d be better off using a writing tool, such as the Writing Assistant, that does not offer a genAI feature to ensure that your content remains crisp, clear, and original, and free of AI content.