Sign inAbout usWe wrote about our discoveries with AI + Google. Read it here.
Written by Patrick A.: Patrick is a key part of building Willie. Has added more than 10M unique visitors to multiple premium publishers and YMYL businesses, including Thought Catalog + Oak Street Health.
We’ve been looking at the data surrounding “AI content” for… well, longer than I would have liked. Since around October 15th, 2022. Which doesn’t sound very long, although it feels like the focus shouldn’t even have to be on this.
Where did this all start? The story starts from a pretty simple experience…
Hiring freelance writers, everything was going well… and then, what happened? Traffic losses across the board got to be pretty major.
The first thing we all think of is, well, it must be AI content! Google doesn’t like it! Run, run, run, go fix ALL AI content that the freelance writers used.
Okay, well wait, now Google says “AI content” isn’t so bad…
So what the h#ll is going on!? Results like this are happening across the board…
There’s a very smart reason that Google said (in its white paper, which you should read here) why it can’t really detect AI content… And some interesting correlations to why it says “helpful content update” (branding here is important) and not “Anti-AI update.”
Firstly, these models are actually really good at detecting poor-quality content. You’ve heard that already, but what does it actually mean, and why is it somewhat dangerous for us?
Google decided to use the model to look at a document and ask, “Could a machine recreate this same prompt and come to the same conclusions?” That’s very important to recognize the question in that format. It says—if your documents looks similar to the way an AI might make a document, it’s going to classify it as a low grade…
Sounds fair! (Sarcasm)
Yes, so think about this from this perspective—authority (or expertise and experience) is not something that a machine can recreate. It can take previously modeled experiences and rewrite it, but that’s not exactly new, is it?
This is why reviews, forums, and other content like this could experience a resurgence. They typically have original experiences. Where the original footprint of the uniqueness of the document is stamped in originality (If you’re asking, does Google do this? Of course, this is what an index is).
If you’re already there, then you’re just asking to game the system. And that’s fine. Although, it’s not going to work. ChatGPT cannot come up with a unique experience to write about.
Maybe in the future. But even then, it would need to be a robot that tries out a nail salon or something to then try and write a unique review. You could lie, yes—if you want to.
Well, this is how they work. This is where they fail. And to transition into Google and your rankings—this is how Google realized these models (when you reverse the question: to analyze rather than to create) are perfect for detecting poor content.
This has to be the biggest misconception right now. After reviewing many verticals, including the following:
I found myself not being able to find websites that got BY (because its mostly a negative ranking signal) the “helpful content update.” Mostly I witnessed everyone getting setbacks...
Yes, websites with AI on them tanked really badly. But, to determine how Google is now “working,” I needed to find out who "won" and why they "won." Note: the word "won" here is basically just me asking, who wrote the best and why does Google like it?
Note: I even asked SEO communities to share with me a site that increased in search traffic (as a byproduct of other sites tanking) and I would pay them $100 just for the site mention... It was crickets...
After weeks, I found a few (and ONLY a few).
The first one was Science of People (I have more, but I don’t want to call out too many sites—it’s frowned upon in our community).
This is a somewhat medical-related topic. And while they have a tremendous backlink profile, one thing really stood out… The writing on the website was ALL (using Willie to analyze this) very humanistic.
It all had touches and elements of real humanistic components. What does that mean? It means that you’re basically trying to prove that you’re human, not just writing "good-quality" content.
Grammar matters, but only the right kind. Poor grammar tends to get marked as AI content. Although, writing style is important to focus on as well (and usually improves your "AI score").
It's probably not what you've already read on the internet (for example, to use Hemingway to make it perfect)...
Gosh, this is confusing and hard to show, already, isn't it? (Yes, it's going to get harder to compete going forward!)
Of the websites I found that "won" in search traffic (or had a benefit of other sites dropping), nearly all of them had kind of funky writing styles. They were using a lot of passive writing styles, using triple ellipses to display emotion, and writing in ways that sounded very human.
This makes sense if you think about it... ChatGPT writes things nearly perfectly, and no human does that. Humans write with emotion and randomization.
A little bit of dog food on this... I took this very article and put it into the RoBERTa detector. Here's what's interesting. It found portions of my article that were not written as great as others (if you think of it this way, it's valid!)
Here's a good section it found:
Compared to a bad section it found:
Metaphors, analogies, and other passive language is (at least today) only something a human can do. Okay, don’t come at me—you’re already typing into ChatGPT to prove me wrong.
Remember: ChatGPT only uses historical information to make these! Just wait until you keep reading.
If you make an analogy about how surfing and snowboarding are different, it really shows that you've done both of them. And they're somewhat related, yet unrelated, when it comes to a knowledge graph. Bingo! Human. Queue "experience" getting added to EEAT.
When you look at things like Yoast, for example, it provides some great guidelines around not over-optimizing headlines.
A headline like this: What is a sweet pea?
And a headline like this: Understanding the sweet pea plant and why I love them for privacy in my backyard
One headline definitely looks more human... One doesn't.
Are you catching on yet? Yes, as an SEO or brand of any type, you will have to write in a way that PROVES you are human.
This is an odd one. What we found by putting writing that wasn’t written by native English speakers is that it often scored as “AI content.” And this is actually correct, to a degree. It’s not AI-content per se, it’s just not well-written English.
Hey, I don’t agree with this, actually. But I’m not Google. If you have complaints with this, file it with the folks over at Google. I do find this to be somewhat of a social issue, definitely.
It means that if you’re not a native English speaker, apparently you don’t write as well. Hmm, I don’t know about that—seems judgmental.
The above writing is from a freelancer in India. This is absolutely written by hand. That was validated by ourselves, but unfortunately, gets tagged and flagged by the detectors.
People are saying, “Well, the AI detectors don’t work!” Well, they do… if you think about it as a measurement for “good” and/or highly humanistic content.
Yes, your handwritten content can get tagged as “low” or “AI” quality — basically, it’s the same now. I hope that makes sense. You’re going to have to write in a way that REALLY shows you're human. (See even me just using "REALLY" in all capital letters and using a passive word—makes it more human).
Getting this "to click" will really help: “Low quality” or “AI content" ... Really its 1-to-1 now…
In a way, if you think about it, Grammarly is telling you how to sound more robotic. Sometimes it's right, sometimes it's wrong.
In fact, if you’re relying 100% on Grammarly and wondering why you’re still getting dinged as AI—you have your answer.
It’s looking for humanistic randomizations. Is that really “good content,” well apparently to Google, it is.
Tools like SurferSEO might not work anymore, because they encourage you to sound like every other search result. I would say this is going to vary based on which niche you’re in and how you use it. If you use it to cover the basics, it might work. And it might not!
Why, you ask? Well, obviously, because you’re still not adding anything NEW to the discussion (your keyword).
All of the reasons above are why Google probably won't come out and say, "We can detect AI content..." Rather, they've said, "We can detect poor quality content (with the same tools)."
Because, well, hand-written content can also get flagged by these "systems." It is a new day, a new era—old tricks won't work anymore.
The branding of this was so sneaky. It’s asking for “good content,” but remember, I’m suggesting that your article doesn’t need to look like a research paper. It just needs to look human.
Okay, you’re going back to ChatGPT figure out how to get it to write in this way, right? You can! I think you're going to do more work to try and get ChatGPT to sound this way than to simply write it by hand. And write it well.
Here’s a big thing I found...
I noticed a strong correlation to word count not being a factor anymore. I’m going to give you one search term to look at to see this (again, I can’t reveal too many).
If you look, the site with the lowest DR and even the lowest word count is ranking in the top 10. Why is this happening? Because the article brings unique insights to the table— and the most amount of them based on what makes an overall percentage of its document.
No, this isn’t about plagiarism! You can rewrite whatever you want, but it won’t work. Just like ChatGPT won’t work ;) It’s just a rewriting machine.
So I dug in and asked why this was the case... I read through every article and found a pattern...
Catching on yet? Google wants BRAND NEW, never-seen-before insights (which is why ChatGPT probably won’t work for you).
And content that’s unique, which matches your E-E-A-T backlink profile.
I suppose you could use it for maybe 30% of your document. I would think that’s safe. But when you really want to go for this, it’s a combination of having the entire domain have all of your documents containing very unique insights.
Queue why you need Willie. We are vouching to be the only humanistic SEO analysis tool on the market. You’ll need to use the tool to analyze your entire domain for both AI content and “poor” quality content (in reality, just content that doesn’t look as human as it could).
If your domain has content across the board that’s strong, your search traffic is strong.
In the screenshot above, we see how the domain scores a 95% on Willie (super hard to do from what we’ve seen), meaning all of their content is very humanistic and mostly unique, too (no not just unique rewriting!). See how the domain started to boom around the time of the helpful content updates.
“Helpful content update” is the most brilliant branding strategy I’ve seen in 5-years. It tells you what to do, without telling you what to do, making it the most confusing thing in the world—which is why you’re here!
Once you start to dig into the way that page quality and detection works, you start to realize that most other detection models on the market aren't telling you the whole truth. Unless you're analyzing roughly 500 words worth of writing (and ONLY that)...
If you have more than 500 words, well, anything past that 500 isn't even being graded. Yikes!
So you're basically getting ripped off. And why many people are skeptical of all things AI detection or "low quality" detection.
Tokens, which are in short, "unique words," and not transition words... make up the count. Looking at a page document, this is about how much is getting measured:
There's a lot more meat left to this fantastic 2,000+ word article that isn't getting analyzed by other systems in the market...
So whether you're asking if the page is AI or generally just "low quality" in Google's eyes, it's a complete blind spot for you, the marketer/SEO.
We can detect the entire document using a method that we've created. And plan to lean more into holistic analysis (much like I've already described in this write-up, the importance of it is quite significant.
Take these new perspectives and dig into your respective niche and industry. I sense you’ll find what I’m seeing. If you don’t, it could be based on a number of other factors.