AI Impersonating Human Writing Is Polluting the Internet, Substack Decides to Hand Judgment Over to Readers

Author: Chris Best

Compiled by: TechFlow

TechFlow Editor's Note: Up to 40% of content on social media is already AI-generated, but readers are often unaware. Substack believes this "Claudefishing" (impersonating human writing) is undermining trust between readers and authors, threatening the livelihoods of content creators. Their response is: let readers judge for themselves, rather than the platform deciding for you.

An Internet Where Real and Fake Are Hard to Distinguish

It is becoming increasingly difficult to distinguish what is real content on the internet.

This problem is relatively minor on Substack, and we hope to keep it that way. As a first step, today we are launching a feature in the Substack app in partnership with Pangram. Going forward, you can scan text to see how much of it is likely handwritten and how much is completed with AI assistance. I will elaborate later, but first let me explain why this is important.

Water Everywhere, But Not a Drop to Drink

According to Pangram's estimates, the proportion of AI-generated text on certain platforms is rising, reaching as high as 40%. This is a huge change. Many people complain that their feeds are filled with soulless spam content. Some others don't even realize they are reading something created by no one.

The core issue is not about people using AI, nor about the quality of AI output. Not all content made with AI is spam, nor is all spam made with AI. The problem arises when readers' expectations do not match reality, especially when they invest attention into content backed by no human thought without knowing it. This is Claudefishing.

Freddie deBoer put it this way:

I engage with human-created art because I know there is a person behind it, which is exactly what I am looking for — other people, showing me through art what they hide within themselves. Deceiving me in this process is nothing but a scam.

We care about this issue at Substack because it touches the core of our work: building an economic engine for culture. This means helping real people — those who think, feel, observe, argue, experiment, consider, struggle, exhort, and care — earn money through work they believe in. This network is built on trust between people, which is why it works.

When content created by no one occupies parts of the internet that should belong to humans, it pollutes the public space, making it difficult to discover and hear human voices. When readers have to doubt whether what they are reading is real, it undermines trust in authorship and threatens writers' livelihoods — including those who thoughtfully use AI tools to create work they believe in. Platforms that reward fake content will trigger a race to the bottom.

What We Are Doing

This is an emerging field, and we dare not say we have all the answers. But one thing we do know is that we don't want to wait until your Substack app turns into LinkedIn before we start learning and making progress. So first, we are launching some new tools, hoping to increase transparency and lay the groundwork for soliciting your feedback.

We are partnering with the leading AI detection tool Pangram. You will be able to scan notes, replies, comments, and posts to see an estimate of how much of the text is handwritten or completed with AI assistance. This will apply to text exceeding 100 words published from today onwards, and the analysis results will only be shown to those who request to view them.

Figure: Interface of the AI detection tool collaborated by Substack and Pangram, showing the proportion of text written by humans versus AI assistance. Source: Substack

Please note that Pangram can only detect whether AI was used to produce the text, it cannot detect whether significant human effort was invested in creation, nor can it detect whether AI tools were used as sources. It is not perfect, but independent research shows it can detect AI-generated text with high accuracy. We believe it is beneficial for people to use such tools to help themselves make judgments.

We have also launched tools for creators, including the ability to add a "How I Created This" statement, where you can explain your process and set expectations for readers. You can run Pangram on drafts before publishing, and you can also report and delete any scan results on your own work that you believe are incorrect.

These features are now available on web and iOS, and will soon be launched on Android. This is just a starting point. Based on feedback and interest, we are considering the following tools:

Allowing you to set preferences regarding AI content for your community in reply rules

Providing creators with more ways to express the personal value they offer, whether through human writing or thoughtful AI use

Allowing readers to choose to set preferences regarding what content is recommended to them

Continuing to improve our systems to combat spam, bots, and various scams as AI accelerates these issues

We are not opposed to people using AI to assist with work; we believe people should be free to choose which tools to use to express themselves. Some people on the platform are human writing maximalists, while others openly explore the frontier of using AI tools, while paying attention to details to produce work they stand behind. We have always used AI at Substack to write software, do research, and build product features like clips and translations.

But people should know what they are getting. For example, you might not mind reading someone's sharp analysis where they rely heavily on AI to present research results, but only want to spend time interacting with human handwritten replies. Either way, you should have the choice.

What Do You Want to See?

We are all entering this strange new world together, and we can think of no better way to figure out the new tools and social norms we need than through open debate. We want to hear your opinion: How can Substack help support valuable human creation in the AI era? When is using AI Claudefishing? When is it good?

No matter how powerful AI becomes, the human perspective is a fundamental element of culture, and this will always be the case. In fact, the more you expect these tools to continue improving, the more important it is to understand the source of what you read. Claude has many strengths! But when I want Claude's opinion, I will ask Claude directly.