When we leave the warning labels behind.

A month ago, on August 2, the EU AI Act’s transparency rules came into effect.

Some AI systems now have to tell you that you’re interacting with a machine.

If it’s happening in Europe then it’s going to happen elsewhere. It already impacts any company doing business in the EU, and the penalties can be significant even for the largest of enterprises.

So regulators and standards bodies around the world are coming to the same answer on AI-generated content.

Give it a warning label.

It’s useful. It tells you if AI was involved.

It doesn’t tell you if the thing is true or not. Where the claim came from. How it was weighed. Who - or what - checked it. Or whether the thing could be reconstructed a few months later.

Disclosure answers how was this made?

Provenance answers why does it say what it says?

Right now we’re spending a lot of effort on the first question.

Warning: you are about to be amazed

A magician can do a card trick that looks impossible.

I don’t believe in magic. If I’m watching it, I know that there’s some kind of mechanism involved. But I also don’t need to see that mechanism.

There’s a tacit agreement between performer and audience.

A lot of AI work is in the same kind of category.

I can ask Grok to give me five subject line suggestions for an email. ChatGPT to rewrite a paragraph for me. I use Granola daily to summarize meetings I attend.

If the output is bad, I recognize it. And I can correct it.

It can be wrong, but that wrongness isn’t going to compound.

Except…when a number from a summary gets put into a competitive analysis. And then that number becomes an important foundation for a strategy document. And I write a product narrative based on that strategy. And that narrative goes into a sales deck.

Sometime early next year, someone might ask;

“Hey, where did that number come from?”

It’s not helpful at that point to know that AI helped write the original document.

They need to see the mechanism.

Disclosure solves a binary problem

The Velvet Sundown released three separate albums in 2025. Their most popular song, Dust on the Wind, has been streamed nearly 5 million times at the time of writing.

Within a month of their first appearance on Spotify, and after presenting themselves as flesh-and-blood, the creators admitted that everything was AI generated.

That’s a disclosure failure. And adding a warning label genuinely changes the understanding of what you’re seeing - or listening to. “AI-generated band” is all you need to solve the problem.

The same principle applies to an AI-generated product claim. It’s not supportable. We label it. The claim is still wrong.

Or a competitive benchmark we’ve copied from an invented source. We label it. The source is still invented.

Disclosure fixes attempts to conceal AI.

I’m worried that organizations are going to treat disclosure as if it’s a more general-purpose control.

But we don’t trust in transparency

The research around AI disclosure suggests that people want AI disclosure, and they use it as a reason to trust the content less.

Klaviyo did an extensive survey across eight countries in 2025.

Ninety-one percent of people expected brands to disclose AI-generated content.

Seven percent said if they saw that disclosure, it increased their trust in the content.

Thirty-one percent said it decreased it.

So our early conclusion for companies to reach might be that introducing transparency is just going to backfire.

But the AI disclosure label is a confession without any defense.

I can make the same claims as a human. The disclosure doesn’t say whether the claim itself is accurate.

AI means that some kind of non-deterministic, probabilistic system was part of making the work.

It doesn’t introduce any differentiation around who checked it. Whether it was based on evidence. Whether it considered alternatives.

It doesn’t say who signed-off on the decision.

The disclosure has introduced additional uncertainty. It hasn’t resolved any of it. Of course confidence is going to fall.

A few of my own receipts

A few months ago I was doing some competitive research.

I use AI in my work.

A number made its way into the corpus of knowledge. A competitor, according to the research, produced results 2.76x faster. Its code quality was 12.8% better.

It even had a citation.

And so that number turned up in a bunch of different documents over the next two weeks.

Even my first-pass provenance verification said: yes - this is public.

But, while there was a public source, which is what the original research had cited, it didn’t have any numbers. It made some much more general qualitative claims about faster work and lower token usage.

...we already know that AI hallucinates. In this case, it invented the number.

That number came perilously close to acquiring institutional legitimacy.

Every new document was another credible-looking artifact sitting between us and the original. It became less and less likely that someone would go back to inspect it.

At a certain point, the source wouldn’t be an inaccurate reading of a competitor’s marketing material.

The source would be us.

And that’s a dangerous failure mode. When wrong information propagates, its origins get left behind.

Leaving the warning label behind

A marketing planning document had a figure about where 85% of our pipeline came from. It had an explicit tag: internal only.

A few days later, the same figure appeared in a draft for a sales one-pager.

The number had traveled.

Its label hadn’t.

Disclosure attaches itself to the artifact. But the risk around it attaches to the actual claims.

So when the artifact stays put, the claims move.

Copied into slides. Summarized in a Slack conversation. Fed from one model to another. Then they become sales copy. They’re paraphrased into a board briefing. Another calculation is based off them.

The original document that was touched by AI might still have the warning label.

The assertion has already moved through the system.

Keeping provenance for when it matters

Every AI-generated sentence doesn’t need a chain of custody. That would be ridiculous, and nobody would do it.

I don’t need provenance for a first-pass concept. I don’t need it for a meeting summary, of a meeting that I attended, that I read immediately after the meeting.

But the distinction isn’t as simple as whether it’s high-stakes or low-stakes. That’s probably too subjective.

Let’s think, instead, about the structure of an error.

  1. Latency. If this is wrong, will the error survive beyond today?
  2. Propagation. Will other work be built on this before it’s verified?
  3. Reconstruction. If this is challenged at a later date, will we need to know how we got here, or can we simply correct an error and move on?

If you can answer “no” to all three, you’re probably fine. Use the AI. Check its work. Then move on.

If there’s at least one yes?

Maybe you should keep the trail.

Two artifacts of the same type might have different answers. Take that meeting summary. If it’s just me checking the summary of a meeting I was in, likely no problem. If that summary gets shared with someone who didn’t attend the meeting, and it hasn’t been checked...now we have potential latency issues.

Any claim about a regulated product is going to qualify. Sustainability, sourcing, accessibility conformance, competitive analysis, pricing and terms.

If there’s any number you might want to use downstream, it qualifies..

What about an AI-drafted executive byline in an article? The accountable person is printed right there with the artifact.

A small but mighty trail

We don’t need another platform for this.

But here are some useful notes you might take:

  1. Source. What were the inputs that produced this output? Specific enough that someone else can find them.
  2. Alternatives. Why did we choose this? What was it measured and weighed against?
  3. Sign-off. Who reviewed it? Who owns it?
  4. Reproducibility. If we used the same inputs would we expect to get a recognizably similar result?

None of this is especially ground-breaking. It’s the same kind of problem that multiple professions and practices have worked out a long time ago.

Journalism uses sourcing. Science has citation and peer review. Business finances have audit trails. We’ve probably all seen a CSI where the forensic chain of custody was a plot point.

There are times when “trust me” is fine. There are times when the assertion is too consequential.

When that happens the claim needs enough preserved to reconstruct how we got to it.

“Has this been changed?” and “Is this right?”

We can determine a lot of useful information about digital assets. Where did an image originate, has it been edited. Even what tools people used to refine it.

We can determine an entirely trustworthy history of the digital image in an advertisement.

The product claim that’s printed on the image might be completely inaccurate.

That’s why we need to know where the asset came from, but also why we arrived at the conclusions the asset contains.

Otherwise we could perfectly authenticate something that’s just plain wrong.

The AI Act solves the problem it needs to

Article 50 of the EU AI Act is specifically about transparency.

It tells people if an artificial system is involved. That’s a completely legitimate, and scalable, target for regulation.

Is there synthetic content? Is it marked? Was the user told.

These are observable, and binary.

“Why did your marketing team decide this claim was supportable?” is much more difficult to integrate into some kind of universal statutory requirement.

In 1994 my dad wrote a book called Corporate Responsibility. It was pioneering because it argued that business doesn’t operate in a vacuum where it only cares about stockholder profit. Business should pursue a social right to operate as well as a legal one.

Organizations shouldn’t confuse the boundaries of legal compliance with the boundaries of their responsibility.

The question isn’t only:

Was AI involved?

It’s:

Where did this come from?

The AI Act is designed for regulators who need to inspect millions of outputs.

Provenance is about whether you can defend just one.

Further reading:

Get started

See how Knapsack helps you reach your design system goals.