Our client sends automated weekly performance reports to e-commerce sellers running paid ads through their platform. The reports were accurate, but they read like every other AI-generated report on the internet: correct GMV, AOV, ROAS, and SKU numbers wrapped in the same flat, hedge-everything sentence construction. That is an AI content quality control problem, and it doesn't get fixed by writing a better prompt once and walking away.
The obvious first move is the wrong one: tighten the prompt, add a line telling the model to "sound more human," ship it again, and watch the same tells creep back within three iterations. That treats the symptom as a wording problem when it's actually a structural one: nobody with fresh eyes ever looks at the output before the client does.
The fix was architectural, not verbal: stop asking the agent that wrote the report to also check its own work. An agent checking a deliverable against the intent it had while writing it will almost always find that intent satisfied, so self-review passes. That's why the system now runs two separate roles that never touch the same draft twice.
The short version
- The trigger: a client's automated weekly seller reports had correct GMV, AOV, ROAS, and SKU numbers but still read like generic AI output.
- The fix restructured the pipeline into four steps: draft, grade, rebuild, and a fresh second grade that never returns to the original writer.
- 46 banned words and AI-tell patterns got pulled out of the prompt and enforced as deterministic code checks instead, synced by CI across every skill file.
- Four versions of the grading system shipped in one day, v2.6.11 to v2.7.1, after a client stakeholder sent new 'stop AI slop' techniques to check against the list.
AI Content Quality Control Starts With Two Separate Roles
A GRADER reads the deliverable like the client's most skeptical stakeholder and never wrote the thing it's grading. If the verdict is a fail, a separate REBUILDER produces the fix, and the rebuilt version goes through a fresh grader pass, not the one who wrote the original draft or the fix.
How a report clears QA before it ships
Draft
writer agent produces the first version of the report
Grade
grader reads it like the client's most skeptical stakeholder, returns a verdict
Rebuild
a separate rebuilder agent produces the fix, never the original writer
Fresh grade
the rebuilt version goes through a new grader pass, not the one who wrote it
What makes the fresh pass different is what it can't see: only the deliverable and the standard reach it, none of the conversation that produced the draft, so there is no intent to confirm. The rule earned its keep early: on one deliverable it caught a legally required disclosure rendering outside the image bounds, invisible on the page, that would otherwise have shipped. Neither the writer nor the rebuilder saw it, because both had already convinced themselves the layout was fine.
Why the Rules Live in Code, Not the Prompt
46 banned words, phrases, and AI-tell patterns are enforced by deterministic code checks, not by asking the model nicely inside a prompt. The lists are closed and synced by CI across every skill file that touches the report, so there's one source of truth instead of five prompts slowly drifting apart.
| Rules in the prompt | Rules in code | |
|---|---|---|
| Enforcement | Model has to remember and choose to obey | Runs every time, no exceptions |
| Drift | Rewritten prompts silently drop old rules | Synced by CI across every skill file |
| Model's attention | Split between prohibitions and the actual standard | Goes entirely to templates, worked examples, calibration pairs |
This split was forced on us, not chosen upfront. The old prompts worked as threats until a model upgrade made the newer model start ignoring that register, and banned words quietly reappeared in output that used to pass. The fix moved every hard prohibition into code and left only the positive standard in the prompt, templates and worked examples, and most prompt fixes since have been deletions.
Why a model upgrade broke prompts that used to work
The old prompts were written like threats: "you are NOT allowed to," forced fail states, re-read loops stacked on top of each other to make sure the model got the message. The older model obeyed that register fine, so nobody had a reason to question it. A model upgrade changed that: the newer model started ignoring the parts written in that register.
Our best read of why: a wall of "you are NOT allowed to" looks less like instructions from your own operator and more like a prompt-injection attempt, and newer models are trained to be suspicious of exactly that shape. Whatever the internal reason, the observed behavior was consistent. The rules didn't get weaker on purpose; the register they were written in stopped working.
Turning Jargon Into Something a Human Actually Reads
The plain-language rule, refined directly with the client, keeps industry acronyms like GMV, AOV, ROAS, and SKU exactly as they are and simplifies only the sentence construction wrapped around them. That's the part that was reading as AI-sounding writing in the first place.
Neither rewrite drops a number or softens a bad week; they just say what the number means. "AD ROAS 0.56 - below 1.0" becomes "losing money." A seller reading "8,689 people saw the products that day and nobody bought" gets the problem in one pass, while a seller reading "8,689 impressions" first has to know what a good impression-to-conversion ratio looks like in their category.
That's close to what happened with AI personalization overpromising in cold email too: the acronym-heavy, feature-complete version of a sentence isn't the one a person actually reads.
How Fast the Standard Has to Move
A bad rule caught in production doesn't wait for a release calendar: four versions of the grading system shipped in a single day, v2.6.11 through v2.7.1. When a stakeholder sent over a thread of outside "stop AI slop" techniques, each source got fetched and read directly, then diffed against the existing banned list, and only genuinely new residue got merged in.
The fetch-and-diff step is doing real work there. Most of what circulates as "AI writing tells" is restated versions of rules already on the list under different names. Merging those in as if they were new would have bloated the list without adding real coverage.
What This Doesn't Catch
The grader enforces a standard, not whether a reader is actually engaged: there's no substitute for a human noticing a report that's technically clean but boring. The closed ban lists only catch what's on the list, which is why the fresh-eyes pass exists for everything else, including failures nobody thought to write a rule for yet. And a second full grading pass on every deliverable costs a second model call, every time, with no shortcut around it.
Key takeaways
- An agent that grades its own writing against the intent it had while writing it will almost always find that intent satisfied, so self-review is not real review.
- A grader, a rebuilder, and a fresh second grader pass need to be three separate roles, with the fresh pass never returning to the agent that wrote the original draft.
- Hard prohibitions belong in deterministic code checks synced across every prompt, not in the prompt itself, because code enforcement can't drift and frees the model's attention for the positive standard.
- Plain language does not mean stripping industry acronyms like GMV, AOV, ROAS, or SKU. It means rebuilding the sentence construction around them so a reader doesn't have to translate the number themselves.
- A closed ban list only catches what's already on it. The fresh, uninvolved grading pass is what catches everything the list was never written for, including a legally required disclosure that once rendered outside an image's bounds.
The Takeaway
If AI-generated writing is shipping to clients with your name on it, the real question isn't whether the prompt is good enough. It's whether anything reviews that writing without the bias of having written it. Unreviewed client-facing output is exactly the kind of delivery drift that ends with clients asking for refunds.
Build the two roles apart, keep the hard rules in code where they can't drift, and let a fresh pass see the draft for the first time before it goes out. It is one of the smaller, unglamorous pieces of AI implementation work that ends up mattering more than the model you picked.
Common questions
How do you stop AI-sounding writing from going out to clients?
Separate the role that writes from the role that grades. A grader reads the draft like the client's most skeptical stakeholder, a separate rebuilder produces the fix, and the rebuild goes through a fresh grading pass rather than back to the agent that wrote it.
Why does a second, fresh QA pass matter if the writer already checked its own work?
Self-review almost always passes, because the agent is checking the text against the intent it had while writing it. A fresh grader with no memory of that intent catches what self-review misses, including things that would otherwise ship.
Should content rules live in the AI prompt or in code?
Hard prohibitions belong in deterministic code checks, not prompts. Code enforcement can't drift and it frees the model's attention for the positive standard: templates, worked examples, and calibration pairs.