Why most review queues are a definition problem
Most senior review time goes to checking rules, not making judgment calls.
Wrong template, wrong date range, missing disclaimer, broken link. Those have one right answer. When the answer is not written down, every reviewer carries their own version of it, so hiring more reviewers makes the work less consistent, not faster. This test shows you which problem you have. The full method is in my agency operations guide.
Run the test
Answer three short steps and you get a diagnosis, the numbers behind it, and next steps.
- 1. Team
- 2. Speed
- 3. Review log
How to run the 3-week review log
You run the log by writing down every change a reviewer asks for, then sorting each one into rule or judgment.
- Pick one deliverable your seniors always review.
- For 3 weeks, every reviewer writes down each change they request, as it happens (not a weekly summary).
- Sort each change into two piles: broke a rule, or needed judgment.
- Count the piles and enter them above.
- If the rule pile is large, write a definition of done before you hire anyone.
Real example: W3whiz
I ran this exact diagnosis on my own white-label team, and it took us from 30 to 37 accounts with the same 7 people.
I run W3whiz, a 7-person white-label SEO team: 2 seniors and 5 juniors. Two years ago we turned down 5 accounts because I could not say whether we had capacity. Production took 3 days. Delivery took 11. Every deliverable sat in a queue for 2 senior reviewers.
For 3 weeks our seniors logged every change they requested. About 70% were rule checks. Only 30% were real judgment. So we wrote a definition of done for each deliverable, enforced it with template checklists before work reached review, and kept an exception log.
| Before | After | |
|---|---|---|
| Accounts | 30 | 37 |
| Turnaround | 11 days | 3-4 days |
| Senior review time | ~30 hrs/week | ~20 hrs/week |
| Team | 7 people | Same 7 people |
Methodology and assumptions
The tool divides your review changes into rule and judgment, compares the split to fixed thresholds, and flags queue and load problems.
Formulas
- Rule share = rule changes / (rule changes + judgment changes).
- Queue days = delivery days minus production days. Queue share = queue days / delivery days.
- Accounts per producer = accounts / producers. Producers per reviewer = producers / reviewers.
- Rule-check hours = senior hours per week x rule share. Hours freed = rule-check hours x the share you think you can catch before review. Remaining review hours = senior hours minus hours freed.
- Monthly value (if you enter an hourly cost) = hours freed x hourly cost x 52 / 12. Review minutes per deliverable (if you enter weekly volume) = senior hours x 60 / deliverables reviewed.
Thresholds
- Rule share of 50% or more: definition problem. From 30% up to 50%: mixed. Below 30%: capacity problem.
- Queue flag when 50% or more of delivery time is waiting.
- Series flag when there are 2.5 or more producers per reviewer.
- Producer load flag at 8 or more accounts per producer.
- Sample size: under 30 logged changes is a thin sample, 100 or more is solid.
- Short-queue note: if under 25% of delivery time is waiting, the tool says the bottleneck may be elsewhere.
- Hours check: senior hours above 60 per reviewer per week are rejected as a likely typo.
Assumptions
- The thresholds come from my experience running a white-label SEO team. They are not from a study.
- The hours estimate uses share of changes as a stand-in for share of time. Judgment calls usually take longer, so it is rough.
- The share of rule checks you could catch before review is your assumption. The default is 50%.
- "Waiting" is delivery time minus production time. It includes any wait, not only review.
- Hourly cost and deliverable volume are optional and only used for the value and minutes figures. They never change the diagnosis.
- The tool gives a direction, not a guarantee.
FAQ
These are the questions agency owners ask me most about the review bottleneck.
What counts as a "rule" change vs a "judgment" change?
A rule change has one right answer you could write down in advance: wrong template, wrong date range, missing disclaimer, broken link, naming error. A judgment change needs context or opinion: which priorities to lead with, whether it fits the client, strategy, risk. If two seniors could disagree on the fix, it is judgment. If they would not, it is a rule.
Why would hiring another reviewer make things worse?
Without a written definition of done, each reviewer applies their own version of the rules. The same work gets different feedback depending on who reads it, so you get more rework and more inconsistency, not more speed. The checking is still done by hand, just by more people.
How long should I log for?
We logged for three weeks. That covers a full reporting cycle and a few audits, and it is short enough that people keep doing it. One week can show a direction if your review volume is high. Fewer than about 30 logged changes is a thin sample, so treat the result with care.
Can AI fix this for me?
AI can draft and check mechanical rules, but only after the rules are written down. If there is no definition of done, AI drafts make your review queue longer, because you get more output to check against rules nobody wrote.
My agency is small. Does this still apply?
Yes. Even with one senior and two producers, the senior's time is the bottleneck, and a lot of it is usually rule checking. The thresholds come from a team of seven, so treat the numbers as a rough guide for very small teams. Writing a definition of done takes a few hours and helps at any size.
Do you store my data?
No. Everything runs in your browser and nothing you enter into the tool is sent or stored. The share link holds only numbers in the URL hash, which browsers do not send to servers.
Send me your two piles and I'll tell you straight
Not sure what your numbers mean? Book 30 minutes. Bring your rule count and your judgment count and I will tell you what I would do.
Book a 30-minute callEmbed this tool on your site
Copy the snippet below to put this test on your own page, with a link back here.
Please keep the attribution link visible.
