How-to · May 2026

Without this, PMs can’t make the right call.

Engineering is accelerating with AI. PMs are the new bottleneck. A four-level workflow to compress the post-launch feedback loop — so your judgment compounds with every release.

Jim CaralisAI News & Strategy 10 min watchHow-to CompanionTo the YouTube video
Ship → Learn → Decide → Improve
Where AI plugs in
01 · Manual PM reads the KPIs, tickets, and interviews by hand. signals
02 · AI Assist AI synthesizes the signal across sources. insight
03 · System Insights become backlog improvements automatically. backlog
04 · Startup The loop runs end-to-end — learn to ship. ship
“The average PM ships features. High-impact PMs build systems that improve their judgment.”— The point of the loop
01The setup

You’re sitting across from your boss, about to review the release.

You hope it goes well, but you’re fidgeting. You didn’t have time to collect all the data you needed. You’re worried you’ll look like you don’t know what you’re talking about — because in some ways, you don’t. And without the data, you can’t pull a single real insight from the release. You’re not learning. And if you’re not learning, you’re not growing.

If you’re early in your PM career, this is the most important part of the job, because this is where the judgment gets built. You define the feature. You get it implemented. You push it into production. You watch how customers respond. You collect the data. That loop — over and over — is how you learn what works and what doesn’t.

Skip that loop and it becomes a weak spot in your career. Judgment — some people call it taste — is the part of the PM job AI doesn’t do for you. Everything around it is being automated. The only edge you have is the speed at which your taste compounds.

02The foundation

If you skip the loop, you’re just shipping.

It’s tempting to move on the moment a feature ships. Progress feels like throughput — how many features can I push out, how fast can I clear the roadmap. Everyone wants the forward momentum. But shipping without learning is one of the fastest ways to stay busy without actually getting better.

The only way to build judgment fast is to compress the entire loop — from requirement to ship to learning. Ship the feature. Measure what happened. Compare it against what you expected. Use what you learned to make a better call the next time. That’s the loop. AI’s job here is to make every step of it tighter and faster.

Shipping without learning is one of the fastest ways to stay busy without actually getting better.
03Level 1

Manually review the release — did it do what you expected?

Start with the basic question: did this feature do what we expected it to do? Look at the KPIs and metrics connected to it — some live inside the acceptance criteria, some sit just below as success metrics. A conversion rate between two steps. Usage. Retention. Engagement. Completion rates. Revenue. Support-ticket reduction. Customer feedback. Whatever the feature was supposed to move.

The point isn’t to admire the numbers. It’s to compare what you thought would happen against what actually happened. What did you expect customers to do? What did they actually do? Where were your assumptions right? Where were they wrong?

This is also where a postmortem earns its keep, especially early in your career. Ideally you do it with the team. If they won’t, do it yourself — walk back through the original requirement, the acceptance criteria, the decisions you made, the trade-offs you accepted, and the result you got. Capture any improvements and put them straight into the backlog. That’s where judgment gets built.

04Level 2

AI-assisted analysis — don’t grade your own homework.

Instead of only looking at the metrics yourself, hand the LLM the original requirement, the Jobs to Be Done context, the acceptance criteria, the post-launch results, and the customer feedback. Then ask it to evaluate whether the feature appears to be working. You don’t want to grade your own homework — the LLM can be the impartial arbiter.

Given the original requirement, JTBD context, acceptance criteria, post-launch metrics, and customer feedback, evaluate whether this feature is working. Summarize what changed, compare expected vs actual, surface patterns in the feedback, and flag where it didn’t land.

This is especially useful when feedback is coming from a lot of places at once — analytics, support tickets, sales comments, user interviews, emails, internal team notes. AI is good at synthesizing across those sources into a clearer picture. Your judgment still matters — it’s what decides whether the picture is right.

05Level 3

Wire the loop into your product systems.

At this level, AI pulls the original requirement, the post-launch results, and the customer feedback on its own. It summarizes what happened, identifies the gaps, and suggests improvements connected back to the Jobs to Be Done framework. That’s why JTBD matters — it’s the basis for how you evaluate everything moving forward. The agent can then create backlog items in the right place, in the right context, ready to execute against.

The learning loop becomes traceable. You’re not vaguely saying “we should improve this.” You’re saying: here’s what we expected, here’s what happened, here’s what we learned, here’s the improvement we should consider next.

The loop goes from ship → check dashboard → read feedback → write notes to ship → collect signals → AI synthesizes → next actions.
06Level 4 · Startup mode

If the change is small, clear, and safe — ship it yourself.

Once the loop is tight, you tighten it more. When the improvement is small, clear, and safe, you can use Claude Code or Codex to push the change into the product directly. Adjusting copy. Fixing a small workflow snag. Improving an empty state. Updating an event. Tweaking a piece of UI behavior. Nothing heroic — the kinds of things that used to wait in the backlog for a sprint that never came.

The idea is to compress learning and improvement into the same flow. You release the feature, you measure what happened, you understand what needs to change, and where appropriate, you push the improvement forward yourself.

Old loop vs. new loop.

The old loop
ship → move on → maybe check metrics later → maybe learn something
The new loop
ship → measure → AI synthesizes → you learn → you improve it yourself

You’re not just shipping anymore. You’re getting smarter with every feature you ship. That’s the difference between an average PM and a high-impact PM.

07Takeaways

Average PMs ship features. High-impact PMs build the loop.

If you want to stay relevant, you have to at least double your impact. If you want to get ahead — earn the next level of responsibility, become impossible to ignore — you have to triple it. That doesn’t happen because AI magically makes you a better PM. It happens because AI removes the friction from the parts that slow PMs down: writing requirements, validating quality, and learning from what actually happened.

  • 01Judgment — some call it taste — is the part AI doesn’t do for you. Build it faster by compressing the loop.
  • 02Level 1: manually compare expected vs. actual. Do the postmortem. Push improvements to the backlog.
  • 03Level 2: hand the LLM the requirement, JTBD, results, and feedback. Don’t grade your own homework.
  • 04Level 3: AI pulls signals, identifies gaps, and creates backlog items traceable back to JTBD.
  • 05Level 4 (startup mode): when the change is small, clear, and safe — ship it yourself with Claude Code or Codex.
  • 06Shipping without learning is staying busy without getting better.
Stop being the bottleneck. That’s how you stay relevant. That’s how you get ahead.

Watch on YouTube.

A focused walkthrough of the four-level post-launch feedback loop — manual analysis, AI-assisted synthesis, AI-powered backlog, and startup-mode shipping.

Watch on YouTube
Without This, PMs Can’t — Make the Right Call