Blog

What Product Managers Take On When Agents Do the Work

What a PM owns when agents do part of the work.

When a PM delegates work to an agent, part of the job becomes defining how that work will be evaluated and what the agent is allowed to do. Suppose an agent recommends next quarter’s priorities after reading a discussion thread. The recommendation looks convincing, but it has treated silence as approval. Someone has to turn that distinction into a rule the workflow can follow and a test it can fail.

That creates concrete responsibilities for PMs, shared with engineering and other specialists:

Choose where delegation helps. Map a workflow such as feedback synthesis or roadmap planning: which work can the agent handle, which decision does it support, and what would make an error consequential?

Configure the conditions for the work. Specify the context the agent needs, the constraints it must respect, what it can do without approval, and when missing information or disagreement should make it stop and ask.

Create evaluations that test product judgment. For feedback synthesis, check whether the agent asks how participants were recruited before generalizing about demand. For prioritization, test whether it distinguishes silence, objections and explicit approval. Keep those cases and expected behaviors so the team can rerun them when models, prompts or policies change.

Account for the ongoing work. Decide who handles exceptions and maintains the checks. Measure review, correction and maintenance time alongside time saved. Routine checks can be automated; consequential exceptions need clear ownership.

The benefit is useful delegation that frees attention for customers and product decisions. A better prompt alone cannot establish that reliability.

The PM work map covers 120 activities and 1,219 tasks. It also proposes 102 new responsibilities across 95 activities. Below, my agent Mol and I explore what these changes mean in practice, with examples and lessons from my own work.

Mol: If AI is supposed to take work off a PM’s plate, what gets added?

Wilton: Consider an agent preparing next quarter’s priorities. It can gather requests, compare options and produce a recommendation. Before that recommendation can support a decision, someone needs to define which constraints matter, what evidence each option needs, and what the agent is allowed to do next.

Can it draft a priority order? Publish it? Change the tracker? What should happen if an engineering objection conflicts with a sales commitment?

A recurring theme in the model-generated map is evaluating delegated work: checking whether important evidence survives, decisions follow agreed criteria, and the agent stays within its authority. These are proposed responsibilities for practitioners to test and refine.

Mol: Haven’t PMs always had to exercise judgment and check other people’s work?

Wilton: Yes. What changes is the operational work around that judgment. A PM may know that silence isn’t approval. An agent workflow needs that distinction made explicit and tested against threads containing silence, objections, conditional agreement and actual approval.

The PM helps turn “use good judgment” into criteria the team can inspect and examples it can run repeatedly. That includes deciding when several answers are acceptable, when more evidence is needed, and when a person must decide. Agreement with the PM isn’t the standard. A well-supported challenge may be the better result.

Mol: What does the PM need to do before asking the agent for an answer?

Wilton: Check whether the workflow supplies the context the decision depends on, including how to respond when it’s missing. For feedback synthesis, that means knowing who contributed, how they were recruited, and what question the research was intended to answer. It also means preserving an isolated but consequential observation rather than letting the most common comments dominate.

I ran into this with Inedito. I personally recruited five alpha participants who wanted to help shape the product; they weren’t representative of the target customer. An agent with partial context recommended changing direction before asking about recruitment. I’ve logged errors and improved its memory, but the problem remains. That experience gives me a useful failure to test, rather than evidence that I’ve solved it.

The responsibility is to establish what the agent needs to know, what it should ask, and which gaps should stop a recommendation or trigger escalation.

Mol: What would a useful evaluation actually contain?

Wilton: Start with a real product decision and an example of how the agent could mishandle it. For the feedback workflow, give it incomplete recruitment information and check whether it asks before making a broad claim about demand. Then provide the missing context and see whether it uses it appropriately.

Include a case where one participant reports a serious problem. A small sample can still expose something worth acting on. The criteria should distinguish that from evidence for changing the whole product direction.

Each example needs the input, the behavior you expect, and an explanation of why it matters. Some checks have a clear answer, such as whether a total reconciles. Others need a rubric: is the recommendation supported, are uncertainties visible, and are alternatives considered? Keep examples of failures alongside successful cases, and revise them as the team learns.

Mol: Does all of that belong to the PM?

Wilton: It needs shared ownership. The PM brings the product goal, decision context and consequences of getting it wrong. Engineering contributes system behavior, technical failure cases and the mechanisms for running checks. Design and research help judge evidence and user impact. Specialists define requirements in their own domains.

Together, they agree on acceptable behavior and who can approve exceptions. The PM should help ensure this work has an owner and fits the product decision. Writing every test or setting every permission personally would be an unhelpful interpretation of the role.

Mol: Once the examples pass, what work is left?

Wilton: Keeping them useful as the workflow changes. In the model-change evaluation workflow, a new prompt or model may behave differently. A policy update may make yesterday’s reference answer wrong. A changed customer fact may affect several recommendations. Broader permissions may turn a drafting mistake into a published commitment.

Someone needs to track those changes, update the relevant examples and rerun the checks before expanding reliance on the agent. For a roadmap workflow, that could mean replaying a changed dependency and checking whether affected priorities are flagged. The PM helps identify what product decisions could be affected; the team decides who maintains each part and when a failed check should pause the workflow.

Mol: Are we turning PMs into full-time reviewers?

Wilton: That would be a poor outcome. The purpose is to make delegation useful enough that the PM can spend more time exploring opportunities, talking with customers and making decisions with the team.

Review should reflect the consequences of an error and the evidence of reliability. Routine work may use automated checks and sampling; consequential exceptions may need a named person. Watch what happens in use, including repeated corrections, missed escalations and the effort needed to maintain the setup. Count review and repair time when judging whether the agent helps. Producing more documents is a weak measure of that.

Mol: What should a PM do first?

Wilton: Pick one workflow you already delegate and one failure that would matter. Write down the decision the output supports, what good behavior looks like, and what the agent may do without approval. Turn the failure into a small evaluation with a clear explanation of what you’re checking. Agree with the relevant teammates who owns the check and what happens if it fails.

Run it, inspect the result and improve the workflow. Keep the example so you can use it again when something changes. Over time, add the cases that reveal meaningful gaps.

That is a concrete skill for PMs to develop: making product judgment explicit enough to guide delegated work, and building evidence that the delegation is helping. The map offers starting points. The next step is to try them in the workflows we actually run.

If you run one of these workflows with agents and want to compare notes, ping me.