Blog

You Are the Product: Pause & Learn

Developing and hiring PMs as the work changes.

If you’re a Product Manager (PM), lead PMs, or are hiring one, I want to share what I’ve been learning about the work we expect PMs to do with AI agents and how we help people become good at it.

Five months ago, I left a large team at a strong company to become a solo founder, bootstrapping my own product. I’ve always been drawn to inventing things and finding better ways to build them. Now, every week, I find another way to work with agents and improve the quality of my work. It’s overwhelming and exciting.

I’ve been looking at this as both a builder and a product leader. What does Ian McAllister’s Top 1% PM look like when agents can take on so much of the work? How would I recognize that capability in someone I was hiring? How do we help PMs develop the skills to work in teams of people and agents? Those questions led me to create a PM work map that I’ve been sharing more and more with friends.

The work is changing faster than the standards we use to develop and hire PMs. We’re still asking what makes someone excellent, but agents can now do substantial parts of the work that used to demonstrate that excellence. I want to explore what that means for the expectations we set, the skills we help people build, and the evidence we look for when hiring.

I wrote this essay as a podcast interview, with my agent Mol asking the questions. I hope it helps you drive your career forward or build stronger teams.

Mol: Which parts of being an excellent PM are changing most?

Wilton: Writing good copy, technical depth, and speed were core skills I looked for in PMs and worked on in my own craft. AI greatly extends all three, especially when agents work with highly capable models.

Thoughtful prioritization and broad communication through roadmaps can also be largely automated. An agent can bring deep technical knowledge into the analysis and reason across more dimensions and data points than I could work through before.

That changes how much work a PM can take on and where their attention is needed. The impact on how we work is massive.

Mol: And what parts of the job are less affected by AI, or are emerging because of it?

Wilton: First, product vision, the opportunity, and the strategy to get there. In my experience, agents still produce mediocre work here. Choosing a direction and building momentum around it, especially before there is consensus, still depends on people.

Second, product principles. They become the basis of how both agents and humans develop the product. Defining them is consequential because they guide so much of the work that follows.

Third, the operating principles and systems for a team of people and agents. This is more relevant to people managers. Deciding how that team should work together still requires strong human judgment.

The PM remains accountable with the team for whether the product succeeds, grows at the expected rate, and actually helps users. Being able to delegate more work doesn’t remove that responsibility.

Mol: Does that make you optimistic or pessimistic about product management as a function?

Wilton: I’m optimistic. I think we need PMs even more! We need people who can define a vision and drive the product toward what people really need.

Mol: What does that change for engineering and design managers?

Wilton: That deserves its own article, but company size and maturity with agentic workflows make a big difference.

I’ve seen early-stage AI-native companies operating with one product leader. Sometimes that person comes from product management; sometimes from engineering or design. Someone takes the lead for the product, and it isn’t necessarily the person with the PM title.

As organizations become larger and more complex, with more legacy systems, separate functional leadership may be needed and the development process becomes more complicated. Who owns what has to make sense for that organization.

Mol: What are PMs struggling with most in the middle of that transition?

Wilton: I regularly talk to PMs who feel insecure about their work. Am I using AI the right way? Is my product iterating fast enough? How can I help my engineering team more?

They’re trying to understand what excellence looks like while the industry is moving so quickly. It’s difficult to judge your own progress when the expectations are changing around you.

I think many knowledge workers will face similar feelings. For PMs, that makes the development question very immediate: what should I learn, and how will I know I’m getting better?

Mol: How do you make “excellent” more concrete while the work keeps changing?

Wilton: Anthropic’s future-of-work scenarios inspired me to look inside the job. I mapped 120 PM activities across seven areas, considering team size, seniority, and company context.

I wanted to understand where AI could help a PM, where it could complete a defined task, and what responsibilities emerge because AI is part of the product or the way we work. The work map gave me something more concrete to examine than the job title.

I used public job postings and take-home prompts to help select relevant work that a candidate could demonstrate in a bounded assignment. The selection covered different areas and levels of PM work and considered how AI changes it. That produced 30 assignment formats, including one I was already using.

In my internal tests, AI alone produced results for the 29 new assignments. In all 29, the work was rated Adequate or Strong on every one of the five assessment criteria. I use these assignments as proxies for defined PM tasks.

That distinction matters for strategy, too. An agent can produce a strong written revision given a brief and evidence. Choosing the opportunity and getting a team behind a direction involves more than producing that document. We need to understand both when developing or hiring a PM.

Mol: Can you show what that looks like in the work itself?

Wilton: Take the quarterly priorities assignment. A fictional wholesale marketplace has 12 competing items for the next quarter. Then a correction moves a payments migration to another team. That frees capacity, but the migration won’t finish until March 31. Ordering work that depends on it still cannot ship within the quarter.

A useful plan has to work through both capacity and timing, explain what makes the cut, and preserve the dependencies. The AI-alone result received Strong on all five written criteria. Agents can help me process the evidence and build a recommendation; getting people to commit to the trade-offs is still part of the job.

Then look at the AI model-change evaluation assignment. A software company is considering a cheaper model for its assistant. The readout says users prefer it two to one, but that claim traces back to one grader result. Sample outputs include an unrequested refund without confirmation. How much should we trust that apparent win?

The task is to recommend what to do and design an evaluation suite for future changes. Here, AI creates work around defining acceptable behavior, checking the evaluator itself, and turning failures into test cases. This is one concrete example of the judgment I’m talking about: a PM working on AI products needs to understand what the evidence does and doesn’t establish.

The appendix lists all 29 tested assignments and links to their underlying work maps, so you can judge how relevant the work is to your context.

Mol: How would you bring this into a larger organization?

Wilton: I’d start by reviewing the product leveling guidelines, function scope, and metrics. If the work changes but our expectations don’t, we risk developing and recognizing people for a version of the job they’re moving beyond.

Then I’d work with my peers and teams on the operating system for product development: what people and agents own, what context they need, where review is required, and how the work moves between them. A sophisticated setup can still create more work for everyone else. I’d look at whether decisions improve, whether less work needs correcting, and whether the team actually gains time after accounting for review and maintenance.

One example is what happens when a human catches an agent’s mistake. We can repair that result. But how does the correction improve the instructions, context, checks, or evaluations so the system becomes more robust? That improvement is work someone needs to own.

Principles and product direction also need more attention because agents rely on them. I’d want the team to become good at shaping that environment, alongside spending time with customers and thinking about future scenarios based on durable needs. Ownership will vary across product, engineering, design, and other leaders.

Mol: Where does a PM start without chasing every new tool?

Wilton: With an activity they actually do. Use the work map to examine its inputs, decisions, outputs, and context. Then try it with agents using your own tools.

The useful learning comes from what happens: where the agent can take more responsibility, what you have to check or repair, and what you would change next time. What could you do with the attention it frees up?

A leader can work through the same exercise with a PM. Instead of a general conversation about becoming better at AI, you have a piece of work to examine together, a capability to develop, and something specific to try again.

Mol: What makes that capability difficult to recognize when you’re hiring?

Wilton: Two people can have the same title and be working very differently. One uses AI to help with individual steps; another has agents owning parts of the workflow. The second person doesn’t automatically create more impact. Their environment may help them or hold them back.

I think about my own situation. I’m learning a lot by building independently, but I couldn’t assume my current setup belonged in a larger company. Its tools, data, security requirements, and ways of working would shape what I could use or rebuild.

A resume can miss what someone has learned outside a familiar role. An impressive agent setup can also leave important questions unanswered. I want to know what made the person effective there and whether they can adapt it here. Agent counts and token usage don’t answer that.

Mol: So, what are you using to hire? Anything new?

Wilton: I still use traditional resume screening and interviews. I rely a lot on my network for referrals. What’s new is that I’ve started using more structured take-home assignments for PMs. That has been my main “innovation” in how I hire PMs.

Mol: What can a take-home reveal if an agent can already complete the assignment?

Wilton: It needs to reveal how the person works with that capability. We can ask for more complex work within a clear time limit, with explicit expectations about the output and the tools they can use.

I’d also change one important assumption during the conversation: the engineering capacity has halved, or the customer evidence contradicts the recommendation. What does the candidate reconsider, what do they keep, and why? That gives me more to work with than a polished explanation of the original submission.

The candidate should know what good looks like and how much time is expected. The employer needs a useful task, criteria that fit the role, and the attention to review the work. The submission matters, and so does the conversation behind it. Interviews, experience, and references still have a role.

Mol: Tell me more about how you run these assignments.

Wilton: I start with a job description. My agent reviews it and suggests assignments based on scope, context, and seniority. I pick the assignment from my knowledge base that I find most relevant. Then I decide how long the candidate will have to complete it and whether to include an AI interview at the end.

I decided to create a whole product to run these assignments because the constraints matter to the outcome (that's RecruitSignal). Once the assignment is ready, I send the candidate a unique link, and they have 24 hours to accept it. They only see the task when they hit Start. The countdown begins then, and they must submit their answer within the allotted time, for example, 90 minutes. This gives them flexibility and time to set up their environment beforehand, while still requiring them to do their best work within the time available.

After submission, my agent evaluates the work and can optionally run an interview based on it. I supervise the process and receive a report with insights into the candidate’s work.

Mol: Where do you think take-home assignments are useful, and where would you use something else?

Wilton: For more senior positions, a take-home assignment will not be the best tool for evaluating impact. For people-manager roles, I would rely more on interviews. My personal experience has been with assignments for more junior roles.

Mol: If someone else wants to use these assignments, are you making them available?

Wilton: I’m sharing this work as an open personal research resource. You can browse the existing synthetic assignments, explore their work maps and judge what is useful in your context.

Appendix: 29 assignments and their work maps

The links open the workflows underlying these assignments. The assignment collection contains the candidate-facing synthetic exercises; model trial reports remain separate research evidence.

  1. Revise a product strategy for leadership review
  2. Investigate a drop in checkout conversion
  3. Write a requirements document ready for review
  4. Build a proposed product roadmap for approval
  5. Propose pricing and packaging for a new AI feature
  6. Build the business case for a monitoring service
  7. Plan a launch and report what is ready
  8. Propose a north star the team can trust
  9. Rollouts competitor analysis
  10. Conclude an A/B test and close it down
  11. Propose a priority order for the next quarter
  12. Decide a model change on evidence
  13. Write the opportunity brief for medication refills
  14. Get a product change through specialist review
  15. Prepare a leadership product review
  16. Size and segment a market and recommend target segments
  17. Recommend how to source invoice capture
  18. Prepare an industry analyst briefing
  19. Turn approved requirements into stories the team can build
  20. Recommend whether a release is ready to ship
  21. Run the monthly indicator review
  22. Q3 trend brief for product leadership
  23. Segment positioning proposal
  24. Plan a safe staged switch to a new alert engine
  25. Get price-alert tracking ready to trust before launch
  26. Prepare the portfolio review pre-read
  27. MVP scope for first customers
  28. Triage two weeks of feature requests
  29. Answer the product questions in a tender