What AI product management actually looks like, day to day

The version you see at conferences

AI PM talks usually cover strategy: how to prioritise AI features, how to measure trust, how to handle hallucination risk. All of it is real, and none of it tells you what a working week actually looks like.

I work on Operate, Better's enterprise AI platform for production operations at operatex.dev. Here is a more honest account of what that involves.

Discovery looks different when the output is probabilistic

Most product discovery follows a predictable arc: you learn what users need, you build something deterministic that meets that need, you measure whether it worked. AI products break the middle step. The thing you build does not always produce the same output for the same input. That changes how you frame requirements.

On Operate, a significant part of my discovery work is about understanding what the acceptable variance in output looks like for each use case. Not "what should the AI do" but "what should the AI never do, and how bad is the worst plausible output." That shapes the guardrails before a single line of model code is written.

The tools are the same — user interviews, journey mapping, success metrics — but the questions are different. "How would your team respond if the AI said X instead of Y" is a question I ask more often than I expected to at this stage.

Guardrails are a product decision, not an engineering decision

There is a tendency in AI teams to treat guardrails as an engineering problem: add a classifier, add a filter, add an eval. Those are implementation details. The product question is: what is the trust contract between the user and the system?

FeatureSmith taught me the clearest version of this. The "AI explains, never decides" principle I used there — where every finding is computed deterministically and the AI only rewrites findings in plain language — was a product decision, not a technical one. The engineering followed from it.

On Operate, the same principle applies with more complexity. The system can surface insights, rank priorities, and flag anomalies. It cannot take action without a human confirming it. That constraint is written in the product spec before it is written in the code.

The work that does not make it into case studies

A lot of what I actually do is less strategic and more operational: reviewing AI output samples to calibrate the eval suite, writing and rewriting the prompts that inform how the system explains its outputs to users, sitting with support tickets that describe the model doing something unexpected and working backwards to understand whether it is a prompt issue, a data issue, or a genuine model limitation.

This is not glamorous. It is also the most important work, because it is where trust in the system gets built or lost. Samsung R&D's CV work achieved 90%+ accuracy in part because someone was paying attention to the failure cases, not just the aggregate score.

What makes it worthwhile

Enterprise users of AI tools are not impressed by the demo. They are impressed by consistency. The question they are asking is not "can your AI do this" but "will it do this the same way every time, and when it does not, will your team tell us."

The PM's job in that environment is to make that promise credible — by building the evals that catch regressions, by writing the release notes that explain what changed and why, and by being honest when the model is not ready yet.

If you are building an AI product and need PM-led discovery or an AI integration done in a way your team can trust, [see how I approach product strategy](/hire/product-strategy) and [AI integration](/hire/ai-integration).