Senior Technical Program Manager · Bay Area

I build the paved road for agentic work.

Power users find the tools. Organizations need a default: one path that is easier than the old way, governed enough for a payments company, and honest about what shipped. I design that operating model, I set the evals that decide what gets keys, and I run the models myself, including open-weight ones on my own laptop.

Mohit Dogra · Walnut Creek / Bay Area · 12 years, 10 at Visa

10→80%
adoption in the product org went 10% to 80% in six weeks on the paved road
~3 mo → ~10 d
time to first working agent, guild era versus the paved road
20+
generative-AI products through eval and rollout gates matched to blast radius
$1,500
per-user AI cost cap set up front, with a business-justification exception
$1B+
Visa Direct API platform; one plan across 60 engineering teams and 10+ orgs
~$80M
annual spend across 20+ programs at 80% delivery predictability
3 orgs
and 500+ engineers on the operating model I designed from scratch

Case studies with named scope

Six builds. Scope is named carefully. Engineering built skills, connectors, and the pipeline; I owned the program design, adoption path, eval rules, and measurement. Scroll the rail.

Product delivery
Product delivery Visa Direct

A decade of Visa Direct delivery

Concept-to-market delivery on a $1B+ API-first payments platform: one integrated plan and OKRs across 60 engineering teams and 10+ stakeholder orgs, sitting next to product without owning the PRD.

  • Argentina market launch from concept to GA against a Central Bank deadline: 30+ engineers across product, engineering, compliance, and regional teams. Landed on the required date.
  • Portfolio of 20+ programs, about $80M annual spend, held at 80% delivery predictability.
  • Operating model designed from scratch (intake, prioritization, capacity, risk and dependencies, review cadence), adopted across three orgs and 500-plus engineers.

Outcome: 30% YoY growth on the platform; dates held under real constraints; cadence other delivery managers could run after handoff.

Agentic AI
Agentic AI Program design

GenAI guild to a governed agentic toolchain

The guild proved AI worked for power users. It did not scale. I defined and led the program for one paved road: one-command setup, identity that non-engineers could use, MCP connectors that inherit the user's own permissions, a human merge rule, evals before keys, and telemetry that classified shipped work rather than vanity usage.

  • Program design was the job (~80% of what I owned). Platform and engineering built skills, connectors, and the pipeline.
  • Two skills, one contract: a product pipeline that carries a raw problem to a PRD, and an engineering skill that reads the PRD and opens a PR. A human merges, always.
  • I did not write CODEOWNERS, own security controls, or build the platform. I owned the rules, the rollout path, and the honesty layer on usage.

Outcome: time to first working agent from about three months (guild era) to about ten days; now the Visa-wide pattern, used by thousands including non-engineers.

Adoption
Adoption Product org

Product org: 10% to 80% adoption in six weeks

Adoption in the product org went 10% to 80% in six weeks. Not company-wide. It happened on the paved road, and it held week over week. Mandate was not the lever. Making the default easier than the old path was.

  • One command to start, skills for work teams already did, a corp-ID path so product people were not locked out by developer-ID gates.
  • Champions and pods: exit was a merged PR, not a slide. Their engineer demoed to their EM.
  • Telemetry for stuck teams (missing connector, quiet champion, bad output) visible in days.

Outcome: adoption in the product org went 10% to 80% in six weeks; this is one org, not a company-wide number.

Evals
Evals Launch gates

The test track 20+ GenAI products had to clear

Leadership got excited about a new model most weeks. I turned that into a rule both sides could trust: beat the incumbent on our test track, or no keys. A one-week benchmark replaced a mandate.

  • Several shiny things failed the gate. Sometimes the new candidate won, and we adopted it. The point was a test, not a veto.
  • After one confidently wrong output stalled a team for weeks, evals moved ahead of any user exposure, and critic checks sat after every pipeline stage.
  • Gates matched to blast radius: an internal summarizer is not treated like anything that touches a transaction. Security and compliance were in from the start.

Outcome: 20+ products (client-portal chat, natural-language BI, document extraction, and compliance prep among them) reached production through the gates. I owned the rule and the gates; teams built the products.

Vendors & cost
Vendor evaluation AI cost governance

Buying AI tools without buying a cost problem

Ran build-versus-buy evaluations on AI tooling and negotiated contracts with outside vendors (Replit, Perplexity), with legal, security, and procurement in the path on pricing and data handling.

  • Judged offerings on technical fit, evals, and model and token economics, not on the demo.
  • Set a $1,500-per-user cost cap before usage grew, with an exception for heavy users who could show business value.
  • Public data later showed what happens when cost governance arrives late: licenses get pulled. We did not wait for that lesson.

Outcome: spend governed from day one; the tail stayed in check while high-return users went further.

Migration
Platform delivery Migration

Platform and migration delivery

Separate from the toolchain. Monolith-to-microservices and CI/CD with canary rollouts across 10+ production applications at six-nines availability. I argued against a big-bang cutover; a 5% canary caught client-specific errors no happy-path test hit.

  • On the toolchain, the quality tell was review latency and change-failure staying flat as volume rose.

Outcome: faster, safer release path on migration programs; clear wall between those numbers and AI adoption metrics.

Experiments, with the results

Each card is a question I actually ran down: the setup, what was measured, what came back, and what I would change. Some ran at Visa. Some run on my own laptop. Anything not finished is marked In progress. I don't write production code; I design the test, run it, and read the results.

L1 Done · Sep 28, 2026 Laptop · open-weight

How good is a 3-bit open-weight model on a laptop, really?

Where can a free local model carry real work, and where does it quietly fail?

Setup
Qwen3.8-Flash-Next, Unsloth UD-Q3_K_XL (3-bit) quant, served by llama.cpp's llama-server on a MacBook Pro (M5 Max, 128 GB), 32K context. 31 model calls across 9 categories: document extraction from two real SEC 8-K filings, finance math, factual recall and hallucination probes, current events without search, reasoning, rubric-based triage, writing, strict JSON instruction following, and low versus high reasoning effort. A frontier model graded every answer and fact-checked against live sources. A small runner script logged every output, the hidden reasoning, and timings.
Measured
Correctness against a verified reference, a 1 to 5 grade per category, tokens per second, and wall time.
Result
Strong reader, unreliable source of facts. It pulled 48 of 48 required deal terms correctly across 4 extraction runs (and once invented a per-share figure), solved 6 of 6 reasoning runs, and passed 7 of 7 JSON checks. It got 0 of 3 current-events facts right (it did flag its mid-2025 cutoff), invented a mountain's height with confidence, attached a false state-tax claim to otherwise correct math, and returned nothing on a 30-year amortization in three tries. High reasoning effort was 2 to 7 times slower with no accuracy gain. Unprompted, it used em dashes in 13 of 31 answers; told not to, it complied.
I'd change
One sample per prompt makes this indicative, not statistical; the next run uses several samples per prompt. Hand long multi-period math to a calculator tool. Default to low effort for extraction and triage.
48/48deal terms extracted from real 8-Ks
6/6reasoning runs correct, any effort
0/3current facts right without search
~40 t/smedian generation on the laptop

Full scorecard and the rules I now use it by: Evals deep dive · Open-weight models

L2 Running Laptop · open-weight

A scheduled local agent that triages feeds against a written rubric

Can a local model do first-pass triage on a daily feed, so I only read what clears a bar?

Setup
A two-stage pipeline on the laptop: stage one scores items in batches against a plain-language rubric (0 to 10, with explicit exclusions); stage two deep-reads the high scorers and writes a short brief. No API keys. It only surfaces ideas; it never acts.
Measured
In the L1 eval: its rankings and scores on real items versus a strict frontier read of the same rubric.
Result
Rankings matched. Scores ran 1 to 3 points generous. It caught a real double-count bug in the pipeline's own input data (amended filings counted twice). Given a truncated source, it filled the gap with optimism; given the full text, it scored sensibly.
I'd change
Deduplicate amended records before scoring. Never truncate the source to save tokens. Calibrate the rubric with anchor examples so "generous" becomes measurable drift.
V1 Shipped Visa

Beat the incumbent, or no keys

How do you decide a new model or tool gets access without turning it into a rank fight?

Setup
One test track. Every candidate model or skill runs against the incumbent on the same track, with the gate set by blast radius.
Measured
Candidate versus incumbent on the track, before any keys were issued.
Result
Several shiny things failed. Sometimes the candidate won and we adopted it. Leadership enthusiasm became a one-week benchmark instead of a mandate.
I'd change
Install the harness before the first disagreement, not during it.

Scope: I owned the rule and the gate. I did not write the harness.

V2 Shipped Visa

Critic checks after every pipeline stage

After one confidently wrong summarization stalled a team for weeks, how do we stop the next one before a person sees it?

Setup
The product pipeline runs eight stages. It opens with Stay in the Problem (who hurts, what is the evidence, what does doing nothing cost), then works backwards: press release, FAQ, and on to the PRD, which comes last as the assembler. An adversarial critic reviews each stage's output before the pipeline advances.
Measured
Whether a weak artifact can pass forward. It cannot: it has to survive a stage and a critic first.
Result
Evals moved ahead of any user exposure. By the time a PRD exists, every claim in it has already survived a critic. Trust is asymmetric; this is where you protect it.
I'd change
Critic loops burn tokens: set budgets per stage, track reviewer fatigue on the human side, and add checkpoints so a failure at stage six does not restart stage one.
V3 Shipped Visa

Telemetry that tells shipped work from usage

Is the agent actually shipping work, or are people just chatting with it?

Setup
Every session gets a correlation ID that follows the work: session, then artifact (a PRD or brief), then merged PR. Metadata only, never prompt or code content. Per-team curves, not just the org average.
Measured
Daily active as real invocations, time to PR as a percentile, started versus finished, and two leaks: sessions with no artifact (a quality problem) and artifacts with no PR (a trust problem). A "productive dead-end" class credits sessions that correctly killed a bad idea.
Result
The data showed PMs routing around the eight-stage pipeline for small work for about a month. That became Ship the Feature, a one-page quick lane with the same guardrails. Flatlined teams showed up in days, not at a QBR.
I'd change
Instrument the funnel in week one, ship the second lane on day one, and audit how often the ID survives each hop, or the funnel quietly undercounts.

Scope: I designed what the telemetry had to answer. Platform built the warehouse.

V4 Shipped Visa

Two-person AI pods before any platform

What does the platform actually need to be, before we build it for thousands?

Setup
Pods of one senior (judgment against confidently wrong output, credibility with skeptics) and one junior (hours in the tool, no habits to unlearn). Each pod took a real feature end to end. Exit test: a merged PR, not a demo.
Measured
What blocked each pod, and whether the new-issue rate was flattening.
Result
Pods surfaced the real blockers: GitHub seats ran out (supply, not demand), a developer-ID wall, and skill gaps. Fixing those is what made the 10 to 80 possible. Early breakage hit two people, not a sprint.
I'd change
Front-load trust work, and plan for non-engineers from day one. CLI-first was a mistake; the biggest untapped demand was outside engineering.
V5 In use My own TPM work

A planning agent and a ticket-quality agent for my own work

Can an agent eat a program manager's worst status toil without owning the date?

Setup
Set up on Claude Code and MCP, with skills, hooks, and evals. The planning agent reads program plans, flags likely slips, and drafts re-plans when scope or dates move. The ticket-quality agent checks whether work is ready before it starts, because work starting before it is ready was our single most expensive failure.
Result
Not quantified for my own use. At team scale the same idea became an intake readiness check that scored every request before work started, which cut rework. It also became my coaching move: give a traditional PM an agent that eats their worst status toil, and they convert themselves.
I'd change
Keep it a copilot for the TPM, not an autopilot for the date. These personal agents are not the Visa platform.
L3 In progress Laptop · open-weight

Local Qwen versus a frontier model for first drafts

Is a free local model good enough for heavy first drafts, so paid models only do research and checking?

Setup
Same brief, two drafts: one from local Qwen, one from a frontier model, for a real one-page resume tailored to a Program Manager role. The local model now handles my first drafts of posts, stories, and message variants.
Measuring
Factual drift against a master fact sheet, echoing the job post's wording, dash use, and how much editing each draft needs.
Result
Not written up yet. Both drafts exist; the scored comparison is next.
L4 In progress Laptop · personal

An eval plan for an AI shopping agent's checkout

What would I need to see before letting an agent pay on someone's behalf?

Plan
50 test scenarios with a pass/fail rubric, 10 red-team cases (prompt injection, over-spend, wrong merchant), and launch-gate thresholds. Run by hand against local Qwen first.
Result
Not run yet. The gate shape is sketched in the agentic commerce section.

If you cannot write an eval for it, you cannot safely delegate it

An eval is a test for a model or agent, the same way a unit test is a test for code. Freeze the inputs, define what good looks like, run the candidate, score it, and promote it only if it beats what you already have.

A demo picks friendly prompts. An eval picks the cases that bite you. One confidently wrong output costs more trust than a hundred good ones buy back.

That is why the rule became evals before exposure: no model or skill touches a person until it clears the track.

  1. 1Offline: a frozen test setBefore any person sees the new model or skill. Hard assertions plus a graded rubric, same harness every time.
  2. 2Online: shadowOptional. Run in parallel on live traffic, compare, no user impact.
  3. 3Human gateAnything irreversible or customer-facing still needs a person to click.
  4. +After merge: telemetryRevert rate, change-failure, and review latency tell you whether the system got better or just busier.

Example launch gate

The shape I hold any candidate model or skill to. Thresholds are set per product and matched to blast radius.

CheckTypeBar to pass
Hard assertionsMust passHigher pass rate than the incumbent on the same frozen set. Any secret or PII leak is an automatic fail.
Graded rubricScoredAt least equal to the incumbent, scored by a pinned judge at zero temperature.
Regression sliceMust passEvery case that already burned us still passes. This list only grows.
Cost and latencyScoredWithin the agreed margin, shown side by side with the incumbent.
Blast radius tierHuman gateInternal summarizer: the offline set is enough. Anything that can touch a transaction, a permission, or a customer: a person approves.

Example rubric: natural-language BI

Natural-language BI was one of the GenAI products that went through the gates. BI fails in a specific way: a confident wrong metric in a pretty chart. These cases are written for this site to show the method; they are not Visa's internal test set.

CaseTypePass means
Synonym collision (revenue, recognized revenue, GMV)HardResolves to one catalog metric ID, or asks. Never invents a definition.
Grain mistake (daily versus monthly)HardTime grain matches the ask exactly.
Filter leakHardAsked for segment A, the answer contains only segment A.
Incomplete dataRubricSays the answer is partial and shows row coverage. Does not smooth over gaps.
Year over year with a missing monthRubricNo smooth narrative without a caveat.
Access probeHardA user without permission for a segment is refused. Access never widens.

Gate order: metric correctness first, then narrative quality. Metric wrong plus a pretty chart is a fail.

How I design the default

Light enough to survive handoff. Strict enough for a payments company. Click through the paved road, or use arrow keys when the toy is focused. The full transformation playbook sits underneath.

The paved road

Power users → Evals → Rings → Kill signals → Measurement. Play it like a system diagram you can walk.

    01

    Evals before exposure

    No keys until the offering beats the incumbent on a real test track. Shiny tools fail that gate on purpose.

    02

    Rings and kill signals

    Cohorts before company-wide. Kill criteria written before rollout, not after the first bad week.

    03

    Measure shipped work

    Invocations that become artifacts, time to PR, started versus finished. Usage alone is vanity.

    04

    Cadence that survives handoff

    Standard artifacts, decision logs, coaching into the seat. If only I can run it, it is not done.

    AI transformation playbook

    What I actually did to take agentic tooling from a guild to a governed default, in order. Each play is something I ran, not a slide.

    01 · Instrument first

    Find where time really goes

    When we measured, the sinks were not coding. They were intake (work starting before it was ready), waits (review latency, flaky CI, environment setup), and archaeology (finding who owns what and why).

    • We tried headcount, templates, and better docs first. Agents went where judgment and context were the bottleneck.
    02 · Proof, not a pitch

    Run it yourself, then ask

    When a new model generation changed the math in early 2026, I ran the experiments myself and took working proof to a leadership showcase. That won the rollout mandate.

    03 · Pods before platform

    Two people, one real feature

    Senior plus junior, a real feature end to end, exit test a merged PR. Pods told us what the platform had to be, and kept early breakage to two people.

    04 · Remove every reason to say no

    Access is usually the real blocker

    • Made the license supply case to the central platform team; seats expanded.
    • Negotiated corp-ID sign-in with security, scoped to one knowledge-base repo: no path to source, build, or production.
    • One command to set up. Connectors inherit the user's own permissions.
    05 · Rings on evidence

    Expand when the data says so

    Next ring only when the new-issue rate flattened, incoming problems were repeats we had answers for, and time to first agent held around ten days. Evidence, not calendar.

    06 · Champions

    One per team, trained by pod alumni

    Same playbook everywhere: setup, one real feature through the toolchain, office hours. Each team's adoption visible on a telemetry scoreboard, plus a shared wins channel. Social proof, not mandate.

    07 · Diagnose stuck teams

    Three causes, three fixes

    A flatlined team was always a missing connector, a quiet champion, or one bad output. Per-team curves made it visible in days.

    08 · Govern cost early

    A budget with an escape hatch

    $1,500 per user, set before spend grew, with an exception for users who could show business value.

    09 · Standardize vs federate

    Two repos, two trust levels

    Standardize where blast radius is shared: install, identity ceiling, connectors, eval gate. Federate where local context is the value: teams contribute skills through a pull request and core-team approval.

    Kill signals I write before the first cohort

    • Review latency climbing as volume rises
    • Change-failure drifting up
    • A stuck team invisible for more than a few days
    • Sessions that never become artifacts (quality leak)
    • Artifacts that never become PRs (trust leak)
    • Every GenAI initiative runs as a bet with a metric, against a published kill list

    What done means, written down before scale

    • Adoption: daily active as real invocations, with retention so a spike does not count
    • Leverage: time to first working agent, about three months down to about ten days
    • Honesty: time to PR as a percentile, started versus finished, review latency and change-failure next to throughput
    • Credit: a session that correctly killed a bad idea counts as success
    • Not counted: lines of code generated, seats, prompts, % using AI

    Agents need delegated authority, not your password

    Payments has spent fifty years letting one party act with someone else's authority, safely. Agentic commerce is the same problem with a new actor. Here is how I read the emerging protocols, in plain language.

    When an agent shops for you, four questions decide whether it is safe: what is it allowed to do, how do you take that back, who did what, and how do you undo a mistake. Handing an agent your login answers none of them. Delegated authority answers all four.

    No single protocol covers the whole trip. One lets the agent reach tools and data. One lets a merchant tell a trusted agent from a bot. One proves the person approved this purchase. One runs checkout inside a chat. One standardizes the shopping journey end to end. They stack; they do not compete.

    The test I would put on any of them is the one I used on agents writing code toward production at Visa: the agent never gets more authority than its person has, it never approves its own action, and every step leaves a record you can audit.

    ProtocolLed byThe question it answersWhat gets signed or scoped
    MCPAnthropic, open standardHow does an agent reach tools and data?Which tools and data an agent can call. Done well, the agent inherits its user's own permissions and nothing more.
    TAPVisaIs this a trusted agent with real commerce intent, and who is it shopping for?The agent signs each request (HTTP Message Signatures, RFC 9421) with a short expiry and a one-time nonce; the merchant checks it against published keys. Linked, signed objects can carry consumer recognition and payment data.
    AP2Google and partners, openDid the person actually authorize this purchase?Mandates as signed credentials: a checkout mandate and a payment mandate. Open mandates carry limits such as a budget; closed ones bind to one specific checkout.
    ACPOpenAI and Stripe, openHow does checkout happen inside a chat?Delegated payment through a token scoped to one merchant and a maximum amount, so card details never reach the agent.
    UCPGoogle and partners, openHow does the whole journey work, from discovery to checkout to orders?Merchant capabilities the agent can discover and call. Designed to work with AP2 for the payment authorization step.

    Sources: Visa Trusted Agent Protocol specifications (developer.visa.com); AP2 specification (github.com/google-agentic-commerce/ap2); Agentic Commerce Protocol (github.com/agentic-commerce-protocol); ucp.dev; modelcontextprotocol.io.

    ScopeWhat exactly may the agent do: which merchant, how much, until when.
    RevocationHow the person takes authority back, quickly and for certain.
    AttributionWho did what, provable after the fact by the merchant, the network, and the issuer.
    RollbackWhat happens when the agent is wrong: dispute, refund, liability.

    What I run on my own laptop, and why

    I keep an open-weight model running locally for heavy drafting and document work, and I tested it before trusting it. The rules below come from that test, not from a leaderboard.

    CostFree first drafts. Paid frontier models are saved for research, tailoring, and verification.
    Stays on the machineThe server listens only on localhost, so prompts and documents do not leave the laptop.
    Know the failure modesRunning a model yourself is the fastest way to learn where it goes wrong. That makes me better at governing the ones teams use.

    Current setup

    Model
    Qwen3.8-Flash-Next
    Quant
    Unsloth UD-Q3_K_XL (3-bit)
    Server
    llama.cpp llama-server, 32K context, all layers on GPU
    Front end
    Open WebUI for chat; plain HTTP for scripts
    Machine
    MacBook Pro, M5 Max, 128 GB
    Speed
    36 to 45 tokens/s generation; reads a 10K-token filing in about 12 to 14 s
    Default
    Low reasoning effort for extraction and triage, medium otherwise

    Trust it for

    • Pulling terms out of a document I paste in
    • Logic, planning, and checklists
    • Strict JSON output
    • First drafts (with "no em dashes" in the prompt)
    • Skeptical first-pass triage

    Double-check

    • Any number it computes
    • Tax, legal, and benefits rules
    • Obscure facts not in my document
    • Rubric scores (it runs generous)

    Do not use it for

    • Anything current: rates, office holders, recent deals
    • Long multi-period math; it loops and returns nothing
    • Final decisions without a second source

    Short pieces from the work

    Not a blog farm. Three longer essays, then four short notes drawn from themes I actually ship against.

    Builds · Agentic paved road

    From power users to a governed default

    A guild proves the tech works for people who already know how to wrestle it. A paved road is different: one-command setup, identity non-engineers can use, connectors scoped on purpose, and a human merge rule. Program design owns the rules and the rollout path. Engineering owns skills, connectors, and the pipeline. Confusing those two jobs is how you get a platform nobody adopts.

    Theme from the GenAI guild → toolchain program

    Notes · Evals & rings

    Evals before exposure, kill signals before hope

    Keys are the scarce resource. Beat the incumbent on a real test track or no keys. Write kill criteria before the first cohort, not after the first bad week. Rings keep the blast radius honest. When volume rises, the quality tell is review latency and change-failure staying flat, not a spike in chat messages.

    Theme from rollout design behind the paved road

    POV · Kill signals

    Agentic kill signals are a design artifact

    If you cannot name what would make you stop the rollout, you are hoping. I write kill signals with the same seriousness as success metrics: review latency climbing, change-failure drifting up as volume rises, stuck teams invisible for more than a few days. Those lights belong in the program design, not in a postmortem.

    Voice from governing agentic rollouts

    POV · Measurement

    Measuring shipped agent work, not chat

    Invocations are cheap. Artifacts are not. I care about invocations that become a PR, a ticket closed, a decision logged. Time to first working agent. Started versus finished. If your dashboard celebrates message volume, you are optimizing for the wrong dopamine. Measure what left the building.

    Honesty layer on agent telemetry

    Who is behind the work

    Mohit Dogra is a Senior Technical Program Manager based in Walnut Creek, Bay Area, with 12 years of experience, 10 of them at Visa (2016 to 2026). A decade on Visa Direct delivery and portfolio leadership, then the GenAI guild and the governed agentic toolchain.

    He designs operating models that take agentic tooling from power users to a governed paved road: evals before exposure, rings and kill signals, and measurement on shipped work. He runs the models himself, including open-weight ones locally, and he is studying the agent protocols (MCP, ACP, UCP, AP2, TAP) that will decide how agents get authority to act and to pay.

    Scope is named carefully. No inflated platform-build claims. Metrics stay with the program that earned them.

    MBA, Operations & Supply Chain · Clarkson University ML Specialization · Stanford & DeepLearning.AI Applied AI in Business · UC Berkeley Certified ScrumMaster