Thirteen months ago we published the first of the four futures we believe in: a world where an enterprise can hire and manage any digital specialist while keeping visibility and control over what those specialists produce. That post laid out a lifecycle. Plan which agents the business needs. Hire them. Manage them once they are running. We named companies building each stage: Superintelligent for planning, Orita for the specialist work itself, TensorStax for autonomous data engineering before Snowflake acquired it, Wayfound for supervision, and WARP as the freight network already running that way.
The lifecycle has held up. What we underweighted is what happens on the other side of it.
An agent that has been hired does not do its work in a vacuum. It works inside software, and much of that software is another company’s product. Every agent an enterprise hires becomes a new user pointed at a product that was built for people.
The population that changes is much larger than the population doing the hiring. It is every company whose product those agents reach: the API, the SDK, the documentation, the CLI, the dashboard. Those companies may not have deployed an agent and may have no AI strategy at all. Now, they have acquired a second class of user they never designed for, and their user count is now humans plus agents while their product is only pointed at one of the two.
The user who never complains
A person who cannot work out how to use your product will open a support ticket, write a review, post to social, ask a question, or leave in a way that eventually shows up in a dashboard. Every feedback channel a software company has was built around a human who gets stuck and says so.
An agent does none of that. It tries, hits an obstacle, works around it, and finishes the job somewhere else. Renzo Viale and Haritha Nair, who founded Oqoqo, name the failure mode precisely:
“Just because your MCP was available doesn’t mean the agent used it, and just because the agent got the right answer doesn’t mean your product helped it get there.”
The user was served and the task was completed, but your product may not have been part of how it happened, and nothing in your telemetry will tell you.
So a product can start losing agent traffic while every dashboard it owns stays green.
The instinct is to check the dashboard
A company that starts to suspect agents are using its product will usually look in two places, and neither answers the question.
The first is its own telemetry. A request log shows traffic, not outcomes. It cannot tell you whether the agent finished what your user asked, or whether it got halfway, abandoned your endpoint, and completed the job another way. A clean 200 is perfectly compatible with the agent giving up on you.
The second is a quick test by hand. Somebody points a coding agent at the product, watches it work, and concludes things are fine. One run is an anecdote. Agents are non-deterministic: the same instruction takes different paths on different days, and the run you watched is not the run your users get. To learn anything you have to run the same task repeatedly, hold everything else constant, and change one variable at a time. Running one task once is not that experiment; a real one is a suite of them, and most teams have no way to run even a single suite.
Public benchmarks do not close the gap either. They rank models against a fixed set of tasks, which serves the frontier labs and tells a product company very little. As Haritha put it at launch:
“Most benchmarks today exist in curated environments and do not translate well to the real world.”
Your product is not a curated environment. An agent that meets it has to work through your codebase, your permissions model, your API, your documentation, and whatever internal tooling has accumulated around them.
What Oqoqo does
Oqoqo is where a team runs that suite. You describe a real task in plain language, the kind of instruction one of your users would actually give: integrate Supabase into my web app to store user signups. You hand Oqoqo the environment an agent needs to attempt it, meaning project state, files, data, tools, and the services it has to call. You define what success means, down to specifics such as whether row-level security was configured. Oqoqo then runs that task across whichever models, agents, harnesses, or versions of your product you want to compare, each trial in its own sandbox, from the same seeded state every time.
The distinction Renzo and Haritha draw is between evaluating an answer and evaluating the work. Traditional evaluation assumes a model that produces output you can score. Agents do not behave that way. They call tools, modify files, navigate repositories, and make long sequences of dependent decisions where each choice narrows what comes next. Scoring the final answer misses nearly everything that produced it.
So Oqoqo reports the work: pass or fail on each requirement with a written reason, plus the full trajectory of every run, covering which tools the agent called, where it retried, what it errored on, which files it changed, how many tokens it spent, how long it took, and the points where it gave up on your product and went around it. Teams wire it into CI, so a change that breaks an agent workflow surfaces the way a failing test does.
The surfaces you can point it at are the ones agents actually touch: MCP servers, CLIs, SDKs, APIs, documentation, and skills.
How they got here
We led Oqoqo’s pre-seed in November 2025 out of Bee IV. Renzo and Haritha are both Berkeley Haas; Renzo spent a summer at Bee as an intern before starting the company.
Originally, they worked on documentation. The premise was that developer docs written for humans are the wrong substrate for an agent, and that any company serving developers would need its documentation rebuilt so a coding agent could read it, use it, and integrate without a person in the middle. The premise still holds, and it was an early version of the same idea: your product has a reader now who is not a person.
What changed is how far it went. Working alongside DevRel teams, Renzo and Haritha kept asking a question nobody could answer: how do you know the documentation and skills you ship are actually helping an agent? The answers came back anecdotal. So they built experiments to find out, running dev-tool products across different agents, with and without skills, and the results were unflattering. Some skills made agent behavior worse. Some inflated token consumption. Often the agent worked around the product, corrected itself, and arrived somewhere suboptimal, because the interface was never usable enough for the task to begin with.
Documentation was one symptom. The wider problem was that no team could tell whether its product was usable by an agent at all, because no instrument existed to ask. The experimentation surface Renzo and Haritha had built for their own use was the more valuable thing, so they opened it up, and documentation became one of several surfaces it measures.
We back people at pre-seed, before there is enough product to evaluate. What gets underwritten at that stage is judgment. Renzo and Haritha followed their own evidence to a larger problem within months of our investment, and did not spend that time defending the original plan. Following their own evidence like that is how a pre-seed company finds product-market fit.
Two bets, two layers
Bee now holds two positions in agent evaluation, and they sit on opposite sides of the same shift.
Okareo measures the agents you deploy. It generates synthetic users that call and converse with your customer-facing agents the way real customers would, voice in particular, finding the edges before your customers find them, then evaluates and monitors what happened. GEICO is among its customers.
Oqoqo measures the agents that arrive: the ones your users and your customers’ enterprises point at your product, and whether your software holds up when they get there.
One is about how your agents treat your customers. The other is about how everyone else’s agents treat your product. Both problems get harder as the number of agents in production climbs.
Last July we wrote that Bee had invested in the management layer before the market was ready to receive it. That layer has an order to it, and measurement comes first, because it converts a hunch about agent behavior into a number a team can act on. With Okareo and Oqoqo, a company can now answer how agents perform on both sides of its business. Management is the half that follows, and it is close behind: once performance is measurable, supervising a hundred agents becomes a governance problem instead of a guessing game. Wayfound is already building for that.
Oqoqo is live at oqoqo.ai and free to try. Point it at your own product and find out what the agents have already worked out about it.