Does agent memory hold up?
We’re re-running a recent agent-memory paper end to end, no-memory baseline included, before building anything on top of it.
In progressResearch lab in New York and Bangalore
Agents that can
show their work.
We’re a small lab working on the unglamorous parts of AI agents: the code around the model, the small models that run them cheaply, and the evidence that a job actually got done.
trace.replay
- runtask "book a qualified call"
- toolcrm.lookup(lead)212ms
- toolcalendar.propose(slots)340ms
- gatehuman approvalheld
- outcomecall_bookedclaimed
- verifyidentity, consent, dedupe3/3
- verdictVERIFIEDreceipt 9f2c…a41e
Research
What we’re working on
Six threads we keep pulling on. Some will turn into papers, some into products, and a few will be dead ends. We’ll tell you which.
Agent harnesses
Most agent failures we see aren’t the model’s fault. They come from the code around it: how tools are described, what gets remembered, when it retries, how it recovers. We treat that scaffolding as the thing worth studying. Right now that means rebuilding a published agent-memory paper from scratch to see which of its gains survive.
Which harness choices change success rates, and which only change the bill?
Small models
A small model tuned for one job can often keep up with a frontier model on that job, at a fraction of the cost. We’re mapping where that holds.
Where’s the crossover?
Simulation
Agents should break in a simulator before they break in production. We replay real runs so a regression shows up as a diff, not a support ticket.
What would a flight simulator for agents measure?
Agent security
An agent that reads email will, sooner or later, read one written to manipulate it. We study injection, leaks, and how to scope what an agent may do with what it reads.
Whose instructions should an agent follow?
Private agents
Some teams can’t send their data to someone else’s model, full stop. We work on agents that live entirely inside an organisation’s own walls.
What’s the smallest stack that’s still useful?
Agents with rules
Ad buying and banking have rules for good reasons. Agents working there need spend caps, sign-offs and an audit trail long before they need a personality.
- Advertising
- Banking
- Regulated operations
- Agent analytics
What can an agent propose, and what should always wait for a person?
Featured program
Paid for outcomes.
But who checks?
More and more, agents get paid when something happens: a call gets booked, a lead qualifies, a case closes. It’s a better deal for buyers, right up until the two sides disagree about whether it happened.
Today those arguments get settled with spreadsheets and goodwill. We’re working on a neutral way to check: one both sides can read, neither side controls, and that doesn’t make money by saying yes.
- 01
Agree
Both sides write down what counts as done, in terms a machine can test.
- 02
Record
Evidence is captured while the work happens, not pieced together afterwards.
- 03
Check
Independent checks run against the agreement. The checker earns nothing from a yes.
- 04
Sign
The verdict is signed, so either side can keep it and anyone can audit it.
- 05
Settle
When people still disagree, there’s a set path, and it runs on evidence.
Still in research. A public preview will live on its own site, and we’ll link it here when it’s up.
Notes
On the bench
Things we’re in the middle of. Write-ups go up when there’s something worth reading.
Think smaller
For most narrow tasks, a frontier model is the wrong default. We’re trying our best to prove ourselves wrong.
In progressEvery run is a dataset
We log tool calls, retries, and every time a person stepped in. If we can’t replay it, we don’t count it.
In progressHow we work
House rules
How we decide what’s worth saying out loud.
Measure first
We define the task, the baseline and what failure looks like before we say anything about capability. A number without a method doesn’t make it onto this site.
Fence the agent
An agent inherits the rules of the place it works: spending limits, approvals, logs. Handing it every tool and hoping for the best isn’t a plan.
Keep the receipts
Every run is recorded, so someone else can step through what the agent saw, called and decided.
No borrowed credibility
You won’t find invented benchmarks, papers or client logos here. When we publish, it’ll show up on this page.
Got a hard agent problem?
We like working with people who have a real one: an evaluation that won’t behave, a domain with strict rules, a pilot that has to stand up to scrutiny. Tell us about it.