The pitch for most AI tools is that they make you faster at the thing you were already doing. You open a window, you type, you read, you copy something out, you close the window. It is genuinely useful and I use them all day. But at the end of it you have still been sitting there the whole time, driving.
The test I care about is different: what is different when I come back?
That is not a chat window. That is a fairy.
The tooth fairy is a good product
Bear with me. The tooth fairy has an unreasonably good user experience:
- You do one small, obvious thing — you leave something out.
- You do not fill in a form, describe the tooth, or schedule a call.
- You go to sleep.
- In the morning the thing is dealt with, and there is evidence.
There is no interface. There is a convention. You leave work out, and it gets done, and there is proof it happened. Every good agent product I can imagine is some version of that shape, and almost none of them are built that way — they are built as a chat box with more buttons around it.
What “leaving it out” actually means
In practice the thing you leave out is a mess, not a spec. It is:
“Sort out the invoice thing with the studio, I think it’s in my email somewhere, and it’s been going on since March.”
That is not a prompt in the way we have been taught to write prompts. There is no context, half the nouns are wrong, and the actual task is buried. But it is exactly how you would brief a competent human, and a competent human would go and find the rest.
So most of the work in building this has not been the clever part — the model is the clever part and I did not build it. It has been the boring machinery around it:
- Finding the invoice thread without being told which mailbox, label, or date.
- Working out that “the studio” is a company you have emailed forty times.
- Noticing that “since March” means there is a history, and reading it before doing anything.
- Deciding which of the six possible next actions is the one you meant.
Every one of those is a small, unglamorous retrieval problem. Together they are the difference between a demo and something you would actually leave work out for.
The morning note is the product
Here is the thing I got wrong for the first month: I thought the output was the completed work. It is not. The output is the note.
If you come back to an inbox that has been reorganised by something you cannot interrogate, you do not feel served, you feel burgled. You immediately start auditing. And auditing takes longer than just doing it yourself, which means the whole thing was a net loss.
So the note has to do real work:
- What was finished, in specifics, with links.
- What was started and is waiting on someone else.
- What needs a yes from you, and why she did not just do it.
- What she deliberately did not touch, and why.
That last one turned out to matter more than everything above it. A tool that tells you what it decided not to do is a tool you can start to trust, because it is showing you its judgment, not just its output. When she leaves the contract alone and says “this one is above my pay grade”, that is a better trust signal than ten completed tasks.
Still very much a work in progress
She is not finished. The parts that work well are the retrieval and the note. The part I am still fighting is knowing when to stop — an agent that is too keen is worse than one that is too cautious, because you only need one confidently-sent email to undo months of goodwill.
More on that, and on why I gave her a name, next time.