--- title: The chat on this page is now a product description: "I built it so my site could answer questions about my work while I was asleep. Then I spent two months on evals so the answers would be right. Not a demo. What it does, what it costs per message, and where it is still early." date: 2026-09-17 tags: [ai, build-in-public, sondo, evals, claude-code] url: "https://mahdif.com/blog/the-chat-on-this-page-is-now-a-product/" --- # The chat on this page is now a product I built it so my site could answer questions about my work while I was asleep. Then I spent two months on evals so the answers would be right. Not a demo. What it does, what it costs per message, and where it is still early. There is a chat on this page. Open it with the sparkle in the bar at the top. It has read everything I have written here. Ask it what I did at DesignQA, or why I shut it down, or which of these posts is worth your time, and it will answer from what I actually wrote. I built it for myself in June and put it on this page in July, when I [rebuilt this site agent-first](/blog/my-new-website-wasnt-designed-for-humans-first/). Then I published it and [people spent three weeks trying to break it](/blog/i-built-a-version-of-myself/). Today it is a product. It is called [Sondo](https://usesondo.com), anyone can put it on their own site, and this post is what it does and what I learned building it. It is not a demo reel. ## It started as a maintenance problem I did not want a chatbot. I wanted the site to answer for me while I was asleep. An about page goes stale the week you write it. I ship something, change my mind about a tool, leave a job, and now three pages are wrong and I am the only person who knows it. The honest version of my background lives in what I write, not in a summary of it. So the requirement was never "add AI." It was: point this at everything I publish, and keep it right without me maintaining it. Add a post, connect a site, drop in a file, and the answers move with it.
Sondo · Knowledge Base
The Sondo Knowledge Base for mahdif.com. A connected site card shows mahdif.com as synced with 28 posts, above a table of documents with their status and chunk counts.
The Knowledge Base behind the chat on this page. A set of documents and this blog, synced nightly. If it is not on this list, the chat does not know it.
## The chat was the easy part Getting a model to sound fluent takes an afternoon. Getting it to be correct took two months of evals. The failure that worried me most was not a wrong fact. It was a false premise. Someone asks about a job I never had, or a company I never worked for, and the model politely agrees and builds an answer on top of it. On one scripted conversation it failed six times out of six. That specific case now passes 24 out of 24. Not because the model got smarter, but because it now treats what a visitor says as a claim to check against the sources, not a fact to build on. Its own earlier answers do not count as sources either. That second part mattered. Once it had agreed to the job on turn one, it defended it for the rest of the conversation.
The mahdif.com chat. A visitor asks about working at the UN and the assistant replies that it has no record of it, then holds that answer when pushed a second time.
The chat on this page, asked about a job I never had. Asked again, more leadingly, it did not budge. This is the case that failed six times out of six in July.
The other one was retrieval. A question about how I lead a team came back as platitudes, six times out of six. The fix was not a prompt. It was one document. After it, six out of six, and it was live within the week. Over that whole stretch I tried four prompt changes and four content changes. The prompt changes measured zero for four. The content changes landed four for four. I mention the before numbers because the after numbers mean nothing without them. ## The bake-off changed nothing, and that was the result Once the answers were right, I ran a bake-off. Same scripted conversations, same rubrics, six repeats each, real money on every run. Gemini 3.5 Flash Lite tied the model I was already running, Gemini 3.1 Flash Lite, exactly: 232 out of 246, at $2.50 per million output tokens against $1.50. GLM 5.3 Flash scored best on the raw number, 236, and was by far the cheapest, but it took 6.9 seconds to start talking where the Gemini models took about one. A visitor does not wait 7 seconds. So the default stayed on Gemini 3.1 Flash Lite.
Sondo · Models
The Sondo chat model picker. Six model cards show live prices per million tokens. Gemini 3.1 Flash Lite is selected and carries a Default badge, with Claude Opus 4.8 at twenty times the output price two cards away.
The model picker, with live prices. The expensive options are right there. The default is still the cheap one the bake-off could not beat.
The newer model was not better. The cheapest model was too slow. The answer was the boring middle, and I only know that because I measured it instead of guessing. ## I can see what every answer costs I did not want to find out what this costs at the end of the month. In the dashboard, every answer shows its own cost, to a fraction of a cent. Not the conversation. The answer. You can watch a single reply cost six hundredths of a cent and decide for yourself whether that was worth it.
Sondo · Conversations
A conversation in the Sondo dashboard with details shown. Under each assistant reply a line gives the model name, the token counts, and the cost of that single reply.
The same conversation in the dashboard, with details on. Under every answer: the model, the tokens, and what that one answer cost. The whole exchange came to $0.0027.
Most AI products hide this, and I understand why. I would rather show it.
Sondo · Usage
The Sondo Usage page. Three tiles show spend for today, the last seven days and this month, above a bar chart of daily spend across thirty days.
The Usage page. Today, the last seven days, this month, and a bar per day. The numbers are small, which is the point.
## My agent drives it more than I do There is an MCP server. Point Claude Code, Codex, or whatever you use at it with an API key, and your agent can read conversations, check what you are spending, and clear the inbox without you opening the dashboard. I use it more than I use the dashboard, which tells me something I have not fully acted on yet.
Claude Code asking Sondo what the chat cost this month, through the MCP server. No dashboard open.
## Where it is still early It answers from what you gave it, and only that. If your content is thin, it will tell a visitor it does not know. That is the correct behavior and it is still a disappointing moment, and the fix is always more content, not a better prompt. There is no docs site yet. Support is an email address I answer personally. And I am still its heaviest user, which means the things I have polished most are the things I happen to need. That will stay true until enough other people use it to pull it somewhere I would not have taken it. ## The feature is the refusal Every AI chat on every site promises it can answer anything. The one on this page is built to tell you when it cannot, and that turned out to be the part worth two months of work. Ask it something about me. If it does not know, it will say so. That is the whole product.