The demo took two days. A chat interface, a well-crafted prompt, a client who was genuinely impressed. Then came the questions that took four months to answer: what happens when someone pastes 400 pages into it, what stops a user asking it to write something the company would be embarrassed by, how do we charge for this when one power user costs more than their subscription, and who is on call when the provider has an incident.
That gap — two days to a demo, four months to a product — is the defining feature of building with generative AI. The model does the impressive part immediately, and then you build a business around something whose cost, latency and output all vary per request.
Here is what actually goes into that four months.
The Unit Economics Come First
Most AI products I have seen struggle commercially made the same mistake: they priced like software and paid costs like a service.
Traditional SaaS has near-zero marginal cost, which is why unlimited flat-rate pricing works. An AI feature has a real cost per use, and usage is wildly uneven. In every product I have measured, a small fraction of users generate the majority of the spend — often dramatically so. A flat monthly price averaged across users looks fine until your heaviest 3% arrive.
What you need before launch, not after:
Cost attribution per user and per feature. Log tokens on every call, tagged with who and what. Without this you cannot tell which feature is unprofitable, and the answer is rarely the one you would guess.
A quota, even a generous one. Credits, messages, documents — some unit the user understands. It caps your exposure and it gives you an upgrade path.
Hard limits per account. Not just for cost. A runaway script hitting your endpoint in a loop is a bill nobody approved.
The pricing model that has worked best in what I have built is a base subscription with an included allowance and clear overage — familiar to buyers, and honest about the fact that heavy use costs more.
Chatbots: The Easy Part Is the Chat
Every AI chatbot project has the same distribution of effort. The conversation works on day one. The remaining 90% of the work is everything around it.
Grounding. A support bot that answers from the model's general knowledge will confidently describe features you do not have. It needs retrieval over your actual documentation, and it needs an explicit instruction to say when it does not know. The moment a bot invents a policy, users stop trusting it permanently — and they tell each other.
Escalation. Design the handoff to a human before you design the conversation. Users who cannot reach a person get angry in a way that costs more than the support hours you saved. A visible route out is the difference between a bot people tolerate and one they resent.
Streaming. A response that takes eight seconds to appear feels broken; the same response streaming token by token feels fast. This is perception, not performance, and it is one of the highest-return frontend decisions in the whole category.
Conversation state. The model has no memory. You resend history every turn, which means cost grows with conversation length. Decide deliberately how much history to keep — a rolling window plus a running summary is the pattern that has worked for me.
Image and Video: Async or Nothing
The architecture here is different enough that people get it wrong by treating it like a text feature.
Generation takes seconds for images and minutes for video. Nothing about that fits a request-response cycle. You need a job queue, a status endpoint or a websocket, a way to notify the user when it is done, and a user interface that makes waiting acceptable rather than confusing.
// The request returns immediately. The work happens elsewhere.
app.post("/v1/generations", async (req, res) => {
const input = GenerationSchema.parse(req.body);
await quota.consume(req.user.id, costOf(input)); // before, not after
const job = await jobs.enqueue("generate", {
userId: req.user.id, ...input,
});
res.status(202).json({ id: job.id, status: "queued" });
});
Consuming the quota before enqueueing rather than after completing matters more than it looks. Otherwise a user can queue two hundred jobs before the first one finishes and the accounting catches up.
Two more things specific to media. Storage costs add up quietly — generated media is large, users generate far more than they keep, and without a retention policy your storage bill grows without limit. And rights and provenance deserve an actual answer, in writing, before customers ask: who owns the output, what your provider's terms say about it, and whether generated media carries a watermark or metadata marking it as synthetic.
Content Generation: The Quality Ceiling Problem
Text generation products — marketing copy, product descriptions, summaries, reports — have an unusual failure mode. They work immediately and then plateau at "adequate."
Adequate is not a product. Anyone can get adequate from a chat interface for free. What makes a content product worth paying for is the things a generic prompt cannot do: your brand voice enforced consistently, your data merged in, your format and constraints applied, integration into the workflow where the content actually gets used, and a review step that fits how the team already works.
The products in this space that survive are the ones that own the workflow rather than the generation. Generation is a commodity; the surrounding process is not.
Safety, Proportionate to Exposure
How much you need depends entirely on who can reach the feature, and teams routinely get this backwards — heavy filtering on internal tools, nothing on the public endpoint.
For an internal tool used by employees, light-touch is reasonable. For anything public, you need input filtering, output filtering, per-account rate limiting, logging good enough to investigate a complaint, and a way to suspend an abusive account quickly.
Two specific risks people miss. Prompt injection — if your feature processes user-supplied content, that content can contain instructions, and the model does not reliably distinguish data from directions. The defence is architectural: the model's output is a request, never a permission, and anything with a real effect runs through your own authorisation check. And data leakage between users — if you build context from a shared source, verify at the retrieval layer that a user can only ever pull their own tenant's data. Relying on the prompt to enforce that is not a control.
The Operational Layer Nobody Demos
These are the things that separate a product from a prototype, and none of them appear in a pitch deck.
Evals. Thirty to fifty saved cases with expected outputs, run on every prompt or model change. Without them, quality regressions ship silently and you find out from a churned customer.
Provider fallback. A thin interface with a second provider behind it. Outages happen, and "our AI is down because their AI is down" is not an explanation customers accept twice.
Caching. Identical requests should not be paid for twice. In most products a meaningful fraction of traffic is repeated, and a hash-keyed cache is an afternoon of work.
Model routing. Cheap model for the simple majority, expensive model for the hard minority. This alone typically halves the bill.
Full request logging. Prompt, response, latency, tokens, user. When someone reports a bad output from Tuesday, you need to be able to look at it.
What I Would Tell Someone Starting
Build the demo — it is genuinely quick and it will tell you whether the idea has legs. Then, before adding a second feature, build cost tracking, a quota, an eval set and a logging path. Those four things take a week and they are what let you operate the thing.
And find the part of the product that is not the model. The generation is available to everyone at roughly the same price and quality. What you own is the data you bring, the workflow you fit into, and the trust that you will not embarrass the customer. That is where the business is, and it always has been.
