Last month a client asked me a question I now get almost every week: "Which AI should we use?" They had already paid for three subscriptions, wired one of them into their support inbox, and were getting mediocre results from all of it. Nobody had ever asked what the models were actually being asked to do.
That is the part most teams skip. The model is not the decision. The workload is the decision — and once you know the workload, picking between ChatGPT, Gemini and Claude stops feeling like a religious argument and starts feeling like choosing a database.
Here is how I actually make that call on real projects, plus the parts of generative AI that matter far more than the model name.
They Are Not Interchangeable, But They Are Closer Than the Benchmarks Suggest
If you read leaderboard posts, you would think there is a clear winner every quarter. In production the gap is much smaller than the marketing. For maybe 70% of the tasks I ship — summarising a document, extracting structured fields from messy text, drafting a reply, classifying an intent — all three frontier models pass. The differences show up at the edges, and the edges are where your product lives.
What I have consistently noticed after wiring all three into client systems:
Long, messy context. Pulling one decision out of a 40-page contract or a year of Slack history. Here I lean on models with large usable context windows rather than large advertised ones. There is a difference. Test it with your own documents before you believe a number on a spec sheet.
Code and refactoring. Claude has been my default for a while, mostly because it holds a repository's conventions in its head across a long session instead of drifting into generic tutorial code by turn six.
Search-grounded answers. When the question depends on something that happened this week, Gemini's tie into Google's index does real work that a raw model call cannot fake.
Ecosystem maturity. OpenAI still has the widest set of SDK examples, community wrappers and Stack Overflow answers, which matters more than people admit when a junior developer owns the integration.
None of that is a permanent ranking. I have rewritten this paragraph in my own notes three times in eighteen months. Treat model choice as a config value, not an architecture decision.
Build the Escape Hatch on Day One
The single most useful thing I do on any AI project takes about two hours: put a thin provider layer between the application and whichever API sits behind it.
// llm/provider.ts
export interface LlmProvider {
complete(input: {
system?: string;
prompt: string;
maxTokens?: number;
}): Promise<{ text: string; inputTokens: number; outputTokens: number }>;
}
// Swap implementations with an env var, not a refactor.
export function getProvider(): LlmProvider {
switch (process.env.LLM_PROVIDER) {
case "anthropic": return anthropicProvider;
case "google": return geminiProvider;
default: return openAiProvider;
}
}
It looks trivial. It is trivial. It is also the reason I moved a client off a provider in a single afternoon when pricing changed, and the reason I can A/B two models against real production traffic instead of arguing about benchmarks in a meeting.
Do not over-engineer it. You are not building an abstraction that supports every exotic feature of every provider. You are wrapping the 80% of calls that look identical everywhere and letting the exotic 20% talk to the SDK directly.
The Cost Conversation Nobody Has Early Enough
Generative AI pricing is deceptively cheap per call and genuinely expensive per month. A summarisation endpoint that costs a fraction of a cent per request sounds free until it runs across 200,000 records in a nightly job.
Three things that have saved my clients real money:
Route by difficulty, not by habit. Most teams send every request to their most capable model. Classification, routing, short extraction and yes/no decisions run perfectly well on a smaller, cheaper one. I usually split traffic roughly 80/20 — small model for the volume, large model for the work that actually needs reasoning — and the bill drops by more than half without a single user noticing.
Cache aggressively. Both at the prompt level, since providers now offer caching for long stable system prompts and documents, and at your own application level. If two users ask the same question about the same document, you should not pay twice. A boring Redis key built from a hash of the prompt catches a surprising amount of duplicate traffic.
Cap output tokens. Output is the expensive half. If you need three bullet points, say so and set a hard limit. Left unconstrained, models write essays that you then truncate in the UI — and you paid for every word you threw away.
What Actually Breaks in Production
The failures I spend my time on are almost never "the model was not smart enough."
They are: the API timed out and nothing retried. The response came back as prose when the code expected JSON. A user pasted 90,000 characters into a field with no length check. The provider had a partial outage and the whole feature went down with it because there was no fallback. Someone tweaked a prompt in a hotfix and nobody noticed quality had dropped for two weeks, because there were no evals.
So this is the checklist I run before any AI feature ships:
Timeouts and retries with backoff, plus a circuit breaker if the feature sits on a critical path.
Structured output enforced by schema — use the provider's JSON or tool-calling mode, then validate with Zod or Pydantic anyway. Models drift.
Input truncation and token counting before the call, not after the 400 error.
A fallback provider for anything user-facing. Two providers behind that thin interface is cheap insurance.
Twenty to fifty saved test cases with expected outputs, re-run on every prompt change. It does not need to be a fancy eval framework — a JSON file and a script catches the obvious regressions.
Logging of the full prompt, response, latency and token counts. When a user says "the AI gave me something wrong yesterday," you need to be able to look it up.
Where Generative AI Is Genuinely Heading
Two shifts feel real to me rather than hype-driven.
The first is that the model is becoming the least interesting part of an AI product. Anyone can call an API. The defensible work is in the data you feed it, the tools you let it use, the guardrails around it and the workflow it slots into. I have watched two teams build the same feature on the same model and get completely different outcomes, because one of them had clean retrievable internal data and the other had a SharePoint graveyard.
The second is that AI is moving out of the chat box and into the background. The chat interface was a brilliant way to introduce the technology, but most of the value I have shipped in the last year runs with nobody watching — enriching records overnight, triaging tickets before an agent opens them, drafting the first version of a document a human then edits. Chat is the demo. Automation is the product.
If You Are Choosing Today
Pick one provider to start, but build behind an interface. Run your own five-example test with your real data before you trust anyone's benchmark. Route cheap work to cheap models. Apply the same discipline you would give any third-party API — timeouts, validation, logging, fallback.
And be honest about whether the feature needs a language model at all. I have replaced two "AI features" with a regex and a lookup table this year, and both came out faster, cheaper and more accurate. That is not a failure of AI. That is engineering.
