- Every LLM API call is stateless — your chatbot only ‘remembers’ history if you manually resend the full message array each time.
- An LLM API call returns a stop_reason field most developers ignore — skipping it guarantees a production bug eventually.
- Output tokens cost 3–5x more than input tokens, making long responses far pricier than long prompts.
- Token counts don’t map cleanly to words — JSON, code, and non-English text consume far more tokens than most developers expect.
- Every LLM API call is stateless — your chatbot only ‘remembers’ history if you manually resend the full message array each time.
- An LLM API call returns a stop_reason field most developers ignore — skipping it guarantees a production bug eventually.
- Output tokens cost 3–5x more than input tokens, making long responses far pricier than long prompts.
- Token counts don’t map cleanly to words — JSON, code, and non-English text consume far more tokens than most developers expect.
What Actually Happens When You Make an LLM API Call
Most developers’ first experience with an LLM API call looks something like this: grab an API key from OpenAI or Anthropic, paste six lines of JavaScript, run it, and watch text appear. It works. It feels like magic. And because it works, most people never look any deeper — which is exactly how subtle, expensive bugs get shipped to production.
The truth is that underneath every SDK wrapper, every chatbot UI, and every AI-powered feature you’ve used recently, there’s a remarkably simple HTTP transaction. It’s a POST request carrying a JSON body with a handful of fields — model, messages, max_tokens — sent to a provider endpoint. The response comes back as another JSON blob. That’s it. Once you actually see the raw mechanics, switching between providers like OpenAI, Anthropic, or Mistral becomes almost trivial. The URLs differ, the authentication header might change from x-api-key to Authorization: Bearer, and the system prompt might live in a different part of the payload. But the fundamental shape of the request stays consistent across the industry.
This matters more than it sounds. A whole generation of developers is building AI features on top of SDKs without understanding the contract those SDKs abstract away. When something breaks at scale — and it will — the people who understand the raw LLM API call are the ones who can debug it quickly.
The Stateless Reality Every LLM API Call Hides
Here’s something that surprises a lot of developers when they first encounter it directly: every LLM API call is completely stateless. The model has no memory of your previous request. None. When you’re building a chatbot that feels like it remembers context, what’s actually happening is that your application is maintaining a messages array and resending the entire conversation history with every single request.
Think about what that means at scale. A ten-turn conversation doesn’t cost ten times the price of the first message — it can cost dramatically more, because by turn ten you’re sending the full history of all previous turns as input tokens on every single call. This is one of the most important architectural facts about working with language models today, and it’s invisible if you’ve only ever used a high-level SDK.
The implication for product design is real. Context window management — deciding what history to keep, what to summarise, what to drop — isn’t just an academic concern. It’s a cost and reliability decision that shapes how your application behaves in practice. Companies like Anthropic have expanded context windows dramatically (Claude 3’s 200K token window being a notable example), but larger windows don’t eliminate the problem — they just push it further out while making the per-call cost even higher if you’re not careful.
The stop_reason Field: The Most Ignored Part of an LLM API Call
Every LLM API call response includes a field called stop_reason — and the majority of developers building with language models don’t check it. That’s a mistake that tends to surface at the worst possible time.
The field tells you exactly why the model stopped generating text. There are four values that matter in practice. end_turn means the model finished naturally — you’re good. max_tokens means you hit your ceiling and the response is truncated mid-thought, possibly mid-sentence. tool_use means the model is signalling it wants to invoke an external function or tool rather than returning a final answer. And stop_sequence means the output matched one of the custom stop strings you defined.
If your code only reads the text content and ignores stop_reason, you’ll eventually ship a response that looks fine — right up until a user notices their answer got cut off in a confusing way, or your tool-calling logic silently fails because the model was trying to hand off to a function your code never checked for. The response will look valid. Your logs won’t flag it. And you’ll spend time debugging something that stop_reason would have told you immediately.
The fix is simple: branch on stop_reason from day one. Treat it as a required field, not an optional curiosity.
Tokens, Pricing, and the Numbers That Will Surprise You
The other side of understanding an LLM API call is the economics — and the economics are less intuitive than most developers expect.
First, words don’t equal tokens. The rule of thumb for English prose is roughly one token per four characters, or about 0.75 tokens per word. But that breaks down fast in practice. A word like “unbelievable” is a single word but can be four tokens. Code is worse: a simple Python function definition like def add(a, b): runs to eight tokens because every bracket, colon, and comma is tokenised individually. JSON is genuinely expensive — {“a”:1} costs seven tokens. If you’re sending bloated tool schemas on every request, you’re quietly burning money at scale before a single word of useful content is processed.
Non-English languages compound this further. Japanese, Hindi, and Arabic text can run 2–4 times the token count of equivalent English content. If you’re building a product for global audiences and you’re estimating costs based on English token rates, your numbers are probably significantly wrong.
Then there’s the pricing asymmetry that sits at the heart of LLM economics: output tokens cost roughly 3–5 times more than input tokens. This single fact changes how you should think about prompt design. Stuffing 50KB of context into a system prompt is relatively cheap. Asking the model to generate 50KB of output is dramatically more expensive — potentially five times more. The formula is straightforward: cost equals input tokens divided by a million, multiplied by the input price, plus output tokens divided by a million, multiplied by the output price. Different for every provider, but the asymmetry holds across all of them.
There’s a subtlety that catches people building with reasoning models specifically: thinking tokens — the model’s internal chain-of-thought that you never see — are billed at the output token rate. You’re paying for the model’s scratchpad even though it never appears in the response. At high volume, this adds up faster than most teams budget for.
The Tool Schema Tax
One cost vector that rarely gets discussed in beginner tutorials is the tool schema overhead. When you define functions or tools that the model can call, those schema definitions get included in every single request — just like your system prompt. They’re input tokens, and they’re resent on every call. A well-designed, minimal tool schema is a genuine cost optimisation. A bloated one with verbose descriptions and unnecessary parameters quietly increases your bill on every interaction your users have.
At $0.006 per call — a reasonable ballpark for many current models — 100,000 daily calls adds up to $600 a month from a single feature. That’s before you account for any spikes. Adding usage logging that tracks input and output tokens from the very first deployment isn’t premature optimisation — it’s basic operational hygiene that will save you from a shocking invoice.
Why Understanding the Raw LLM API Call Still Matters in 2025
The instinct in 2025 is to reach for a framework — LangChain, LlamaIndex, Vercel’s AI SDK, or any of a dozen others — and let it handle the mechanics. That’s a reasonable choice for moving fast. But frameworks abstract away exactly the details that bite you hardest when something goes wrong at scale: statefulness assumptions, token counting, stop condition handling, cost attribution.
The developers who understand what’s happening at the HTTP layer — what fields are in the request, what comes back in the response, how tokens are counted, why stop_reason matters — are the ones who can build reliable systems rather than just demos. They’re also the ones who can evaluate new providers and models on their actual merits rather than based on which SDK has the prettiest documentation.
As language models move deeper into production infrastructure — handling customer support, writing code, processing documents, making decisions — the gap between developers who understand the underlying LLM API call and those who don’t is only going to widen. The good news is that the raw mechanics are genuinely simple. You don’t need a framework to understand them. You just need to look.
Source: https://dev.to/jasmin/an-llm-api-call-in-4-gifs-33b1
