- Rohit Gahukar’s 435 AI engineering lessons teach every algorithm by hand before introducing production frameworks like PyTorch.
- These AI engineering lessons are MIT-licensed and free, covering 20 phases from linear regression to multi-agent systems.
- An 84% daily AI tool usage rate among CS students contrasts sharply with just 18% feeling ready to ship real work.
- Each lesson produces a reusable output — prompt templates, agent definitions, or MCP servers — turning the curriculum into a working toolbox.
- Rohit Gahukar’s 435 AI engineering lessons teach every algorithm by hand before introducing production frameworks like PyTorch.
- These AI engineering lessons are MIT-licensed and free, covering 20 phases from linear regression to multi-agent systems.
- An 84% daily AI tool usage rate among CS students contrasts sharply with just 18% feeling ready to ship real work.
- Each lesson produces a reusable output — prompt templates, agent definitions, or MCP servers — turning the curriculum into a working toolbox.
The Gap Nobody’s Talking About
There’s a quiet crisis running through AI education right now, and it has nothing to do with access. The AI engineering lessons that dominate boot camps and university syllabuses share a common flaw: they hand you the steering wheel before you’ve seen the engine. A survey published last year found that around 84% of computer science students use AI tools daily. Only about 18% feel confident enough to ship production work with them. That’s not a tooling gap. That’s a comprehension gap — and it’s getting wider as frameworks get more powerful.
Rohit Gahukar, a developer who spent eighteen months writing what became the ai-engineering-from-scratch curriculum, noticed the same pattern. You can fine-tune a model without ever writing a forward pass. You can wire an agent to a function without ever defining attention. You can pip install transformers, ship a demo at a hackathon, and never touch a gradient. Frameworks invite that trade-off. Most of the time, it holds. Then your loss curve diverges, your tokenizer balloons token counts for non-English text, or your agent starts hallucinating because the context window is stuffed with duplicated boilerplate — and none of that failure mode is documented in the library’s README. Good AI engineering lessons anticipate exactly these failure modes by grounding learners in what’s happening beneath the API surface.
Build It, Then Use It — Why This Method Works
Gahukar’s answer was a rule simple enough to fit on a Post-it note: build the small version by hand first, then run the same thing through the production library. Not as separate exercises. As two halves of the same lesson. His AI engineering lessons all follow this structure without exception.
The insight came from a tokenizer. He wrote a byte-pair encoding implementation in Python — a for loop, counting byte pairs by hand, merging the most frequent ones. It took about forty seconds to chew through a small corpus. Then he ran the same input through tiktoken, OpenAI’s tokenizer library, and watched it finish in forty milliseconds. Same algorithm. Tiktoken is largely implemented in Rust with warmed caches and unrolled loops. But he already understood what it was doing because he’d done it the night before, slowly and manually.
That’s the core mechanism. When you’ve written the small version yourself, the framework version stops being an oracle and starts being a familiar thing running faster. You can read the framework’s output and know what you’re looking at. When something breaks, you have a mental model to debug against. This is precisely why structured AI engineering lessons that enforce a build-first sequence produce more confident, capable practitioners than top-down alternatives.
The curriculum structures every topic around this duality. The first half — “Build It” — uses NumPy and Python’s standard library only. No frameworks. You derive the chain rule, then write backprop. You count byte pairs in a loop, then call it a tokenizer. You compose three matrices, then call it attention. The code is slow. It’s short enough to read in a single sitting. You can stick a print statement anywhere and watch the math happen in real time.
The second half — “Use It” — runs the identical algorithm on identical data through PyTorch, scikit-learn, tiktoken, or whichever production tool owns that problem. You diff the outputs. You watch the framework smooth over the engineering noise. The framework stops being a black box because the hand-rolled version is sitting in the file right next to it.
AI Engineering Lessons That Actually Show the Math
Gahukar’s transformer lesson is a useful example. Here’s the attention mechanism as he teaches it — no framework, no checkpoint, just NumPy:
The function computes scaled dot-product attention: queries multiplied by transposed keys, scaled by the square root of the key dimension, passed through a softmax, then applied to the values. That’s it. The numerical stability trick — subtracting the max score before the exponential — gets a dedicated paragraph because it’s the one step that consistently trips readers. The mask parameter is what separates encoder attention from decoder attention later in the phase. Run this through a 4-token, 8-dimensional toy example and you’ve computed the same operation that runs inside Llama, GPT-class models, and virtually every open-weight transformer anyone’s using today.
The second half of the same lesson runs through torch.nn.MultiheadAttention with head count set to one and confirms the outputs match to numerical precision. At that point, PyTorch’s attention implementation isn’t magic — it’s the same thing, compiled for CUDA.
The same structure applies to the linear regression lesson in Phase 2. Six lines of NumPy implement gradient descent from scratch — prediction, error, weight update, bias update. Three of those lines are the gradient. Run the same fit through scikit-learn and the coefficients match. Run it through PyTorch with optim.SGD and the loss curves overlay. optim.SGD stops being something you trust on faith. These are the kinds of AI engineering lessons that replace blind trust with genuine understanding.
Twenty Phases, One Tower
Stack this pattern across twenty phases and the progression becomes striking. Phase 10 produces a working small LLM — hand-built, then framework-verified. Phase 14 introduces a full agent loop. Phase 19 reaches multi-agent systems. At every layer, there’s a hand-built version sitting directly underneath the framework version. The tower has foundations.
This is worth contrasting with how most AI curricula are structured. The dominant approach in 2024 — from fast.ai’s top-down philosophy to most corporate upskilling programs — starts with the high-level API and hopes that intuition trickles down eventually. That works for a large slice of use cases. But intuition that trickles down tends to be shallow, and shallow intuition is exactly what fails when models misbehave in production. Gahukar’s bottom-up AI engineering lessons are closer in spirit to how systems programming courses approach operating systems: write a scheduler before you use one, write a memory allocator before malloc is just a given.
There’s a case to be made that the AI industry’s current talent bottleneck isn’t the number of people who can call an API — it’s the number who can tell you why it’s misbehaving. That’s the gap these AI engineering lessons are aimed at.
A Curriculum That Ships Artifacts, Not Just Knowledge
One aspect of the project that Gahukar says he didn’t anticipate: by the end of the curriculum, you haven’t just learned things — you’ve built things. Each lesson is designed to produce at least one reusable artifact. Some lessons produce prompt templates for specific tasks. Others produce skill specs compatible with Claude, Cursor, or Codex. Some output agent definitions with clearly scoped jobs. Others build MCP servers that expose the lesson’s logic as a callable tool. Well-designed AI engineering lessons work this way — every concept leaves behind something you can actually use.
By Phase 19, you’ve accumulated hundreds of these. They’re not exercises that live in a notebook and gather dust. They’re components you can reach for when a real task lands. Gahukar describes it as “a textbook on the way in and a toolbox on the way out” — which is a more honest description of practical education than most course landing pages manage.
The whole curriculum is MIT-licensed and available for free at aiengineeringfromscratch.com. No login, no paywall, no setup required to read it in the browser. Clone the GitHub repo if you want to run the code.
Why This Matters Beyond One Developer’s Side Project
It’s tempting to file this under “impressive personal project” and move on. But there’s something structurally interesting happening here. As AI tooling gets more abstracted — as foundation model APIs replace model training, as agent frameworks replace bespoke orchestration, as code generation replaces boilerplate — the floor for building demos drops to nearly zero. The ceiling for understanding what you’ve built doesn’t move at all.
That gap is going to produce real problems at scale. Not in demos. In production systems, in regulated industries, in applications where a hallucinating agent or a misconfigured tokenizer causes actual harm. The developers who’ll be equipped to fix those problems aren’t the ones who know which API to call. They’re the ones who worked through AI engineering lessons that required them to write the small version by hand first — and who therefore know what the production library is actually doing underneath.
Gahukar’s 435 AI engineering lessons are one person’s attempt to close that gap, published for free, under the most permissive license available. Whether the broader education ecosystem — universities, boot camps, corporate L&D programs — takes the same approach is a different question. Given how fast frameworks are advancing, it’s one the industry probably can’t afford to defer much longer.
