LLM Production Engineering: Ship AI Apps That Scale

This course contains the use of artificial intelligence.Your LLM demo works. Your production system is where it goes to die — in the latency tail your users feel, in a bill nobody measured, in an eval set that drifted away from real traffic, in an injection attack a user pastes in on a Friday. This course is the engineering between the demo and the system that survives: serving and inference, prompts as code, latency and cost engineering, reliability ladders, context budgets, evaluation with golden sets and CI gates, observability, and guardrails — every topic taught as a measurement or a build, never as a slideware claim.The core ten lectures map the whole discipline: the four forces that kill LLM products (latency, cost, reliability, drift); hosted APIs versus self-hosting, decided with numbers; prompts under version control with structured outputs; latency engineering — streaming, caching, and the tails that matter more than the average; token economics and cascades; retries, fallbacks and circuit breakers; context budgets, memory and fresh RAG; evaluation with golden sets, judges and CI gates; traces and injection defence; and the deployment architecture that assembles it all.Then you build three production systems. Project 1: AnswerDesk, a RAG service over a real corpus — designed by contract and census, built with citation-bound answers and abstention guards that took three measured calibration rounds to get honest, and shipped behind an evals gate that can block its own deploy. Project 2: the Optimization Lab — a 240-request workload measured to the token with a real tokenizer, then compressed, cached and cascaded from $479.66 to $25.91 a month (94.6% down) behind a quality gate that had to pass before the savings counted. Project 3: ShipGate — structured traces for every request, a 12-attack injection battery refused 12/12 with zero false refusals, PII masked before it ever reaches a log, and a five-gate ship board you will watch fail three different ways on purpose.Then you crack the interview. Four system-design lectures with the arithmetic done live: a chatbot that survives launch day (40 requests/second becomes $1,656 per peak hour — then gets rescued, honestly), RAG at ten million documents (twenty billion tokens become five hundred gigabytes of vectors — then get designed down), cutting inference cost 10× with measured lever factors, and a five-move playbook with a ten-question rapid-fire drawn from the whole course.Every number in this course was produced by running the system on screen — census counts, ranking scores, gate boards, cost figures. Nothing is asserted from memory, and where a board is a fixture's property rather than a law of nature, the lecture says so. You also get downloadable code and exercise books, six checkpoint quizzes with a 70% mastery rule, five hands-on assignments with worked solutions, and a capstone: ship your own production LLM service behind your own ship gate.AI disclosure: this course's narration uses a cloned voice of the instructor, created with the instructor's consent; all content, code, and curriculum are authored and verified by the instructor team. Music: "Frozen Star" by Kevin MacLeod — CC BY 4.0.