Optimizing AI Models for Inference
How to make trained AI models smaller, faster and cheaper to run: quantization, pruning, distillation, batching, compilers and runtimes, LLM tricks like the KV cache and speculative decoding, plus serving cost and how to verify nothing broke. Built for developers new to ML. Builds on this library's mlops course for the wider production lifecycle (registries, pipelines, monitoring) — see it for that foundation.
aimachine-learningoptimizationinferencellmintermediate
Get Yaaddi on Google Play Free. Install the app, then add this course from its Course Library.
View this course's repo on GitHub → — star it, open an issue, or suggest a fix.
32decks
148main cards
697practice cards
0%with images
v1.2.1version
Card type mix
- multiple choice: 442
- select blank: 148
- match pairs: 108
- multi select: 88
- categorize: 52
- order: 51
- true false: 33
What's covered
- What a Model Is: Weights and Tensors
- Training vs Inference
- Latency, Memory and Money
- Measure First: Profiling and Benchmarks
- Accuracy vs Speed Trade-offs
- Numeric Precision: FP32 to INT8
- Post-Training Quantization
- Quantization-Aware Training
- Pruning
- Knowledge Distillation
- Low-Rank and Weight Sharing
- CPU, GPU and Accelerators
- Batching
- Operator Fusion and Graph Compilers
- Model Formats: ONNX and GGUF
- Inference Runtimes
- Tokens and Context Length
- The KV Cache
- FlashAttention and Attention Tricks
- Speculative Decoding
- Continuous Batching and Serving Engines
- LoRA and QLoRA
- LLM Serving Metrics
- Splitting Models Across GPUs
- Caching and Model Routing
- Autoscaling and Cold Starts
- Cost per Request
- Edge and On-Device Deployment
- Regression-Testing After Optimizing
- Choosing an Optimization Strategy
- Common Optimization Pitfalls
- Recap: Key Terms
Example cards
multiple choice
What are a model's weights?
multiple choice
What is a tensor?
Latest: Haiku review fixes: corrected cards that did not match what their lesson teaches.