← All courses

Optimizing AI Models for Inference

How to make trained AI models smaller, faster and cheaper to run: quantization, pruning, distillation, batching, compilers and runtimes, LLM tricks like the KV cache and speculative decoding, plus serving cost and how to verify nothing broke. Built for developers new to ML. Builds on this library's mlops course for the wider production lifecycle (registries, pipelines, monitoring) — see it for that foundation.

aimachine-learningoptimizationinferencellmintermediate

Get Yaaddi on Google Play Free. Install the app, then add this course from its Course Library.

32decks
148main cards
697practice cards
0%with images
v1.2.1version

Card type mix

What's covered

  1. What a Model Is: Weights and Tensors
  2. Training vs Inference
  3. Latency, Memory and Money
  4. Measure First: Profiling and Benchmarks
  5. Accuracy vs Speed Trade-offs
  6. Numeric Precision: FP32 to INT8
  7. Post-Training Quantization
  8. Quantization-Aware Training
  9. Pruning
  10. Knowledge Distillation
  11. Low-Rank and Weight Sharing
  12. CPU, GPU and Accelerators
  13. Batching
  14. Operator Fusion and Graph Compilers
  15. Model Formats: ONNX and GGUF
  16. Inference Runtimes
  17. Tokens and Context Length
  18. The KV Cache
  19. FlashAttention and Attention Tricks
  20. Speculative Decoding
  21. Continuous Batching and Serving Engines
  22. LoRA and QLoRA
  23. LLM Serving Metrics
  24. Splitting Models Across GPUs
  25. Caching and Model Routing
  26. Autoscaling and Cold Starts
  27. Cost per Request
  28. Edge and On-Device Deployment
  29. Regression-Testing After Optimizing
  30. Choosing an Optimization Strategy
  31. Common Optimization Pitfalls
  32. Recap: Key Terms

Example cards

multiple choice

What are a model's weights?

  • Hand-set rules
  • Learned numbers
  • Training inputs
multiple choice

What is a tensor?

  • Number grid
  • Neural layer
  • Model file

Latest: Haiku review fixes: corrected cards that did not match what their lesson teaches.