Loading lesson page...
AI From Scratch/Lesson 21/~60 minutes
A/B Testing LLM Features — GrowthBook, Statsig, and the Vibes Problem
Traditional A/B testing was not built for non-deterministic LLMs. The critical distinction: evals answer "can the model do the job?" A/B tests answer "do users care?" Both are required; shipping on vibe checks is over. What to test in 2026...
LearnPython (stdlibtoy sequential test simulator)