AI Engineering by Chip Huyen: Chapter 2 Notes and summary
Chapter 2: Understanding Foundation Models Overview Foundation model design choices (training data, architecture/size, post-training) are increasingly opaque. The training process splits into pre-training (makes model capable) and post-training (aligns model to human preferences). Sampling (how outputs are chosen from all possibilities) is a crucial, often-underestimated factor impacting model behavior and Read more