LLMs from Scratch
Sebastian Raschka's step-by-step guide (and book code) to implementing a GPT-style large language model in PyTorch from the ground up.
LLMs from Scratch
The official code for Sebastian Raschka's book — build a ChatGPT-like LLM in PyTorch, step by step: tokenization, attention, the transformer block, pretraining, and fine-tuning for classification and instruction following.
What's inside
- Chapter-by-chapter notebooks, from a tiny model to a working GPT
- Bonus material on efficient attention, LoRA and more
Runtime: Python / PyTorch