LLMs from Scratch
Sebastian Raschka's step-by-step guide (and book code) to implementing a GPT-style large language model in PyTorch from the ground up.
Installation & usage
README summary
Sebastian Raschka's step-by-step guide (and book code) to implementing a GPT-style large language model in PyTorch from the ground up.
The official code for Sebastian Raschka's book — build a ChatGPT-like LLM in PyTorch, step by step: tokenization, attention, the transformer block, pretraining, and fine-tuning for classification and instruction following. Chapter-by-chapter notebooks, from a tiny model to a working GPT. Bonus material on efficient attention, LoRA and more. Runtime: Python / PyTorch
Auto-extracted from the repository README. The authoritative steps live in the repository on GitHub.
Full README
LLMs from Scratch
The official code for Sebastian Raschka's book — build a ChatGPT-like LLM in PyTorch, step by step: tokenization, attention, the transformer block, pretraining, and fine-tuning for classification and instruction following.
What's inside
- Chapter-by-chapter notebooks, from a tiny model to a working GPT
- Bonus material on efficient attention, LoRA and more
Runtime: Python / PyTorch