Sign inSign in

AI & ML · Book

Build a Large Language Model (From Scratch)

Sebastian Raschka

Raschka builds a GPT-style large language model, comparable to GPT-2, in code and without existing LLM libraries.

Sebastian RaschkaCourse textManning Publications2024368 pages

Watch the course it teachesFind a copy

Opens a bookshop outside Lectern.

Good fit if

Step-by-step architectural guide to implementing tokenization, multi-head self-attention, pretraining, and alignment from first principles.

About the book

The chapters move from preparing text data and coding attention mechanisms to implementing the model, pretraining it on unlabeled data, and loading pretrained weights. Later chapters fine-tune it for text classification and then to follow instructions, with appendices on PyTorch and on parameter-efficient fine-tuning with LoRA. The book assumes intermediate Python skills and some knowledge of machine learning.

Author

Readers

No reader has written about it yet.

Write the first review

Where next

7 connections