10Gb/s Ethernet: what I did to get it working in my home

10Gb Ethernet: what I had to (re)learn

LLM from scratch, part 33 – what I learned from the appendices

LLM from scratch (32l) – Interventions: updated instruction fine-tuning results

How an LLM becomes more coherent as we train it

LLM from scratch, part 32k – Interventions: gradient accumulation

Provision: LLM-powered server setup from Markdown

LLM from scratch, part 32j – trying to train a better model in the cloud

Writing an LLM from scratch, part 32i – Interventions: what is in the noise?

Writing an LLM from scratch, part 32h – Interventions: full fat float32

Writing an LLM from scratch, part 32g – Interventions: weight tying

Writing an LLM from scratch, part 32f – Interventions: weight decay

Writing an LLM from scratch, part 32e – Interventions: the learning rate

Writing an LLM from scratch, part 32d – Interventions: adding attention bias

Writing an LLM from scratch, part 32c – Interventions: removing dropout

Writing an LLM from scratch, part 32B – Interventions: gradient clipping

Writing an LLM from scratch, part 32a – Interventions: training a baseline model

Getting a Custom PyTorch LLM onto the Hugging Face Hub

Writing an LLM from scratch, part 31 – the models are now on Hugging Face

Writing an LLM from scratch, part 30 – digging into the LLM-as-a-judge results

LLM from scratch, part 29 – using DDP to train a base model in the cloud

LLM from scratch, part 28 – training a base model from scratch on an RTX 3090

Writing an LLM from scratch, part 27 – what's left, and what's next?

Writing an LLM from scratch, part 26 – evaluating the fine-tuned model

Writing an LLM from scratch, part 25 – instruction fine-tuning

Writing an LLM from scratch, part 24 – the transcript hack

Retro Language Models: Rebuilding Karpathy's RNN in PyTorch

Writing an LLM from scratch, part 23 – fine-tuning for classification

Writing an LLM from scratch, part 22 – training our LLM

Revisiting Karpathy's 'Unreasonable Effectiveness of Recurrent Neural Networks'

Writing an LLM from scratch, part 21 – perplexed by perplexity

Writing an LLM from scratch, part 20 – starting training, and cross entropy loss

How Do LLMs Work?

The maths you need to start understanding LLMs

What AI chatbots are doing under the hood

LLM from scratch, part 18 – residuals, shortcut connections, and the Talmud

The fixed length bottleneck and the feed forward network

Writing an LLM from scratch, part 17 – the feed-forward network

Writing an LLM from scratch, part 16 – layer normalisation

Leaving PythonAnywhere

Writing an LLM from scratch, part 15 – from context vectors to logits

Writing an LLM from scratch, part 14 – the complexity of self-attention at scale

Writing an LLM from scratch, part 13 – attention heads are dumb

Writing an LLM from scratch, part 12 – multi-head attention

Writing an LLM from scratch, part 11 – batches

The Business of the AI Labs

Writing an LLM from scratch, part 10 – dropout

Adding /Llms.txt

Writing an LLM from scratch, part 9 – causal attention

Writing an LLM from scratch, part 8 – trainable self-attention