'AI' just means LLMs now

Ada Lovelace and the Analytical Engine

How long before superintelligence? (1997)

Attention is your scarcest resource (2020)

DeltaNet Explained

All AI models might be the same

Life Update – On Health

Asymmetry of Verification and Verifier's Law

Soviet College Admission – My Dad's Story (1970)

H-Net – Inference

How to scale RL to 10^26 FLOPs

Britain is cheap, and should learn to love it

Database Sharding

Microdosing Willpower: My Takeaways from Microdosing Ozempic

The upcoming GPT-3 moment for RL

The Tradeoffs of SSMs and Transformers

Things you can do –with uv

The era of exploration

Just Ask for Generalization (2021)

Will Scaling Solve Robotics?

VLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention

LLM Memory

What Problems to Solve (1966)

Test-Time Training

Thnickels

SFStreets: History of San Francisco place names

Muon Doesn't Clearly Grok Faster

René Girard and Mimetic Theory for Non-Philosophers

Becoming a Better Programmer by Tightening Feedback Loops

Approximating Language Model Training Data from Weights

Street photos of every building in New York in 1939/1940

The Launch of GPT-4

Superintelligence, from First Principles

How to Shuffle a Big Dataset

Q-learning is not yet scalable

Why are neural networks and cryptographic ciphers so similar?

So You Want to Work in Mechanistic Interpretability?

Bowling Alone: America's Declining Social Capital [pdf] (1995)

Introduction to Parallel Programming with CUDA

Machine Learning of Sets

The Different Components of Intelligence

Cutlass Tutorial: Sub-Byte GEMM on Nvidia Blackwell GPUs

The case for more ambition (in AI research)

Why is AI hard and Physics simple?

Generative Modelling in Latent Space

Understanding the Neural Tangent Kernel

Specification Engineering is a bet on better code gen and more complexity

Efficient Streaming Language Models with Attention Sinks

PhDs for Entrepreneurs

The WeightWatcher tool for predicting the accuracy of Deep Neural Networks

Margins of My Dissertation: Life Lessons That My PhD Taught Me

Test-Time Training

LLM Visualization

ML models don't need that much data to be better than you

Ranking Foods by Protein Efficiency

What I learned about running a betting market game night contest

'To Easy LoL' – New Orleans jail break may have been inside job

Softmax – research that inspires us

Neural Network Checklist

Highly Opinionated Advice on How to Write ML Papers

You could have invented Transformers

The Annotated Kolmogorov-Arnold Network (Kan)

What Does Any of This Have to Do with Physics?

What I'm thinking about these days

How to cheat at settlers by loading the dice (2017)

Neural Vector Embeddings

Harnessing the Universal Geometry of Embeddings

Text Embeddings are All Alike

Why We Think

A Meticulous Guide to Advances in Deep Learning Efficiency over the Years (2024)

Deep Laziness

Embeddings are underrated (2024)

What Physics Can Teach Us About AI (OpenAI's Dan Roberts)

Deep Learning for Natural Language Processing (Without Magic) (2013)

Robotics Predictions for 2025

Giant Inscrutable Matrices: Not Worse Than Anything Else

AI crawler wars threaten to make the web more closed for everyone

Why GRPO Is Important and How It Works

The Speed of VITs and CNNs

Good Research Takes Are Not Sufficient for Good Strategic Takes

Thank you for holding my duck (2021)

The Maintenance Race

Product Quantization: Compressing high-dimensional vectors by 97%

Notes on "An Observation on Generalization"