How many paths of length K are there between A and B? (2021)

How A Neuron Learns

GPT, Fast (2023)

GPT-Fast

Exploring EXIF (2023)

The Practitioner's Guide to the Maximal Update Parameterization

The scientific method and its application to the science of deep learning

Solving Humanity's Last Exam Problems

Why We Think

Philosophical Thoughts on Kolmogorov-Arnold Networks (2024)

Matmul() using PyTorch's MPs back end is faster than Apple's MLX

The Making of Gemini Plays Pokémon

Facebook is not worth $33B (2010)

Comefrom

Diffusion Language Models Are Super Data Learners

How to build a router for MOE models

The Eponymous Principles of Management – Coase's Ceiling and Floor

No One Is Working

No One Is Working

SFT Is Bad RL

A Simple CPU on the Game of Life (2021)

Trends in LLM-Generated Citations on ArXiv

'AI' just means LLMs now

Ada Lovelace and the Analytical Engine

How long before superintelligence? (1997)

Attention is your scarcest resource (2020)

DeltaNet Explained

All AI models might be the same

Life Update – On Health

Asymmetry of Verification and Verifier's Law

Soviet College Admission – My Dad's Story (1970)

H-Net – Inference

How to scale RL to 10^26 FLOPs

Britain is cheap, and should learn to love it

Database Sharding

Microdosing Willpower: My Takeaways from Microdosing Ozempic

The upcoming GPT-3 moment for RL

The Tradeoffs of SSMs and Transformers

Things you can do –with uv

The era of exploration

Just Ask for Generalization (2021)

Will Scaling Solve Robotics?

VLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention

LLM Memory

What Problems to Solve (1966)

Test-Time Training

Thnickels

SFStreets: History of San Francisco place names

Muon Doesn't Clearly Grok Faster

René Girard and Mimetic Theory for Non-Philosophers

Becoming a Better Programmer by Tightening Feedback Loops

Approximating Language Model Training Data from Weights

Street photos of every building in New York in 1939/1940

The Launch of GPT-4

Superintelligence, from First Principles

How to Shuffle a Big Dataset

Q-learning is not yet scalable

Why are neural networks and cryptographic ciphers so similar?

So You Want to Work in Mechanistic Interpretability?

Bowling Alone: America's Declining Social Capital [pdf] (1995)

Introduction to Parallel Programming with CUDA

Machine Learning of Sets

The Different Components of Intelligence

Cutlass Tutorial: Sub-Byte GEMM on Nvidia Blackwell GPUs

The case for more ambition (in AI research)

Why is AI hard and Physics simple?

Generative Modelling in Latent Space

Understanding the Neural Tangent Kernel

Specification Engineering is a bet on better code gen and more complexity

Efficient Streaming Language Models with Attention Sinks

PhDs for Entrepreneurs

The WeightWatcher tool for predicting the accuracy of Deep Neural Networks

Margins of My Dissertation: Life Lessons That My PhD Taught Me

Test-Time Training

LLM Visualization

ML models don't need that much data to be better than you

Ranking Foods by Protein Efficiency

What I learned about running a betting market game night contest

'To Easy LoL' – New Orleans jail break may have been inside job

Softmax – research that inspires us

Neural Network Checklist

Highly Opinionated Advice on How to Write ML Papers

You could have invented Transformers

The Annotated Kolmogorov-Arnold Network (Kan)

What Does Any of This Have to Do with Physics?

What I'm thinking about these days

How to cheat at settlers by loading the dice (2017)

Neural Vector Embeddings