Back in the old days, people used to do general-purpose GPU programming by using shaders like GLSL. This is what inspired NVIDIA (and other companies) to eventually create CUDA (and friends). This is an implementation of GPT-2 using WebGL and shaders. Enjoy!