To Make Language Models Work Better, Researchers Sidestep Language