Consider a neural network that has been training on modular arithmetic for fifty thousand steps. It can get most training examples right. Ask it a new one—something it has never seen—and it fails. The ...