On the Spatiotemporal Dynamics of Generalization in Neural Networks

Why do neural networks fail to generalize addition from 16-digit to 32-digit numbers, while a child who learns the rule can apply it to arbitrarily long sequences? We argue that this failure is not an engineering problem but a violation of physical postulates. Drawing inspiration from physics, we identify three constraints that any generalizing system must satisfy: (1) Locality – information propagates at finite speed; (2) Symmetry – the laws of computation are invariant across space and time; (3) Stability – the system converges to discrete attractors that resist noise accumulation. From these postulates, we derive – rather than design – the Spatiotemporal Evolution with Attractor Dynamics (SEAD) architecture: a neural cellular automaton where local convolutional rules are iterated until convergence. Experiments on three tasks validate our theory: (1) Parity – demonstrating perfect length generalization via light-cone propagation; (2) Addition – achieving scale-invariant inference from L=16 to L=1 million with 100% accuracy, exhibiting input-adaptive computation; (3) Rule 110 – learning a Turing-complete cellular automaton without trajectory divergence. Our results suggest that the gap between statistical learning and logical reasoning can be bridged – not by scaling parameters, but by respecting the physics of computation.