Neural Resonance
A Markovian theory of why generative models collapse when trained on their own output.
A representation of the five iterative feedback processes that resulted in seven feedback experiments.We train all diffusion models from scratch, use pre-trained CycleGAN models, and implement a functional analogue of Lucier's experiment.
As generative models flood the internet with synthetic images, text, and audio, the training sets of future models will inevitably contain the output of earlier ones. That creates a feedback loop — models trained on models — and it is known to end in model collapse, where quality and diversity degrade over generations. What was missing was an account of why: does collapse happen chaotically, or does it follow predictable dynamics?
This project, a preprint under review with David Crandall and Samuel Goree, gives that account. The central move is to model iterative feedback as a Markov chain across generations, where each new distribution of images depends only on the previous one. From that framing, a single phenomenon emerges that we call neural resonance: repeated feedback drives the latent representation toward a low-dimensional invariant subspace, damping most directions of variation while a few persist. The analogy is Alvin Lucier’s 1969 sound piece I Am Sitting in a Room, where re-recording speech in a room until only the room’s resonant frequencies survive — here the “room” is the model’s own geometry.
The theory identifies two conditions that are jointly necessary for resonance: ergodicity (the chain forgets where it started and converges to a unique stationary distribution) and directional contraction (latent features shrink toward progressively fewer axes). When both hold, collapse is orderly and predictable; when either fails — as with the non-ergodic CycleGAN and audio feedback loops we test — the system instead settles into disconnected attractors or drifts without bound. We make the argument concrete across diffusion models on MNIST and ImageNet, and introduce an eight-pattern taxonomy describing how local and global manifold geometry co-evolve under feedback.
Beyond explanation, the framework offers practical diagnostics: simple drift measures that reveal, generation by generation, whether a model is sliding toward collapse — and a finding that data compressibility governs the failure mode, with diverse datasets losing semantics far faster than compressible ones. The work suggests why models trained early on clean data hold an advantage, and where to intervene before degradation becomes irreversible.