The research paper 'Navigating the Latent Space Dynamics of Neural Models', accepted at ICLR 2026, explores autoencoders (AE) as dynamic systems. The study analyzes the latent space by treating the composition of the encoder and decoder as a mapping of the latent space onto itself.
Researchers define the latent space as a dynamic system where the encoder compresses input into a representation and the decoder reconstructs it. By repeatedly applying this composition, the model reveals latent structures, fixed points, and attractors that emerge as a byproduct of the training process.
The study uses a model trained on 10,000 MNIST images with a specific latent representation size. This setup allows for the visualization of vector fields and trajectories without information loss. The analysis distinguishes between discrete-time cascades and continuous-time flows, noting that iterations function as a form of Euler's method.
The authors identify fixed points where the encoder-decoder composition remains unchanged. These points act as attractors for nearby trajectories. The research highlights that the norm of the vector field increases with distance from training data, suggesting potential applications for out-of-distribution (OOD) detection.
