Linear Algebra and Duality of Neural Networks

Bases, mappings, projections and metrics, natural for Neural network training, are introduced. Graph-theoretical interpretation is offered. Non-Gaussianity naturally emerges, even in relatively simple datasets. Training statistics, hierarchies and energies are analyzed, from physics point of view. Duality between observables (for example, pixels) and observations is established. Relationship between exact and numerical solutions is studied. Physics and financial mathematics interpretations of a key problem are offered. Examples support all new concepts.