6533b851fe1ef96bd12a96e3

RESEARCH PRODUCT

Channel Capacity in Psychovisual Deep-Nets: Gaussianization Versus Kozachenko-Leonenko

subject

description

In this work, we quantify how neural networks designed from biology using no statistical training have a remarkable performance in information theoretic terms. Specifically, we address the question of the amount of information that can be extracted about the images from the different layers of psychophysically tuned deep networks. We show that analytical approaches are not possible, and we propose the use of two empirical estimators of capacity: the classical Kozachenko-Lonenko estimator and a recent estimator based on Gaussianization. Results show that networks purely based on visual psychophysics are extremely efficient in two aspects: (1) the internal representation of these networks duplicates the amount of information that can be extracted about the images with regard to the amount of information that could be obtained from the input representation assuming sensors of the same quality, and (2) the capacity of internal representation follows the PDF of natural scenes over the chromatic and achromatic dimensions of the stimulus space. This remarkable adaptation to the natural environment is an example of how imitation of biological vision may inspire architectures and save training effort in artificial vision.