Unsupervised learning predicts human perception and misperception of gloss

Reflectance, lighting and geometry combine in complex ways to create images. How do we disentangle these to perceive individual properties, such as surface glossiness? We suggest that brains disentangle properties by learning to model statistical structure in proximal images. To test this hypothesis, we trained unsupervised generative neural networks on renderings of glossy surfaces and compared their representations with human gloss judgements. The networks spontaneously cluster images according to distal properties such as reflectance and illumination, despite receiving no explicit information about these properties. Intriguingly, the resulting representations also predict the specific patterns of 'successes' and 'errors' in human perception. Linearly decoding specular reflectance from the model's internal code predicts human gloss perception better than ground truth, supervised networks or control models, and it predicts, on an image-by-image basis, illusions of gloss perception caused by interactions between material, shape and lighting. Unsupervised learning may underlie many perceptual dimensions in vision and beyond

Artificial neural network · Cognitive psychology · Computer vision · Generative grammar · Generative model · Illusion · Optics · Perception · Physics · Specular reflection · Aesthetic Perception and Analysis · Color Science and Applications · Computer Science · Neuroscience · Psychology · Visual perception and processing mechanisms · Artificial Intelligence

• Material perception for philosophers

  Open Access•J Brendan Ritchie, Vivian C Paulun et al.•Philosophy Compass•2021

  Incluir item

• The Architecture of Object-Based Attention

  Open Access•Patrick Cavanagh, Gideon P Caplovitz et al.•Psychonomic Bulletin & Review•2023

  Incluir item

• Material category of visual objects computed from specular image structure

  Open Access•Alexandra C Schmid, Pascal Barla et al.•Nature Human Behaviour•2023

  Incluir item

• The free-energy principle

  Open Access•Karl J Friston, Karl Friston•Trends in Cognitive Sciences•2009

  Incluir item

• Representation Learning

  Open Access•Yoshua Bengio, Aaron Courville et al.•IEEE Transactions on Pattern…•2013

  Incluir item

• Pingouin

  Open Access•Raphael Vallat•Journal of Open Source Software•2018

  Incluir item

• Reducing the Dimensionality of Data with Neural Networks

  Open Access•Geoffrey E Hinton, Ruslan Salakhutdinov et al.•Science•2006

  Incluir item

• ImageNet Large Scale Visual Recognition Challenge

  Open Access•Olga Russakovsky, Jia Deng et al.•International Journal of Computer…•2015

  Incluir item

• Some informational aspects of visual perception.

  Fred Attneave•Psychological Review•1954

  Incluir item

• Deep learning

  Open Access•Yann LeCun, Yoshua Bengio et al.•Nature•2015

  Incluir item

• Letter perception emerges from unsupervised deep learning and recycling of natural image features

  Open Access•Alberto Testolin, Ivilin Stoianov et al.•Nature Human Behaviour•2017

  Incluir item

• Material category of visual objects computed from specular image structure

  Open Access•Alexandra C Schmid, Pascal Barla et al.•Nature Human Behaviour•2023

  Incluir item