Identification of letters distorted by physiologically-inspired spatial scrambling

In the geniculostriate pathway of the human visual system, neuronal projections carry signals from a particular retinal locus in parallel from one anatomical area to the next. Imprecision in the fidelity of these projections would place constraints on the ability of the system to perform tasks requiring positional information. We investigated the impact that "spatial scrambling" between stages would have on visual performance. We consider two stages in a simple canonical model of the early visual cortex where scrambling might occur: either the input to the first orientation-tuned mechanisms (analogous to V1 simple cells), or the output from those mechanisms. These are referred as "subcortical" (SCS) and "cortical scrambling" (CS). We developed a wavelet decomposition and resynthesis algorithm to mimic these effects, and measured human performance in letter identification affected by the two types of scrambling. Our results showed SCS and CS have distinguishable effects on both perceived noisiness of letters and letter identification threshold. Comparing human performance against a suite of pre-trained and custom convolutional neural networks (CNNs) that were trained on the scrambled stimuli, relative efficiency (calculated from the ratio of human:CNN thresholds) is higher for CS than SCS. However, in modelling human inefficiency by reducing the proportion of wavelets available to the CNNs, humans are less efficient in CS than SCS. These differences in efficiencies show humans are better at processing orientation redundant stimuli (CS) than orientation noisy stimuli (SCS). We hypothesize this reflects differences in integration properties at the input and output stages of simple cells in the cortex.