Modeling human observer performance with neural network observers in a forced localization task using undersampled...

Modelling human performance in forced localization tasks is a challenge for model observers because of the many possible signal locations along with capturing human perception. This is particularly challenging when using anatomical images and needing generalization to out-of-distribution images such as in varying undersampling in magnetic resonance imaging (MRI). In this study, we modeled human forced localization (FL) performance using images with varied percentage of low frequencies fully collected in 5× MRI undersampling, ranging from 0% (aliasing) to 20% (blurring). We used a modified EfficientNet-B1 architecture trained to directly predict coordinates (FLNet) as well as using a biologically inspired V1Block, which incorporates Gabor filters to mimic early stages of human visual processing as a preprocessing to FLNet (V1FLNet). When trained on images with 20% of low frequencies collected, both neural networks surpassed average human performance but diverged from human-like performance on out-of-distribution data. Training on both 0% and 20% data, leading to interpolation in the generalization and more training data led to better results. Training on all the conditions resulted in the best results which also matched human performance and would lead to the same choice of which low k-space frequencies to collect as the human observer studies. The adaptive models, trained and evaluated across each of the seven conditions, performed better than the model trained on one condition but not as well as the model trained on all conditions. V1FLNet had similar performance as FLNet especially for the model that used all conditions for training.