Search papers, labs, and topics across Lattice.
The paper introduces SPGen, a deep learning model based on a Fully Convolutional Neural Network (FCNN) with differentiable fixation selection and learnable Gaussian priors, to predict scanpaths of viewers observing paintings. To bridge the domain gap between photographs and artworks, the model employs unsupervised domain adaptation using a gradient reversal layer, and incorporates a random noise sampler to model the stochasticity of eye-tracking data. Experimental results demonstrate that SPGen outperforms existing methods in predicting scanpaths on paintings.
SPGen accurately predicts human eye movements on paintings by cleverly adapting models trained on natural images, opening new avenues for AI-driven art analysis and preservation.
Understanding human visual attention is key to preserving cultural heritage We introduce SPGen a novel deep learning model to predict scanpaths the sequence of eye movementswhen viewers observe paintings. Our architecture uses a Fully Convolutional Neural Network FCNN with differentiable fixation selection and learnable Gaussian priors to simulate natural viewing biases To address the domain gap between photographs and artworks we employ unsupervised domain adaptation via a gradient reversal layer allowing the model to transfer knowledge from natural scenes to paintings Furthermore a random noise sampler models the inherent stochasticity of eyetracking data. Extensive testing shows SPGen outperforms existing methods offering a powerful tool to analyze gaze behavior and advance the preservation and appreciation of artistic treasures.
