Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces the effective rank (ERank) of channel covariance from deep feature maps as a novel, label-free measure of visual richness in images, derived from a single pass through a frozen pretrained encoder. The authors demonstrate that ERank effectively distinguishes between visually plain and rich images, correlating with codec bitrate, sharpness, edge density, and human complexity ratings, achieving a correlation coefficient of 0.72. Importantly, the study shows that using ERank as a data-selection criterion enhances performance in super-resolution and OCR tasks, indicating its practical utility in optimizing model training based on input richness.
ERank reveals that image complexity can be quantified efficiently, leading to significant performance gains in specific tasks like super-resolution and OCR.
We propose the effective rank (ERank) of the channel covariance of an image's deep feature map as a per-sample, label-free measure of visual richness, computed from a single forward pass through a frozen pretrained encoder. ERank counts how many decorrelated channel directions an image activates, and we characterize its properties, including its behavior under noise. Empirically, ERank orders images from plain to visually rich, correlates with codec bitrate, sharpness, and edge density, and correlates with human complexity annotations on IC9600 with $r = 0.72$. As a data-selection criterion, removing low-ERank samples improves super-resolution and removing high-ERank samples improves OCR, in both pretraining and finetuning, while selection does not help classification, segmentation, or denoising. ERank is thus a cheap richness signal, useful exactly when task difficulty is governed by input richness.