Search papers, labs, and topics across Lattice.
This paper introduces Bag of Bags (BoB), a novel image representation for manuscript join retrieval that replaces the global visual codebook of Bag of Words (BoW) with a fragment-specific vocabulary of local visual words. BoB uses a sparse convolutional autoencoder to encode connected components from fragment patches, clusters the embeddings with per-image k-means, and compares images using set-to-set distances between local vocabularies. Experiments on Cairo Genizah fragments demonstrate that BoB outperforms the strongest BoW baseline, achieving a 6.1% relative improvement in top-1 accuracy.
Ditching the global visual vocabulary boosts manuscript fragment retrieval accuracy by 6%, proving that local context matters more than universal features when piecing together ancient texts.
A join is a set of manuscript fragments identified as originally emanating from the same manuscript. We study manuscript join retrieval: Given a query image of a fragment, retrieve other fragments originating from the same physical manuscript. We propose Bag of Bags (BoB), an image-level representation that replaces the global-level visual codebook of classical Bag of Words (BoW) with a fragment-specific vocabulary of local visual words. Our pipeline trains a sparse convolutional autoencoder on binarized fragment patches, encodes connected components from each page, clusters the resulting embeddings with per image $k$-means, and compares images using set to set distances between their local vocabularies. Evaluated on fragments from the Cairo Genizah, the best BoB variant (viz.\@ Chamfer) achieves Hit@1 of 0.78 and MRR of 0.84, compared to 0.74 and 0.80, respectively, for the strongest BoW baseline (BoW-RawPatches-$\chi^2$), a 6.1\% relative improvement in top-1 accuracy. We furthermore study a mass-weighted BoB-OT variant that incorporates cluster population into prototype matching and present a formal approximation guarantee bounding its deviation from full component-level optimal transport. A two-stage pipeline using a BoW shortlist followed by BoB-OT reranking provides a practical compromise between retrieval strength and computational cost, supporting applicability to larger manuscript collections.
