Search papers, labs, and topics across Lattice.
This paper introduces GraNatPy, a Python package that employs systematic metrics to enhance the realism and diversity of synthetic image datasets for deep learning applications in computer vision. By correlating quantifiable improvements in rendered scenes with higher zero-shot performance in object detection models, the authors provide a robust framework for bridging the domain gap between real and synthetic images. Additionally, the study reveals that gradient similarity significantly impacts small object detection performance, which can be optimized through a blend of real and synthetic data, further facilitated by an automated procedural rendering skill called SynthClaw.
Quantifiable metrics in synthetic image rendering can dramatically boost object detection performance by bridging the gap between real and synthetic data.
Deep learning computer vision for scientific applications requires collecting and annotating large datasets in a laborious, expensive and error-prone process. Synthetic data generation through 3D modelling and rendering may simplify this process and increase the accuracy of annotations by generating them programmatically. However, minimising the domain gap between real and synthetic images visually is subjective and lacks systematic quantitative guidance. We present GraNatPy, a Python package with metrics to guide improvement of the rendered scene. We show that quantifiable increase in realism, diversity and size of rendered dataset correlates with improved visual perception of the scene and higher zero-shot performance of an object detection model. Furthermore, we demonstrated using photographs of virological plaque assays that gradient similarity affects performance on small object detection, which can be improved by mixing real and synthetic data. Finally, we turn procedural data rendering into an agentic skill (SynthClaw) to automate the procedural parameter optimisation.
