Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces SceneMiner, a novel pipeline for mining challenging driving scenarios using only camera data, which addresses the lack of difficulty labels in safety-critical scene identification. By leveraging a frozen vision-language backbone, SceneMiner generates multiple complementary outputs—including a retrieval embedding, a multi-label scene-tag distribution, and a continuous risk score—while overcoming the issue of cross-task interference through identity-preserving multi-task fine-tuning. The method achieves a mean Average Precision (mAP) of 0.4614 for scene tagging, demonstrating its effectiveness in identifying complex scenarios without additional sensor data.
Cross-task interference in multi-task models can degrade performance, but SceneMiner's innovative fine-tuning approach preserves task identities while enabling effective scene mining from camera data alone.
Mining hard, safety-critical scenes from driving logs is bottlenecked by the absence of difficulty labels, and no single proxy, collision risk, trajectory ambiguity, or semantic rarity suffices to find such scenes on its own. We present SceneMiner, a unified, camera-only bird's-eye-view pipeline that emits complementary mining signals from a frozen vision-language backbone in a single forward pass, with no LiDAR or radar: a retrieval embedding for text-prompted scenario search, a multi-label scene-tag distribution, and a continuous physics-based risk score (a motion forecast is a byproduct, not a contribution). Building such a multi-head model exposes our central finding, a failure mode we term cross-task interference: adding or upgrading one head shifts a shared activation stream and degrades weight-frozen sibling heads, so freezing parameters alone is insufficient. Our contribution, identity-preserving multi-task fine-tuning, removes this interference by zero-initializing every new sub-module and freezing every parameter that feeds the shared stream. The mining heads are thereby preserved bit-identically while training only ~102k parameters. The tagging head reaches mAP 0.4614 (micro-F1 0.5557) on 20 scene tags by pooling each scene into 32 visual tokens, and the embedding head supports text-prompted retrieval, validated qualitatively. Code is available at: https://anonymous.4open.science/r/sceneminer_anonymous-64E5