Search papers, labs, and topics across Lattice.
CARLA-GS introduces a modular pipeline for synthesizing corner cases in autonomous driving by decoupling visual representation, reasoning, and physics simulation while ensuring cross-module coupling. This approach leverages real driving data to create editable Gaussian scenes, employs a multi-agent LLM for scene reasoning and intent generation, and utilizes CARLA for low-level motion control, resulting in high-fidelity simulations. Experiments demonstrate that CARLA-GS achieves controllable corner-case generation with photorealistic and spatiotemporally consistent outputs, addressing a critical gap in safety evaluation for autonomous vehicles.
Photorealistic corner-case generation for autonomous driving is now achievable with a unified framework that balances high-level reasoning and low-level physics.
Safety evaluation for autonomous driving is dominated by rare, safety-critical interactions, motivating simulators that can deliberately synthesize corner cases with photorealistic observations. Corner-case generation is inherently a multi-source problem spanning visual representation, scene reasoning, and vehicle trajectory generation and control. Prior knowledge- and model-based approaches typically focus on scene or trajectory components in isolation, while diffusion-based methods attempt end-to-end generation but still struggle to ensure spatiotemporal consistency and physical realism. To unify these aspects within a single framework, we propose CARLA-GS, a modular corner-case synthesis pipeline that decouples visual representation, semantic reasoning, and physics-based execution while maintaining tight cross-module coupling. Starting from real driving data, we reconstruct an editable gaussian scene with additional geometry-consistent constraints. A multi-agent LLM then performs scene-level reasoning to identify risky interactions and generate intent-level waypoint trajectories, while the low-level motion control is delegated to CARLA, where a PID controller ensures kinematic and dynamic feasibility. The simulated vehicle states are finally re-projected into the gaussian scene for ego-centric rendering. This design enables high-level semantic reasoning, low-level physically executable motion, and photorealistic corner-case generation within a unified pipeline. Experiments on the Waymo Open Dataset show, both quantitatively and qualitatively, that our framework enables controllable corner-case generation and produces photorealistic, spatiotemporally consistent videos aligned with semantic intent and physically feasible motion.
