Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces two lightweight architectures, RTM and PMG, for real-time weather attribute detection from images, framing weather conditions as variations in visual style. The architectures leverage Gram matrices, truncated ResNet-50, and PatchGAN-style discriminators within a multi-task learning framework. The models achieve high F1 scores on both internal and external datasets, demonstrating strong generalization with the PMG architecture running in real-time on embedded systems.
Real-time weather detection in embedded systems is now possible, thanks to a lightweight PatchGAN-based architecture that achieves >78% F1 score in zero-shot evaluation.
We present lightweight and efficient architectures to detect weather conditions from RGB images, predicting the weather type (sunny, rain, snow, fog) and 11 complementary attributes such as intensity, visibility, and ground condition, for a total of 53 classes across the tasks. This work examines to what extent weather conditions manifest as variations in visual style. We investigate style-inspired techniques, including Gram matrices, a truncated ResNet-50 targeting lower and intermediate layers, and PatchGAN-style architectures, within a multi-task framework with attention mechanisms. Two families are introduced: RTM (ResNet50-Truncated-MultiTasks) and PMG (PatchGAN-MultiTasks-Gram), together with their variants. Our contributions include automation of Gram-matrix computation, integration of PatchGAN into supervised multi-task learning, and local style capture through local Gram for improved spatial coherence. We also release a dataset of 503,875 images annotated with 12 weather attributes under a Creative Commons Attribution (CC-BY) license. The models achieve F1 scores above 96 percent on our internal test set and above 78 percent in zero-shot evaluation on several external datasets, confirming their generalization ability. The PMG architecture, with fewer than 5 million parameters, runs in real time with a small memory footprint, making it suitable for embedded systems. The modular design of the models also allows style-related or weather-related tasks to be added or removed as needed.