Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Hierarchical Instance-Conditioned Mixture-of-Experts (HI-MoE), a novel MoE architecture tailored for object detection that routes computation based on both scene-level and instance-level information. HI-MoE employs a two-stage routing mechanism: a scene router selects a subset of experts, and an instance router assigns object queries to experts within that subset. Experiments on COCO and LVIS demonstrate that HI-MoE outperforms dense DINO baselines and simpler MoE variants, particularly for small objects, while also showing preliminary evidence of expert specialization.
Object detection gets a MoE makeover: HI-MoE's hierarchical routing boosts performance, especially on small objects, by matching experts to both the scene and individual instances.
Mixture-of-Experts (MoE) architectures enable conditional computation by activating only a subset of model parameters for each input. Although sparse routing has been highly effective in language models and has also shown promise in vision, most vision MoE methods operate at the image or patch level. This granularity is poorly aligned with object detection, where the fundamental unit of reasoning is an object query corresponding to a candidate instance. We propose Hierarchical Instance-Conditioned Mixture-of-Experts (HI-MoE), a DETR-style detection architecture that performs routing in two stages: a lightweight scene router first selects a scene-consistent expert subset, and an instance router then assigns each object query to a small number of experts within that subset. This design aims to preserve sparse computation while better matching the heterogeneous, instance-centric structure of detection. In the current draft, experiments are concentrated on COCO with preliminary specialization analysis on LVIS. Under these settings, HI-MoE improves over a dense DINO baseline and over simpler token-level or instance-only routing variants, with especially strong gains on small objects. We also provide an initial visualization of expert specialization patterns. We present the method, ablations, and current limitations in a form intended to support further experimental validation.