Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study evaluates 12 multi-agent collaboration topologies for software architecture design through a controlled experiment utilizing a $2\times2\times2$ factorial design across 520 runs. Key findings reveal that the structural adversarial variant (v4b) outperforms others by demanding comprehensive rewrites, while cross-model review consistently ranks second, highlighting the importance of evaluator diversity in assessing design quality. The results indicate that certain collaboration methods, like parallel merge, are fundamentally flawed due to issues like token starvation, underscoring the need for careful topology selection in multi-agent systems.
Structural adversarial collaboration outperforms traditional methods, revealing that demanding rewrites rather than patches leads to superior software design outcomes.
We present a controlled experiment evaluating 12 multi-agent LLM collaboration topologies for software architecture design. Using a $2\times2\times2$ factorial design (Authority $\times$ Roles $\times$ Dynamics), we conducted 520 experimental runs across 8 design tasks of varying complexity, with 5 repetitions each. Designs were evaluated on a 12-dimensional rubric by three independent automated evaluators (GPT-OSS 120B, Claude Opus 4.6, Claude Sonnet 4.6). We report four core findings. First, structural adversarial (v4b) ranks #1 by ensemble -- a prompt-engineered adversarial variant that demands rewrite mandates rather than patches (weighted ensemble: 4.637/5.0). Second, cross-model review wins unanimously at #2 -- generate with one model, review with another -- ranking #2 by all three evaluators (weighted ensemble: 4.606). Third, evaluator diversity is itself a finding -- all three evaluators agree v4b is best and v3 is worst, but disagree sharply on v2b (Claude d=1.44 vs. GPT-OSS d=0.45), revealing how different model families weight design qualities. Fourth, parallel merge is fundamentally broken -- all three evaluators place merge variants in the bottom tier (3.65-3.79), due to token starvation and the Frankenstein effect. The weighted ensemble ($2\times$Opus + $2\times$Sonnet + $1\times$GPT-OSS) provides robust rankings across 520 runs, confirmed through independent cross-validation.