Search papers, labs, and topics across Lattice.
This paper introduces FlavourBench, an automated benchmarking system for evaluating language models using a culinary framework that provides dense, executable ground truth. By assessing 27 frontier models across 534 tasks involving ingredient selection and composition, the study eliminates biases from human judges and other models, yielding a robust ranking system. The results indicate that Grok 4.6 outperforms its peers with a FlavourBench Score of 65.1, highlighting significant correlations between independently compiled panels and resolving 101 model pairs.
FlavourBench reveals that automated culinary evaluations can provide a more reliable ranking of language models than traditional human or model-based assessments.
Open-ended language-model benchmarks usually inherit a judge: a human preference panel, another model, or a brittle exact-match key. We introduce FlavourBench, an automated benchmark in which a versioned culinary system supplies dense, executable ground truth. Each task presents eight ingredients and asks for a three-ingredient portfolio; before model execution, Epicure scores all 56 possible portfolios. We evaluate 27 frontier endpoints on an identical 534-task core spanning substitution, pairing, and constrained composition. Every ranked model has exactly 89 valid responses per panel and family (14,418 model-task cells total), eliminating differential missingness from the leaderboard. The FlavourBench Score is the equal-family mean of the frozen task scores. We use 50,000 anchor-cluster bootstrap replicates for simultaneous 95% score bands and 100,000 sign-flip draws for all 351 paired model contrasts, with Holm control. The two independently compiled panels correlate at r = 0.89 (rank rho = 0.80). Grok 4.6 has the largest point estimate at 65.1 (simultaneous 95% CI 61.0-69.2); 101 of 351 model pairs are resolved. The release includes the prompts, all portfolio score maps, raw responses, exact routes, content hashes, and an offline verifier that reconstructs every result.
