Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces usage-aware experimental design, a novel framework that extends Bayesian experimental design (BED) to scenarios where external, potentially unknown decision-makers act on experimental data. The core innovation is training an adaptive design policy to minimize "usage risk," defined as the experimenter's expected loss considering these external rules, which can be non-Bayesian or black-box. This framework strictly generalizes standard BED by acknowledging that more informative data can sometimes be detrimental due to the actions of these external parties.
Experimental data can actively harm decision-making when external, unchangeable agents act on it, necessitating a new design paradigm beyond standard Bayesian methods.
Standard Bayesian experimental design (BED) methods assume the experimenter alone will act on the data, choosing the best action given their own beliefs and goals. In practice, data are often acted on by external parties whose procedures the experimenter cannot change and may not know, particularly when the data are released publicly. To extend BED to such settings, we introduce usage-aware experimental design, a framework in which an adaptive design policy is trained to minimise the usage risk: the experimenter's expected loss when one or more external rules act on the collected data. These rules may be non-Bayesian or black-box, and when several may apply, we either average over them or take the worst case. We show that, unlike in standard BED, more informative data can be harmful under such rules, so the usage risk cannot in general be expressed as an expected uncertainty reduction; our framework thus strictly generalises standard BED. We then use it to tackle a range of problems that are difficult to address with existing approaches.