Search papers, labs, and topics across Lattice.
This paper introduces DNQ, a novel framework for training bidding agents in multi-turn simultaneous bidding scenarios, which serves as a model for partially observable n-player games. By employing a shared critic for payoff estimation and an external solver for equilibrium computation, DNQ significantly reduces the computational burden associated with traditional methods while maintaining effective policy imitation. Experiments demonstrate that the pairwise formulation of DNQ scales better with the number of agents compared to the exact formulation, highlighting a crucial trade-off between strategic fidelity and scalability in competitive environments.
Scaling bidding agents in multi-player games just got easier鈥擠NQ cuts training costs while maintaining strategic effectiveness.
Many real-world competitive systems require multiple decision-makers to act simultaneously under shared constraints, limited information, and repeated interaction, as in auctions, resource allocation, and security competition. We study multi-turn simultaneous bidding as a controlled testbed for such problems and propose DNQ, a solver-in-the-loop equilibrium supervision framework for training bidding agents. DNQ alternates between trajectory collection, critic-based payoff estimation, equilibrium computation, and policy imitation. At each visited state, a shared critic predicts either pairwise payoff matrices or an exact N-player payoff tensor, an external solver computes equilibrium strategies, and the agents are trained by minimizing the KL divergence between their masked policies and the solver-derived equilibrium targets. We focus on a scalable pairwise formulation that greatly reduces equilibrium-solving cost and training time compared with the exact formulation, while the shared critic amortizes payoff learning across agents and states. Experiments compare the pairwise and exact variants using critic loss, policy entropy, bidding resource usage, and training cost, showing that the pairwise method scales to larger numbers of agents, whereas the exact method becomes computationally impractical as the joint game grows. These results illustrate the trade-off between strategic fidelity and scalability in repeated competitive environments.
