Search papers, labs, and topics across Lattice.
This paper introduces Generative Meta-Learning with Human Feedback (GMHF), a framework designed to enhance generalization in machine learning models when faced with distribution shifts, particularly in scenarios with limited or no target domain data. By integrating expert intuition into the data synthesis process, GMHF employs a Conditional Neural ODE (cNODE) alongside a Reinforcement Learning agent to iteratively refine latent parameters, aligning generated data with human beliefs about the target environment. Empirical results demonstrate that GMHF significantly reduces deployment loss and divergence between generated and target data, especially as expert reliability increases, validating the theoretical bounds on generalization error derived in the study.
Human feedback can dramatically enhance model generalization in unseen environments, reducing deployment loss and divergence through expert-guided data synthesis.
Generalizing machine learning models to environments that differ from their training distribution remains a critical hurdle, particularly when data from the target domain is entirely or partially unavailable. We propose Generative Meta-Learning with Human Feedback (GMHF), a novel framework that bridges this domain gap by leveraging expert intuition to guide data synthesis. Grounded in a theoretical analysis of generalization error, we derive bounds demonstrating that aligning the distribution of generated data with human beliefs regarding the target physics significantly mitigates risk. GMHF operationalizes this insight by employing a Conditional Neural ODE (cNODE) as a generative digital twin, coupled with a Reinforcement Learning (RL) agent. The agent iteratively refines the latent physical parameters of the generated trajectories based on feedback, effectively steering the meta-learner toward the unobserved target distribution. Empirical validation on a nonlinear Duffing oscillator shows that GMHF substantially reduces deployment loss as expert reliability increases, and that the divergence between generated and target data falls under reliable feedback, directly corroborating the divergence-minimisation mechanism predicted by our theory. Further experiments on a non-dynamical probabilistic model confirm that the framework extends beyond ODE-governed systems, establishing human-AI collaboration as a rigorous catalyst for robust generalisation under distribution shift.
