Search papers, labs, and topics across Lattice.
This paper addresses the vulnerability of tabular in-context learning (ICL) to membership inference attacks, demonstrating that private records can be compromised through model predictions. To counter this, the authors introduce TabPATE, a differentially private defense mechanism that effectively protects private data without the need for public in-distribution datasets. Evaluations show that TabPATE maintains competitive performance on tabular benchmarks while significantly mitigating the risk of membership inference attacks.
Membership inference attacks can exploit tabular ICL, but TabPATE offers a robust defense that preserves model utility without requiring public data.
Tabular foundation models enable accurate in-context learning (ICL) from small labeled datasets, but the private records placed in context can leak through model predictions. We first show that even basic membership inference attacks succeed against tabular ICL, motivating formal privacy protection. We then introduce TabPATE, a differentially private PATE-style defense for tabular ICL that does not require public in-distribution data. TabPATE partitions the private context across teacher models, privately aggregates their labels on synthetic tabular queries, and releases the resulting labeled queries as a student context. Because tabular features are bounded and relatively low-dimensional, useful queries can be generated from feature ranges alone or from lightly privatized marginals. Across tabular benchmarks, TabPATE preserves competitive utility while reducing membership inference to near-random success, providing a practical path to private tabular ICL without public data.
