Search papers, labs, and topics across Lattice.
This paper introduces a three-layer architecture, the Separable Expert Architecture (SEA), to enable privacy-preserving LLM personalization. SEA uses a static base model, composable domain-expert LoRA adapters, and per-user proxy artifacts to decouple personal data from shared weights, enabling deterministic unlearning via proxy deletion. Experiments on Phi-3.5-mini and Llama-3.1-8B demonstrate effective per-user differentiation and successful unlearning, confirmed by a return to baseline performance after proxy removal and minimal cross-user contamination.
Achieve LLM personalization with the guarantee that deleting a small user-specific proxy deterministically erases all traces of their data, sidestepping the need for computationally expensive retraining.
Current model training approaches incorporate user information directly into shared weights, making individual data removal computationally infeasible without retraining. This paper presents a three-layer architecture that decouples personal data from shared weights by combining a static base model, composable domain-expert LoRA adapters that shape behavior without imparting user data, and per-user proxy artefacts whose deletion constitutes deterministic unlearning. Evaluation on Phi-3.5-mini and Llama-3.1-8B confirms per-user differentiation in which personal data influences outputs while remaining isolated, verified by a return to baseline after proxy removal (KL divergence of approximately 0.21 nats, 82-89% verification pass rate) and near-zero cross-user contamination. Because user-specific information never enters shared weights, the architecture mitigates model inversion, membership inference, and training-data extraction against shared model components by construction. The approach converts machine unlearning from an intractable weight-editing problem into a deterministic deletion operation that preserves personalization alongside privacy-enhancing guarantees and is compatible with differentially private stochastic gradient descent (DP-SGD) for privacy-preserving shared model improvement.
