Search papers, labs, and topics across Lattice.
This paper analyzes the challenges of integrating AI workloads directly into database management systems (AIxDB) to improve efficiency and security for AI-driven analytics. It identifies key challenges including query optimization, execution scheduling, distributed execution across heterogeneous hardware, and the need to adapt database components like transaction management and access control for AI lifecycle management. The paper presents a design and preliminary results that highlight key performance considerations for serving AIxDB queries.
Integrating AI into databases introduces a complex web of challenges, from query optimization to security, that demand a fundamental rethinking of traditional database architectures.
AI-driven analytics are increasingly crucial to data-centric decision-making. The practice of exporting data to machine learning runtimes incurs high overhead, limits robustness to data drift, and expands the attack surface, especially in multi-tenant, heterogeneous data systems. Integrating AI directly into database engines, while offering clear benefits, introduces challenges in managing joint query processing and model execution, optimizing end-to-end performance, coordinating execution under resource contention, and enforcing strong security and access-control guarantees. This paper discusses the challenges of joint DB-AI, or AIxDB, data management and query processing within AI-powered data systems. It presents various challenges that need to be addressed carefully, such as query optimization, execution scheduling, and distributed execution over heterogeneous hardware. Database components such as transaction management and access control need to be re-examined to support AI lifecycle management, mitigate data drift, and protect sensitive data from unauthorized AI operations. We present a design and preliminary results to demonstrate what may be key to the performance for serving AIxDB queries.
