Search papers, labs, and topics across Lattice.
This paper introduces the bi-channel networking paradigm for cloud database systems, which separates communication into a high-performance data path using user-space UDP for latency-sensitive operations and a reliable control path leveraging kernel-based TCP for coordination. By co-designing networking with database operations, the authors address the inefficiencies of traditional kernel TCP stacks in modern high-speed networks, significantly reducing CPU overhead. The proposed approach achieves remarkable performance, demonstrated by saturating 200 Gbit/s in a distributed shuffle and processing millions of messages per second in a replicated key-value store.
Separating communication into high-performance and reliable channels allows cloud databases to achieve unprecedented throughput while maintaining essential guarantees.
When network links were slow, cloud and distributed database systems could rely on generic kernel abstractions and treat network communication as a black box. With today's fast cloud networks, this approach breaks down: database performance becomes limited by the CPU overhead of the kernel TCP stack. Replacing TCP with user-space UDP can reduce this overhead, but it requires reimplementing essential guarantees, such as reliability and ordering. To solve this conundrum, database systems should no longer treat networking as a black box but co-design it with database operations. We propose the bi-channel paradigm for database systems, which separates communication into two channels: A high-performance data path for latency- and bandwidth-sensitive operations, and a reliable control path for coordination and recovery. We implement the paradigm by combining user-space UDP and kernel-based TCP, though other stack combinations are possible. This design exploits modern NIC capabilities while preserving TCP's reliability. We demonstrate the paradigm's efficiency and simplicity in two representative settings: a distributed shuffle saturating 200 Gbit/s with three CPU cores, and a replicated key-value store processing millions of messages per second.
