Search papers, labs, and topics across Lattice.
This paper introduces scarHW, a novel network card design that enhances the scalability and performance of consistent data replication in datacenters by leveraging a new consensus algorithm called POPUC. Unlike traditional protocols that rely on mostly asynchronous coordination, POPUC employs a collaborative consensus approach, allowing multiple simultaneous decisions while maintaining safety guarantees even in the face of failures. The FPGA implementation of scarHW demonstrates remarkable improvements in throughput and latency for services like Redis and Zookeeper, achieving up to two orders of magnitude better performance than existing solutions.
Achieving up to 100x performance improvements in datacenter replication, scarHW challenges the limits of traditional consensus protocols.
Consistent replication of data among distributed processes -- a task involving the well-known consensus problem -- is notoriously expensive and hard to scale, affecting especially datacenter services with stringent performance requirements. To mitigate this problem, we introduce scalable replication in-hardware ( scarHW ): a network card design that improves throughput and latency of consistent replication even when increasing the number of replicas, whereas current systems operate at a small scale or with relaxed consistency guarantees. At the heart of scarHW is our novel POPUC consensus algorithm, implemented in an FPGA smartNIC to take full advantage of the"mostly synchronous"behavior of programmable network devices in the datacenter. Unlike widely-adopted"mostly asynchronous"coordination protocols such as Paxos or leaderless alternatives, POPUC implements a generalized variant of consensus dubbed collaborative consensus which allows for several simultaneous decisions, achieving great scalability without compromising availability. POPUC preserves safety guarantees in the presence of process crash-stop and message send/receive omission failures (capturing incidental asynchrony) and has been formally specified and verified in TLA+. Our FPGA prototype improves throughput and latency of widely-used services Redis and Zookeeper by up to two orders of magnitude compared to the state of the art. scarHW-based services also achieve zero downtime upon failure of a minority of replicas, offering a highly-robust, wire-speed, scalable replication system.
