Search papers, labs, and topics across Lattice.
This paper investigates the design of hash tables optimized for Remote Direct Memory Access (RDMA) in the context of IoT, where the growing number of devices and data demands efficient memory usage. It identifies critical challenges such as increased RDMA requests due to collision handling, the inadequacy of traditional concurrency controls, and the limitations of RDMA network interface resources. The authors provide a comprehensive review of existing approaches and propose optimization strategies to enhance the performance of remote hash tables in big-data applications.
Hash tables in RDMA environments can face significant performance bottlenecks due to increased network requests and concurrency issues, challenging traditional designs.
Hash tables complete the insertion, lookup, and deletion of a single key in constant time on average, and they are widely used in databases, key-value stores, and network systems. In the Internet of Things (IoT), the number of devices and the volume of sensed data keep growing, so the hash tables that store or index these data consume more and more memory. When a single server runs out of memory, the system can place part of the data in the memory of other nodes. One-sided operations in Remote Direct Memory Access (RDMA) let one machine read and write the memory of another machine directly, with low latency and high bandwidth, and are therefore widely used to build disaggregated memory systems. Deploying a hash table on RDMA-based remote memory exploits the memory of other nodes and thus relieves the capacity limit of a single server. However, this deployment raises three problems. First, one logical hash-table access may translate into one or more remote network accesses, and collision handling and probing further increase the number of RDMA requests. Second, because the remote CPU is bypassed, traditional concurrency control that relies on remote threads no longer applies directly. Third, the limited resources of RDMA network interface cards, such as queues, caches, memory registration, and atomic operations, impose new constraints on hash table structures. This paper focuses on hash table design for RDMA. We review existing work, distill the key challenges, and discuss promising optimization directions and coping strategies, aiming to provide a reference for designing remote hash tables in IoT big-data scenarios.
