Search papers, labs, and topics across Lattice.
MemExchange is a novel multi-tenant memory management system designed to optimize memory utilization in cloud environments by dynamically reallocating in-memory caching resources based on real-time workload demands. By employing marginal-utility-based allocation and the Miss Ratio Curve (MRC) estimation, it effectively transforms isolated memory pools into a cohesive, logically aggregated resource, facilitating efficient memory exchange across physical nodes without centralized coordination. The implementation demonstrated significant improvements, achieving up to 2.3x lower remote-access overhead and a 63% reduction in cache miss rates for memory-constrained tenants, thereby enhancing overall cluster performance.
Transforming isolated memory pools into a cohesive resource can reduce cache miss rates by up to 63% for memory-constrained tenants.
To handle unpredictable workloads, cloud providers typically over-provision memory to meet peak demand, resulting in substantial underutilization across datacenter clusters. At the same time, memory-constrained tenants may suffer elevated cache miss rates, even when idle capacity remains stranded elsewhere in the infrastructure. MemExchange is a cluster-wide, multi-tenant memory management system that dynamically right-sizes in-memory caching tenants according to workload demand. Leveraging marginal-utility-based allocation derived from online Miss Ratio Curve (MRC) estimation, MemExchange redistributes idle memory between tenants across physical nodes using RDMA. This approach transforms the dedicated caching memory scattered across servers into a logically aggregated pool, enabling cross-node memory exchange without centralized coordination or forced tenant co-location. To support efficient remote access, we design the MemExchange Tracker Communication (MTC) protocol, an application-layer mechanism that coordinates memory reallocation and enables one-sided RDMA operations without involving remote CPUs. We implement MemExchange in Memcached and evaluate it through microbenchmarks, medium and rack-scale deployments of up to 100 CloudLab servers. Our results show up to 2.3x lower remote-access overhead compared to TCP-based designs, a 13% increase in cluster-wide memory utilization at rack scale, and up to 63% reduction in miss rate for memory-constrained tenants under skewed workloads.
Microsoft Research