Latency Aware Data Partitioning Techniques for Distributed Systems
Author(s): Naveen Kumar Bandaru
Publication #: 2602022
Date of Publication: 07.01.2024
Country: United States
Pages: 1-21
Published In: Volume 10 Issue 1 January-2024
DOI: https://doi.org/10.62970/IJIRCT.v10.i1.2602022
Abstract
Data partitioning plays a central role in distributed systems by determining where information is stored and how it is accessed across multiple nodes. The placement of partitions directly influences the communication path that each request must follow to retrieve or update data. Conventional partitioning strategies commonly rely on static placement rules such as hashing or predefined mappings that do not consider runtime access locality. Although these approaches simplify data distribution, they frequently lead to requests being served by remote nodes rather than nearby ones. As a result, each operation must traverse multiple intermediate nodes before reaching the target partition. This repeated traversal increases the average hop count per request. Every additional hop introduces communication delay, queuing overhead, and processing time at intermediate nodes. When the number of hops grows, requests experience longer travel paths across the cluster, leading to inefficient communication patterns. In large scale environments with many nodes, static placement often causes partitions to be widely scattered, further increasing the hop distance between clients and data. As workloads intensify, higher hop counts generate excessive cross node traffic and amplify network contention. These effects accumulate and limit the scalability of the system. Empirical observations show that systems using fixed partition placement exhibit steadily increasing hop counts as cluster size expands. The absence of locality awareness results in unnecessary network traversal and inefficient resource utilization. This paper addresses the problem of excessive hop count in distributed data access and focuses on improving partition placement to minimize the number of network hops required for each request.
Keywords: Partitioning, Latency, Hops, Locality, Scalability, Throughput, Placement, Clustering, Networking, Routing, Optimization, Scheduling, Distributed, Performance.
Download/View Count: 4
Share this Article