Adaptive Compression Scheduling for Network Efficient Data Transfers
Author(s): Vijaya Krishna Namala
Publication #: 2602021
Date of Publication: 16.08.2023
Country: United States
Pages: 1-20
Published In: Volume 9 Issue 4 August-2023
DOI: https://doi.org/10.62970/IJIRCT.v9.i4.2602021
Abstract
Modern distributed and cloud-based systems frequently transfer large volumes of data across networks for replication, backup, analytics, and service synchronization. These transfers often involve binary artifacts, logs, database snapshots, and container images that must be delivered quickly and reliably. While uncompressed transfers consume excessive bandwidth and increase transmission time, static compression approaches introduce continuous processor overhead even when network capacity is sufficient. This imbalance results in inefficient utilization of both network and computational resources. In many practical deployments, compression is applied indiscriminately to all data, leading to unnecessary encoding and decoding costs. For small or moderately sized files, the time spent compressing and decompressing may exceed the actual transmission time, increasing overall latency. Conversely, disabling compression during congested network conditions leads to larger payloads and prolonged transfer durations. As workloads scale and multiple nodes perform transfers concurrently, these inefficiencies accumulate, causing higher CPU utilization, reduced throughput, and longer completion times. The lack of runtime awareness in existing systems prevents optimal trade offs between bandwidth consumption and processing overhead. Empirical observations across distributed infrastructures indicate that static transfer pipelines fail to adapt to variations in file size, network congestion, and resource availability. This leads to inconsistent performance and limits scalability in latency sensitive environments. Excessive transfer time directly affects service responsiveness, deployment speed, and system efficiency. This paper addresses the problem of inefficient data movement in distributed systems and focuses on improving transfer time efficiency by enabling more balanced utilization of network bandwidth and processing resources during large scale data transfers.
Keywords: Compression, Scheduling, Networking, Transfers, Bandwidth, Throughput, Latency, Utilization, Efficiency, Adaptation, Runtime, Optimization, Scalability, Systems.
Download/View Count: 2
Share this Article