Allocation of Graph Jobs in Geo-Distributed Cloud Networks
share
Recently, processing of big-data has drawn tremendous attention, where cloud computing is a natural platform. Big-data tasks can be represented as a graph job consisting of multiple sub-jobs, which must be executed in parallel with predefined communication constraints. This work develops the foundation for allocating graph jobs in geo-distributed cloud networks (GCs) with various scales. The problem of graph job allocation in GCs is formulated with respect to the incurred power consumption of the tasks, which turns out to be an integer programming problem. Considering the intractability of the solution, a sub-optimal analytical approach, suitable for small-scale GCs, is proposed. Further, in medium scale GDCNs, we address the graph-job allocation problem using a distributed algorithm. Finally, for large-scale GCs, given the intractability of the feasible set of allocations, we propose a novel algorithm called cloud crawling, which consists of a decentralized crawler exploring the network to determine the "potentially good" feasible allocations for the graph jobs. Based on the suggested strategies of the cloud crawler, we address the problem of graph job allocation from the proxy agents' perspective eliminating the burden of job allocation from the cloud datacenters (DCs). We address this problem under robust and adaptive pricing of the DCs, for each of which we propose an effective online learning algorithm.
READ FULL TEXT