Enhanced cluster computing performance through proportional fairness

Thomas Bonald (1, 2) , James Roberts (1, 3)

1 LINCS - Laboratory of Information, Network and Communication Sciences
2 LTCI - Laboratoire Traitement et Communication de l'Information
3 IRT SystemX

The performance of cluster computing depends on how concurrent jobs share multiple data center resource types like CPU, RAM and disk storage. Recent research has discussed efficiency and fairness requirements and identified a number of desirable scheduling objectives including so-called dominant resource fairness (DRF). We argue here that proportional fairness (PF), long recognized as a desirable objective in sharing network bandwidth between ongoing flows, is preferable to DRF. The superiority of PF is manifest under the realistic modelling assumption that the population of jobs in progress is a stochastic process. In random traffic the strategy-proof property of DRF proves unimportant while PF is shown by analysis and simulation to offer a significantly better efficiency-fairness tradeoff.

Domains: Networking and Internet Architecture [cs.NI]

Thomas Bonald, James Roberts. Enhanced cluster computing performance through proportional fairness. Performance Evaluation, 2014, 79, pp.134 - 145. ⟨10.1016/j.peva.2014.07.009⟩. ⟨hal-01112964⟩