Fair Scheduling in High Performance Computing Environments

This book introduces a new scheduler to fairly and efficiently distribute system resources to many users of varying usage patterns compete for them in large shared computing environments.  The Rawlsian Fair scheduler developed for this effort is shown to boost performance while reducing delay in high performance computing workloads of certain types including the following four types examined in this book:

i.        Class A – similar but complementary workloads

ii.      Class B – similar but steady vs intermittent workloads

iii.    Class C – Large vs small workloads

iv.    Class D – Large vs noise-like workloads

This new scheduler achieves short-term fairness for small timescale demanding rapid response to varying workloads and usage profiles.  Rawlsian Fair scheduler is shown to consistently benefit workload Classes C and D while it only benefits Classes A and B workloads where they become disproportionate as the number of users increases.

A simulation framework, dSim, simulates the new Rawlsian Fair scheduling mechanism. The dSim helps achieve instantaneous fairness in High Performance Computing environments, effective utilization of computing resources, and user satisfaction through the Rawlsian Fair scheduler.

Chapter 1 Introduction 1.- Chapter 2 Financial Market Risk 9.- Chapter 3 Scheduling in High Performance Computing 24.- Chapter 4 Fairshare Scheduling 33.- Chapter 5 Multi-Criteria Scheduling: A Mathematical Model 43.- Chapter 6 Simulation & Methodology 56.- Chapter 7 DSIM 67.- Chapter 8 Simulation Scenarios 73.- Chapter 9 Overview of Results 90.- Chapter 10 Class A Results and Analysis 101.- Chapter 11 Class B Results and Analysis 118.- Chapter 12 Class C Results and Analysis 139.- Chapter 13 Class D Results and Simulations 153.- Chapter 14 Conclusion 173.