Obsidian + 🪴 Quartz 4.0

CC - Scaling in the Cloud

9 min read

CC - High Performance and Scaling in the Cloud

Public clouds are build from the ground up to allow customers to scale their services to fit their needs. The most common motivation for scaling in industry is to allow services to support thousands or millions of concurrent users.

Example:

  • A online media stream company can start deploying a single virtual machine instance.
  • At peak times instance 100s or 10.000s servers.
  • Then scale back when resources are not needed anymore.

Scientists and Engineers:

  • Run a bigger simulation or process more data.

Paradigms:

  • Single Program Multiple Data - SPMD
  • Many Task Parallelism - MTP
  • Bulk Synchronous Parallelism - BSP
  • Graph Execution Model - GEM
  • Microservices

Paradigms of Parallel Computing in the Cloud

Single Program Multiple Data - SPMD

Cloud providers deployed:

  • High-speed node types.
  • Specialized interprocessor communication hardware.
  • Accelerators (GPUs, FPGAs).

Now is possible achieve excellent HPCC on a modest number of cores/nodes.

Many Task Parallelism - MTP

A large queue or queues of tasks may be executed in any order, with the results of each task stored in a database or in files, or used to define a new task that is added to a queue.

Bulk Synchronous Parallelism (BSP)

BSP is based on processes or threads of execution that repeatedly perform independent computations, exchange data, and then synchronize at a barrier.

Cloud Message Passing Interface - MPI

Is the industrial standard for large HPC systems:

  • It is portable and modular.
  • Allows to exchange frequent messages at high speed.
  • Both amazon web services and microsoft azure offer HPC style clusters.

MPI applications on EC2 have comparable performance if compared with HPC on 256 nodes up to 1024 for some peculiar workloads (Cycle Computing, 2015)

GPU in the Cloud

GPUs are accelerating HPC since 10 years.

  • Dense Linear Algebra.
  • Deep Neural Networks.
  • DNNs are at the nexus of the in machine learning.

GPUs offered as specialized machine types on clouds.

  • Usually NVIDIA / CUDA.
  • Single Instructions on Multiple Data - SIMD.
  • Each instruction is applied on an array of data.

Microsoft Azure and Amazon Web Services offer instance types with multiple CUDA devices each running a couple of GPUs.

With a tiny cluster of those virtual machines is possible to reach a petaFLOPS.

Passthrough:

  • Virtual machine instance directly connected to the device.
  • No multiplexing.
  • High performance, high compatibility, high costs.

Deploying HPC cluster in AWS

Cloud Formation: A tool for automated deployment of complex cloud infrastructures with services configuration and resource management.

Cloud Formation Cluster - CfnCluster: A tool for automatic deployment of a private and custom HPC cluster. This tool is a set of Python scripts.

mkdir mycluster
cd mycluster
python3 -m venv venv
. venv/bin/activate
pip install cfncluster
cnfcluster configure

The command starts the cluster configuration:

Cluster Template [default]: MyCluster
AWS Access Key ID []:
AWS AWS Secret Access ID []:
AWS Region ID: us-east-1
VPC Name [public]:
Key Mame []: cloud_computing_class
Virtual Private Cloud (VPC) and subnet IDs:

Pipeline overview:

  • This command creates the cluster in about 10 minutes.

The “create” command returns a Ganglia URL. Ganglia is a well-known and frequently used cluster monitoring tool. Following that link takes you to a Ganglia view of your HPC cluster. It also uses a local scheduler called SLURM.

The autoscale is enabled by default, new nodes are instanced if needed. This works well for non MPI batch jobs. To delete the cluster use the delete command:

cfncluster delete mycluster

better configured cluster can be deployed creating a configuration file manually ~/.cnfcluster/config:

[Cluster mycloud]
...
compute_instance_type = c3.xlarge
Initial_queque_size = 4
maintain_initial_size = true
scheduler = slurm

One type of instance to use for HPC is c3.xlarge:

  • Enhanced networking.
  • Single-Root I/O Virtualization.
  • Reduce the latency, increase the bandwidth.
  • The virtual machine image is directly connected to the network interface card (NIC passthrough)

All the needed libraries are already installed on the image. The computing nodes are 4 and just 4 (no autoscale). Slurm is the local job scheduler.

cfncluster create mycluster This command creates the new configure cluster. Create a machinefile on the head node:

#include <stdio.h>
#include <mpi.h>
#include <stdlib.h>
main(int argc, char **argv) {
	char hostname[1024];
	gethostname(hostname,1024);
	printf(“%s\n”,hostname);
}

export PATH=/usr/lib64/mpich/bin:$PATH
export LD_LIBRARY_PATH=/usr/lib64/mpich/lib:$LD_LIBRARY_PATH
export I_MPI_PMI_LIBRARY=/opt/slurm/lib/libpmi.so

Prepare the environment. Compile and run:

mpicc ip-print.c -o ip-print.exe
srun -n 4 $PWD/ip-print.exe >> machines

The file machine will contain the IP addresses of the computing nodes.

This command creates the new configure cluster. In order to measure the performance of the cluster:

wget https://raw.githubusercontent.com/SciEngCloud/SciEngCloud.github.io/master/aws-hpc-cluster/ring-simple.c

mpicc ring-simple.c -o ring-simple.exe
mpirun -np 7 -machinefile ./machines $PWD/ring-simple.exe

The example measure the transit time, 70 microseconds. 10 times the typical value in a HPC, but with no Infiniband

Deploying HPC cluster in Microsoft Azure

Two different approaches.

The first is Azure’s service deployment orchestration device Quick Start:

  • Based on templates, like Amazon Cloud Formation.
  • The templates are stored on github.
  • For HPC-like applications the template is Slurm Cluster.
  • Define the cluster in terms of compute resources.

The second approach is Azure Batch, which support the management of large pools of VMs that supports:

  • Many task parallelism (MTP)
  • MPI jobs (SPMD)

Tasks are usually just command line scripts to be executed on each node.

Tasks are managed as pipelines:

  1. load binary and data;
  2. perform computing;
  3. stage out.

Scaling Further

Tightly coupled HPC applications can scale to modest-sized clusters. Why can’t they scale further?

  • As the number of nodes increases, the communication became heavier.
  • Supercomputers use specialized networks, while Cloud uses standard internet TCP/IP protocols over Ethernet switches
  • The two protocols have different emphasis.

Bisection bandwidth: A quantity that measure of how much time data traffic can flow from one half of the cluster to the other half in a specified period of time.

The networks used in supercomputers have an extremely high bisection bandwidth. The first cloud data centers had low bisection bandwidth, but with recent changes and redefining of the topology of networks, modern data centers are supplemented with more sophisticated and custom networks.

Nevertheless, one needs to consider the nature of the service level agreement (SLA) that cloud providers make with their users.

Many Task Parallelism - MTP

Scenario, suppose that you have:

  • Many samples to process.
  • Each processing is independent.
  • All the samples are queued.
  • A large number of workers take in charge the samples.

Each worker pulls input data from a queue, processes the data and stores the data in a shared storage.

Applications:

  • Genomics, DNA processing.
  • NCBI BLAST (Seattle Children’s Hospital)
  • Globus Genomics

Efficient mechanism leveraging on multiple workers pulling data from the storage and push them to a table.

Map Reduce & Bulk Synchronous Parallelism

A slightly more sophisticated approach to parallelism is based on a concept called bulk-synchronous parallelism (BSP).

This is used when worker tasks have to synchronize periodically and exchange data with each others.

Tasks proceed in parallel until a barrier imposes all tasks have to be completed before continue.

No computation is allowed after the synchronization point unless a condition is verified.

Map-Reduce is a form of BSP, for example consider the summation:

  • Given a sequence of input data objects with
  • A function to apply
  • The final result is:

Map on each then apply the sum (reduce).

A MapReduce computation:

  1. starts with a distributed data collection partitioned into non-overlapping blocks.
  2. It then maps a supplied function over each block.
  3. Next it applies the reduction. This is where the barrier comes in.
  4. It combines the result of the map operator in a treelike way, perhaps through several stages, until it produces the result.

Graph Dataflow Execution & Spark

MapReduce can be seen as a Directed Acyclic Graph group of tasks.

Define the computation as a graph of functions that are executed in a dataflow form until certain evaluation points are reached.

Each function node in the graph represents a parallel invocation of the function on distributed data structures, as shown in figure:

At this level, the execution is essentially BSP-style, and the evaluation points are barrier synchronizations.

The dataflow is unrolled in parallel at the execution time.

Spark (http://spark.apache.org) is a popular example of the graph dataflow diagram.

In Spark the control flow program is a version of SQL, Python or Scala. Conseguently they can be easily executed from a Jupyter Notebook.

Databricks:

  • Databricks provides an online development environment for Spark
  • The spark-enabled clusters can be deployed on AWS

Spark can also be deployed on Microsoft Azure and is included in AWS MapReduce.

HTCondor

High Throughput Condor is a really mature technology for distributed scientific calculations.

It is the underlying technologies of successful projects, such as:

Conclusions

  • The many task parallelism and the bulk synchronous parallelism are efficient and effective for “big-data” processing because tasks are loosely coupled or not coupled.
  • Applications frameworks as Map-Reduce and Spark are offered out the box by many cloud vendors as off the shelf tools.

Graph View

Backlinks