---

CC - Cloud as Platform - Data Analysis, Streaming and Machine Learning in the cloud

Source: Slides + Cloud Computing for Science and Engineering by Ian Foster and Dennis B. Gannon

Cloud as Platform

The cloud is a lot more than a virtual computer: it is a rich ecosystem of services that can slash the expertise, time, and money required to build sophisticated applications.

For example, say that you want to build an application to monitor environmental sensors and alert you when a certain combination of sensors indicates anomalous behavior. Cloud services make each of them surprisingly easy.

In this regard, a cloud serves as a platform: an environment that allows you to develop, run, and manage applications without the need to set up, run, and maintain the hardware and software infrastructure that would otherwise be needed to host those applications.

Your application can, furthermore, scale to handle more events automatically, without you having to implement specialized load balancing logic. Need access control? Archiving? Auditing? Data analytics? Each of these capabilities is easily available.

Data Analysis

Public clouds were born as internal data centers to support services such as e-commerce, e-mail, and web search, each of which involved the acquisition of large data collections.

Each one of this service collect data, lot of data.

In order to optimize this services, companies begun to analyse large amount of data.

With the rising of the public clouds, analyzing methodologies were offered as services or open source software.

Universities and companies both contribute to create an enormous collections of open source tools and creative commons datasets.

Both Microsoft Azure and AWS offer Apache YARN as Map-Reduce like tools.

Hadoop and YARN

• See Apache Hadoop

Hadoop implements the Map-Reduce strategy.

Is is not necessary efficient, but it works effectively when processing a large size of data on a cluster of servers.

Hadoop Distributed File System (HDFS)

The Hadoop Distributed File System (HDFS) is a key Hadoop building block. Completely based on TCP/IP network sockets

When deployed, it has a single NameNode, used to track data location, and a cluster of DataNodes, used to hold the distributed data structures.

Files are partitioned in 64MB blocks. Distributed across DataNodes. Replicated in order to make the system fault tolerant.

HDFS is not a POSIX file system: it is write-once, read-many, and only eventually consistent.

Example:

wget https://dumps.wikimedia.org/enwiki/latest/enwiki-latest-pages-articles-multistream-index.txt.bz2
bunzip2 enwiki-latest-pages-articles-multistream-index.txt.bz2
hadoop fs -mkdir /users/wiki
hadoop fs -put enwiki-latest-pages-articles-multistream-index.txt /users/wiki/enwiki-latest-pages-articles-multistream-index.txt
hadoop fs -ls

• Create a directory in HDFS
• pull a copy of Wikipedia from a website
• push those to HDFS (where they are blocked, replicated and stored)
• List the directory

Hadoop and HDFS were originally created to support only Hadoop MapReduce tasks. However, the ecosystem rapidly grew to include other tools.

It DON’T support:

• Iterative Map-Reduce.
• Reuse of distributed data structures.

Yet Another Resource Negotiator (YARN)

Apache YARN is an evolution of Hadoop into a full distribution job management system, that has:

• Resource Manager.
• Job Scheduler.

It distributes processes on working nodes.

Applications connect to the resource manager, which then spins up an application manager for that application instance

The Application Manager interacts with the Resource Manager in order to obtain “containers”. The containers are executed on the working nodes.

This model allows multiple applications can run on the same cluster concurrently.

Spark

Designed in order to overcome the Hadoop Map-Reduce limitations.

In Hadoop’s linear dataflow structure, programs read input data from disk, map a function across the data, reduce the results of the map, and store reduction results on disk. Spark supports a more general graph execution model.

Spark enables:

• Iterative Map-Reduce.
• more efficient data reuse.

It is also much faster and interactive than Hadoop. It runs on YARN.

Resilient Distributed Dataset

A central Spark construct is the Resilient Distributed Dataset (RDD), a data collection that is distributed across servers and mapped to disk or memory, providing a restricted form of distributed shared memory.

Spark is implemented in Scala, an interpreted, statically typed object-functional language.

Spark has a library of Scala parallel operators, similar to the Map and Reduce operations used in Hadoop, that perform transformations on RDDs.

Spark Basic Operations:

• Transformation: map an RDDs into a new RDDs.
• Actions: return values to the main program.

In Spark, the partitions are distributed to the workers. Parallelism is achieved by applying the computational parts of Spark operators on each partition in parallel, using, furthermore, multiple threads per worker.

Map-Reduce on Public Clouds

• Deploying Hadoop on a cluster requires a team of professionals.
• Map-Reduce like services are available on public clouds by Amazon, Google, Microsoft…
• Using the Infrastructure as a Service a foundation, deploy ad-hoc virtual clusters is easy.
• Preconfigured list of tools.

Amazon Elastic MapReduce (EMR)

Amazon’s Elastic MapReduce (EMR) service makes creating a YARN cluster trivial. All you need to do is select your favorite combination of tools from their preconfigured lists, specify the instance type and number of worker nodes that you want, set up your usual security rules, and click Create cluster . In about two minutes you are up and running.

One interesting combination is called Zeppelin, similar to Jupyter that combines Spark and YARN.

When the EMR cluster comes up, it is already running one instance of Spark on YARN as well as HDFS.

---

Data Streaming

While batch analysis of big data collections is important, real-time or near- real-time analysis of data is becoming increasingly critical.

We refer to the activity of analyzing data coming from unbounded streams as data stream analytics.

Example:

• data from instruments that control complex systems such as the sensors onboard an autonomous vehicle or an energy power grid.

Data stream analytics - available tools:

• Spark Streaming (http://spark.apache.org/streaming)
• Twitter’s Storm (http://storm.apache.org)
• Apache Flink (http://flink.apache.org)
• Apache Beam (Flink+Spark+Google’s Cloud=Google’s Cloud Dataflow, http://beam.apache.org)
• Amazon Kinesis
• Azure Event Hubs

Data pre-analysis:

• Apache Edgent (http://edgent.apache.org )
• Works on Single Board Computers like Raspberry Pis

Use Cases:

• Wide-area Geophysical Sensor Networks.
  • Southern California Earthquake Center.
  • Geodesy Advancing Geosciences and EarthScope (GAGE) Global Positioning System (GPS).
  • Ocean Observatories Initiative (OOI).
• Urban Informatics/Smart Cities:
• The Array of Things (University of Chicago, Argonne National Laboratory, http://arrayofthings.github.io )
• DYNAMO, The Internet of Floating Things

DYNAMO

• (D)istributed leisure
• (Y)atch-carried
• sensor-(N)etwork
• for (A)tmosphere and (M)arine
• data cr(O)wdsourcing applications

Use Case:

• Leisure boats are equipped by marine electronics.
• Instruments and sensors gathering navigation data.
• Visualized and deleted!
• Sensors at the edge.
• Data stored on board…
• …send to the cloud when it is possible!

This use case requires:

• Huge amount of data
• Problem size characterized by a remarkable variability.
• Data have to be updated on time basis.
• Not homogeneous / quality proved data sources

Another use case is High Resolution Weather Models; the need for more accurate models for extreme weather events predictions.

For the “Internet of the floating things” DYNAMO is used with Signal K: SignalK is a modern and open data format for marine use. Internet friendly standard built on common web technologies, such as JSON and WebSockets.

Smart, cloud-enabled, multifunctional navigation system for yachts and leisure boats. DYNAMO collects, stores and sends to the cloud logged data from sensors. DYNAMO participates in a Internet of floating things ecosystem thanks to the SignalK data format.

Another use case for DYNAMO is the Data Movement, because DYNAMO is:

• Resilient because it takes advantage of short connection times using a dynamic thread generation algorithm;
• Lightweight, because data parcels could be compressed;
• HTTP-based, mainly using GET and POST with file attach verbs;
• Bidirectional, but loosely coupled;
• Firewall and proxy friendly, enforcing the security without a mandatory use of HTTPS;
• Application independent, could be used in different contexts;
• Fully customizable, because each feature could be deactivated, depending on the application design-

SQL Data Encoder

Design Challenges of Streaming Systems

Designers of streaming systems face a number of basic challenges, including cor- rectness and consistency

To present results from the analytics, is not possible to wait until the end of time. So instead you present results at the end of a reasonable time window.

If the processing is distributed and the window of time is short, you may not have a way to know the global state of the system, and some events may be missed or counted twice

Strongly consistent: event systems guarantee that each event is processed once and only once. Weakly consistent: system may give you only approximate results.

See also Strong and eventual consistency in DynamoDB

A second concern is the semantics of time and windows. Many event sources provide a time stamp when an event is created and pushed into the stream. However, the event will not be processed until a later time. Thus, we need to distinguish between event time and processing time.

Atleast four types of time windows exists:

• Fixed time windows: divide the incoming stream into logical segments, each corresponding to a specified interval of processing time.
• Sliding windows: allow for the windows to overlap: for example, windows of size 10 seconds that start everyfive seconds.
• Per session windows: divide the stream into sessions of activity related to some key in the data.
• Global windows can encapsulate an entire bounded stream.

Example: mouse clicks from a particular user may be bundled into a sequence of sessions of clicks nearby in time.

Associated with windows there must be a mechanism to trigger an analysis of the content of the window and publish the summary.

“Embarrassingly parallelism of data streams.”

Operations on streams often resemble SQL-like relational operators, but there are important differences. In particular a join operation is not well defined when streams are unbounded.

The natural solution involves dividing streams by windows in time and performing the join over each window.

Data Streaming with AWS

Amazon provides an impressive event-streaming software stack called Kinesis, comprising the following three services:

• Kinesis Streams: provides ordered, replayable real-time streaming data.
• Kinesis Firehose: designed for extreme scale, can load data directly into S3 or other Amazon services.
• Kinesis Analytics: provides SQL-based tools for real-time analysis of streaming data from Kinesis Streams or Firehose.

AWS Kinesis Streams

• Each Kinesis stream is composed of one or more shards. Think of a stream as a rope composed of many strands.
• Each strand is a shard, and the data that move through the stream are spread across the individual shards that make up the stream.

Producer-Consumer: Data producers write to the shards and consumers read from the shards.

Each shard can support writes from producers at up to 1,000 records per second, up to a maximum data write total of 1 MB per second.

No individual record can be bigger than 1 MB.

Reads by data consumers can be at most five transactions per second, with a total throughput of 2 MB/sec.

While these bounds may seem rather limiting, it is possible having streams with thousands of shards.

Big events must split up and spread over multiple shards.

Example of Kinesis with Boto3:

client = boto3.client('kinesis')
...
update_message = {
  "vessels": {
    "urn:mrn:signalk:uuid:705f5f1a-efaf-44aa-9cb8-a0fd6305567c": {
      "navigation": {
        "position": {
          "value": {
            "altitude": 0.0,
            "latitude": 37.81479,
            "longitude": -122.44880152
          },
          ...
          "timestamp": "2017-05-16T05:15:50.007Z"
        }
      }
    }
  }
}
 

client = boto3.client('kinesis')
 
data = json.dumps(update_message)
 
client.put_record(
	StreamName='DYNAMO',
	Data= bytearray(data),
	PartitionKey = 'a'
)

Every record must to have:

• a stream name,
• a binary encoded data component,
• and a way to identify the shard.

Sends our record to the stream.

Every record loaded into a shard has a sequence number.

To read the records, you need to provide a shard iterator.

One way is to specify a timestamp so that only the records that arrive after the specified time are read.

Another way is to request that the iterator be positioned to be the latest point in the stream so that only get the new records that come after that point are read.

Creating an iterator from a sequence number.

The shardID is needed: Obtain both the shardID and a starting sequence number for that shard by calling describe_stream(StreamName).

client = boto3.client('kinesis')
 
 
iter = client.get_shard_iterator(
	StreamName='DYNAMO',
ShardId='a',
ShardIteratorType = ' AT_SEQUENCE_NUMBER ',
StartingSequenceNumber='seqno')
 
iterator = iter['ShardIterator']
while True:
time.sleep (5.0)
resp = client.get_records(
ShardIterator =iterator)
iterator = resp['NextShardIterator']
analyzeData(resp['Records'])
 

Data Streaming Conclusions

Other open source tools: Apache Kafka, Storm, and Heron Streams

Data streaming is a key feature of the modern Platform as a Service Schema.

Costs have to be carefully evaluated in order to avoid to run out of the allocated budget.

---

Machine Learning in the cloud

Machine learning has become central to applications of cloud computing.

What has enabled these breakthroughs has been a convergence of the availability of big data plus algorithmic advances and faster computers that have made it possible to train even deep neural networks

Case uses:

• Predicting protein structure.
• Predicting the pharmacological properties of drugs.
• Identifying new materials with desired properties.
• Developing Covid19 vaccine in a fraction of the usual time.
• Detecting weather patterns in weather forecasts.
• Discovering and tracking “sub-grid” extreme weather events.

Spark Machine Learning Library (MLib)

Spark MLib provides a set of high-level APIs for creating ML pipelines.

It implements four basic concepts:

1. DataFrames: containers created from Spark RDDs to hold vectors and other structured types in a manner that permits efficient execution. Similar to Pandas dataframe
2. Transformers: operators converting one DataFrame to another.
3. Estimators: encapsulate ML and other algorithms.
4. Pipeline:
   • links Transformers and Estimators to specify an ML workflow.
   • Pipeline inherit the fit(...) method from the contained estimator.
   • Onece estimator is trained, pipeline model has atransform(...) method that can be used to push new cases through the pipeline to make predictions.

Many transformers are shipped with the APIs.

• Turn text documents into vectors of real numbers
• Convert columns of a DataFrame from one form to another
• Split DataFrames into subsets.

Estimators:

• Transform vectors by projecting them onto principal component vectors.
• n-gram generators taking text documents and returning strings of n consecutive words.
• Diverse and different classification models.

Classification models:

• logistic regression
• decision tree classifiers
• random forests
• naive Bayes.

Clustering methods:

• K-means
• latent Dirichlet allocation (LDA).

Azure Machine Learning Workspace

Azure Machine Learning is a cloud portal for designing and training machine learning cloud services.

• It is based on a drag-and-drop component composition model, in which you build a solution to a machine learning problem by dragging parts of the solution from a pallet of tools and connecting them together into a workflow graph.
• You then train the solution with your data.
• When you are satisfied with the results, you can ask Azure to convert your graph into a running web service using the model you trained.

Azure ML provides customized machine learning as an on-demand service.

Serverless computation:

• It does not require to deploy and manage VMs;
• The infrastructure is deployed as needed.
• If the web service needs to be scaled up because of demand, Azure scales the underlying resources automatically (automatic scaling).

Amazon Machine Learning Platform

• Provides an impressive set of ML services.
• Each service is designed to allow developers to integrate cloud ML into mobile and other applications.
• Deep Learning

Amazon Lex:

• Allows users to incorporate voice input into applications.
• Example NASA: Mars rover replica controlled by voice commands.

Amazon Polly:

• The opposite of Lex: it turns text into speech.
• Supports 27 languages with a variety of voices.
• Speech Synthesis Markup Language (SSML)
• Conversational computing

Polly and Lex do not do real-time. However, they provide a great platform to deliver voice based services.

Amazon Rekognition:

• Cutting edge of deep learning applications.
• Takes an image as input and returns a textual description of the items seen.
• Example:
  • Given an image of a scene with people, cars, bicycles,and animals, returns a list of those items, with a measure of certainty associated with each.

The service is trained with many thousands of captioned images in a manner not unlike the way natural language translation systems are trained.

Considers a million images containing a cat, each with an associated caption that mentions “cat,” and a model association is formed.

Rekognition can also perform detailed facial analysis and comparisons.

Amazon Machine Learning service:

• Allows you to create a predictive model-based on training data that you provide.
• Requires much less understanding of ML concepts than does Azure ML.

The dashboard presents the list of experiments, models, and data sources from previous Amazon Machine Learning work.

From the dashboard it is possible to define data sources and ML models, create evaluations, and run batch predictions.

One reason that it is so easy to use is that the options are simple. You can build only three types of models—regression, binary classification, or multiclass classification—and in each case, Amazon Machine Learning provides a single model.

Provides a single model for each case.

For instance, for multiclass classification:

• Multinomial logistic regression with a stochastic gradient descent optimizer.

Machine Learning on Cloud Conclusions

Private, hybrid and public clouds are the perfect playground for machine learning applications.

The use of the cloud resources can be done:

• Infrastructure-as-a-Service (IaaS): deploy tools as Spark ML or TensorFlow
• Platform-as-a-Service (PaaS): use services as Azure ML or Amazon Machine Learning

Mixing cloud services:

• Machine Learning
• Internet of Things
• Databases

---