Infrastructure for Advanced Analytics and Machine Learning
Blockvorlesung im Sommersemester 2018
Prof. Dr. D. Kranzlmüller
Dr. Andre Luckow
Maximilian Höb
Aktuelles
Die Veranstaltung finden vom 03.04.2018-07.04.2018 statt.
Inhalt
The ongoing data deluge driven by the increasing digitalization of science, society and industry, leads to a significant increase in demand for data storage, processing and analytics within several industrial domains. Sciences and industry are overwhelmed by the need to store large amounts of transactional and machine-generated data resulting from the customer, service and manufacturing processes. Examples of machine- generated data are server logs as well as sensor data that is generated in finer granularities and frequencies. Further, datasets are often enriched with web and open data from social media, blogs or other open data sources. The Internet of Things (IoT) will further blur the boundaries between the physical and the digital world causing an even further increase in the digital footprint of the world. In this course, we will learn about data applications and their requirements. Further, we will discuss the core infrastructure necessary to handle the large data volumes and analytical problems. As part of the exercises students will utilize different frameworks, e.g. MapReduce and Spark to implement different algorithms.
This class will cover the following topics:
- Data Applications in Industry and Sciences
- Resource Management: YARN, Mesos and Kubernetes
- Hadoop Processing Engines: Spark, Flink
- SQL on Hadoop: Impala, Hive, Spark, Presto
- Stream Processing: Kafka, Spark Streaming, Flink, Heron
- Fault Tolerance: CAP Theorem, Eventual Consistency, Quorum Protocols, Apache Zookeeper
- Data in the Cloud: Elastic MapReduce, Azure HDInsight, Google Cloud Dataflow
- Machine Learning (MLLib)
- Natural Language Processing
- Deep Learning: Convolutional Neural Networks
- The course will be offered as a block lecture.
Hörerkreis
Die Vorlesung richtet sich an Master- und Bachelor-Studenten der Informatikstudiengänge.
Übungen
Aufgaben und Code zur Übung sind unter:
https://github.com/scalable-infrastructure/exercise-students-2018 verfügbar.
Umfang und Prüfung
Die Vorlesung ist zweistündig und besitzt eine Übungen (6 ECTS).
Die Endnote der Veranstaltung wird mit einer schriftlichen Prüfung ermittelt. Für die Zulassung zur Prüfung ist die Übung zu bestehen.
Für den Erfolg der Vorlesung müssen muss mindestens die Note 4 erreicht werden.
Vorkenntnisse
Besuch der Vorlesungen Rechnernetze und verteilte Systeme, Betriebssysteme, Rechnerarchitektur oder vergleichbare Kentnisse
erforderlich.
Programmierkenntnisse in Python und Umgang mit Linux Kommandozeile erforderlich.
Zeit und Ort
Zeit: 03. - 07. April 2018, 09 Uhr bis 16 Uhr
Ort: Oettingenstraße 67, Raum: 165 (VL), CIP-Pool Kalahari - BU 102 (Übung)
Anmeldung: Die Plätze werden über UniWorX vergeben werden: UniWorX-Bewerbung.
Wir bitten Sie in der Bewerbung Ihre Vorkenntnisse zu beschreiben und Ihre Teilnahme zu motivieren.
Schriftliche Prüfung
Zeit: 20.04.2018, 10:30 Uhr s.t.
Ort: Oettingenstraße 67,
Raum: BU 101
Notes:
- Absolutely no electronic devices of any kind allowed!
- Writing utensils are the only allowed aids.
- All questions must be answered in German or English.
- Please do not write in red color or with pencils.
- Please have both your student ID and government-issued photo ID ready.
- The duration of the exam is 90 minutes.
Contact
For questions or inquiries please contact Andre Luckow.
Downloads
Introduction
Hadoop and Spark
SQL on Hadoop and Data Science
Machine Learning
Scalable Machine Learning and Model Deployment
Solution to Exercises