logo-polimi
Loading...
Risorse bibliografiche
Risorsa bibliografica obbligatoria
Risorsa bibliografica facoltativa
Scheda Riassuntiva
Anno Accademico 2021/2022
Scuola Scuola di Ingegneria Industriale e dell'Informazione
Insegnamento 056895 - STREAMING DATA ANALYTICS
Cfu 5.00 Tipo insegnamento Monodisciplinare
Docenti: Titolare (Co-titolari) Della Valle Emanuele

Corso di Studi Codice Piano di Studio preventivamente approvato Da (compreso) A (escluso) Insegnamento
Ing Ind - Inf (Mag.)(ord. 270) - MI (474) TELECOMMUNICATION ENGINEERING - INGEGNERIA DELLE TELECOMUNICAZIONI*AZZZZ056895 - STREAMING DATA ANALYTICS
Ing Ind - Inf (Mag.)(ord. 270) - MI (481) COMPUTER SCIENCE AND ENGINEERING - INGEGNERIA INFORMATICA*AZZZZ056895 - STREAMING DATA ANALYTICS
Ing Ind - Inf (Mag.)(ord. 270) - MI (487) MATHEMATICAL ENGINEERING - INGEGNERIA MATEMATICA*AZZZZ056895 - STREAMING DATA ANALYTICS

Obiettivi dell'insegnamento

The course provides the foundational concepts, methods, languages, and systems for ingesting, processing, and analyzing data that flows to enable real-time decisions. The course aims to the tame velocity dimensions of Big Data without forgetting the volume and variety dimensions.


Risultati di apprendimento attesi

Dublin Descriptors

Expected learning outcomes

Knowledge and understanding

Students will learn how to:

  • Identify problems that can be addressed with big data techniques tailored for velocity
  • apply the stream data analysis technologies for solving real-world problems

Applying knowledge and understanding

Given specific project cases, students will be able to:

  • Define and implement a streaming data analysis solution for the problem
  • Apply it on real data streams from social media and IoT sensors

Making judgements

Given specific project cases, students will be able to:

  • Learn how to decide which streaming data analysis solution to apply and how to evaluate this decision

Communication

Students will learn to:

  • Write a report on a project describing and motivating the decisions taken and the results obtained
  • Present their work in front of their colleagues and teachers

Lifelong learning skills

  • Students will learn how to develop a realistic streaming data analysis project in all its phases

 


Argomenti trattati

Foundations of streaming algorithms

  • when random access is forbidden and a polylog complexity is acceptable
  • the turnstile and cash register models
  • the sliding window model

Streaming Data Engineering

Streaming Data Science

  • foundations:
    • analysing time-series
    • detecting anomalies
    • Streaming Machine Learning
      • learning one sample at a time
      • prequential evaluation
      • concept drift
      • streaming algorithms for classification
  • systems and languages
    • using Flux to analyse time-series and detecting anomalies
    • using River for Streaming Machine Learning

Prerequisiti

Students are expected to know the basics about: database management and SQL


Modalità di valutazione

The exam consist of a theoretical part (written exam) and an optional practical part (project work with oral presentation)

The written exam is composed of a mix of theoretical questions regarding any course subjects and exercises regarding the technical content and how to apply it in practice. Students can get up to 30L in the written test.

The optional practical project requires to use of one or more of the technologies presented in the lectures. It consists in solving a realistic streaming data analysis problem based on real or realistic datasets publicly available or provided by the teachers. Only students, who will get at least 26/30 in the written exam, can opt for it.

The final grade is computed as follows: written text result + optional practical project result. E.g., written text 26 + optionalpractical project 5 = 30L

Type of assessment

Description

Dublin descriptor

Written test

  • Theoretical questions
  • Exercises focusing on streaming data analysis aspects

1,4

1, 2, 3

Assessment of project artefacts

  • Assessment of the design and implmenetation of the practical peoject work developed by the student

2, 3, 5

Oral presentation

  • Assessment of the presentation of the practical peoject developed by the student

2, 3, 4, 5

 


Bibliografia
Risorsa bibliografica facoltativaKreps, Jay, I Love logs: Event data, stream processing, and data integration., Editore: O'Reilly Media, Inc., Anno edizione: 2014 https://books.google.it/books?id=gdiYBAAAQBAJ&lpg=PR3&ots=3yOdY3Vi7S&dq=I%20love%20logs&lr&pg=PR3#v=onepage&q=I%20love%20logs&f=false
Risorsa bibliografica facoltativathe event processing language (EPL) https://esper.espertech.com/release-6.1.0/esper-reference/html/epl_clauses.html
Risorsa bibliografica facoltativa ksqlDB Documentation https://docs.ksqldb.io/en/latest/concepts/
Risorsa bibliografica facoltativaSpark Structured Streaming https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html
Risorsa bibliografica facoltativaFlux language https://github.com/influxdata/flux/
Risorsa bibliografica facoltativaGeoff Holmes, Ricard Gavaldà, Albert Bifet, Bernhard Pfahringer, Machine Learning for Data Streams: With Practical Examples in MOA, Editore: MIT Press, Anno edizione: 2018 https://www.google.it/books/edition/Machine_Learning_for_Data_Streams/0C9ZDwAAQBAJ?hl=en&gbpv=1

Software utilizzato
Nessun software richiesto

Forme didattiche
Tipo Forma Didattica Ore di attività svolte in aula
(hh:mm)
Ore di studio autonome
(hh:mm)
Lezione
29:00
30:00
Esercitazione
20:00
20:00
Laboratorio Informatico
0:00
0:00
Laboratorio Sperimentale
0:00
0:00
Laboratorio Di Progetto
1:00
25:00
Totale 50:00 75:00

Informazioni in lingua inglese a supporto dell'internazionalizzazione
Insegnamento erogato in lingua Inglese
Disponibilità di materiale didattico/slides in lingua inglese
Disponibilità di libri di testo/bibliografia in lingua inglese
Possibilità di sostenere l'esame in lingua inglese
Disponibilità di supporto didattico in lingua inglese
schedaincarico v. 1.15.13 / 1.15.13
Area Servizi ICT
11/09/2026