Extraction, Transformation, Loading and Visualization

Base Knowledge

Knowledge acquired in the Data Analysis and Database Course Units.

Teaching Methodologies

Classes will be theoretical and practical, with presentation of content and application of concepts based on a Case Study to be worked on by groups of students – the objective is to allow students to deal with:

– preparation of data for analysis and decision-making;

– extraction, processing and loading of data to support analysis;

– construction of data visualizations oriented towards decision-making;

– use of data extraction, processing, loading and visualization tools, making the most of the education licenses available at the School.

Always with the aim of using real company data whenever possible, various examples will be worked on in class to allow the necessary monitoring for the acquisition of knowledge, and hours of independent work for students will also be planned.

Students will be organized into groups of 3 or 4 students so that they can develop exercises in class based on the examples mentioned above and providing discussion and exchange of ideas, essential for the Project work in the second half of classes in the Semester.

To complement group work, groups will be called for presentations during the Semester, contributing to the exchange of ideas and knowledge of the whole class. The presentations will be supported by peer-review assessment between groups, that is, each group must present their work and answer questions in addition to analyzing the work of another group, proposing suggestions for improvement and addressing constructive criticism.

The School promotes extracurricular events provided by partner entities. In this context, a Meet Up will be held with one of the partner companies, focusing on the new functionalities planned for the tools in this area of ​​study and use cases in the related companies. Establishing connections with the Curricular Units of the last semester of the Undergraduate Course and promoting and expanding the network of business knowledge, enhancing its opportunities for connection with the job market, are also premises included in the Meet-Up networks.

Finally, and taking into account the excellence of learning, students are optionally given the opportunity to participate in the submission of a Scopus indexed conference, in the form of a scientific article, which includes the work developed in order to promote research activities among these graduation students.

Learning Results

The learning objectives of this subject are as follows:

O1 – Understand the real-life ETL (Extraction, Transformation, Loading) cycle.

O2 – Master the different stages of the ETL process, from establishing the connection to the knowledge acquired in the Data Analysis Curricular Unit, through to the data transformation component.

O3 – Understand, compare and use ETL tools.

O4 – Understand and apply data visualisation rules that are appropriate to the context of the data users to promote better understanding and decision-making based on the data.

O5 – Know how to use the most appropriate techniques for the data under analysis to obtain the most effective visualisations possible by going through the ETL process.

 

At the end of the course, students should have acquired the following skills:

C1 – Understand the ETL process in a real context (data sources, extraction, cleaning, transformation, integration, and loading);

C2 – Design and implement complete ETL processes: from raw data to analysis and visualization;

C3 – Apply extraction techniques to data sources with diverse characteristics (databases, files, APIs, etc.);

C4 – Perform data transformation operations by combining this knowledge with that of Data Analysis;

C5 – Perform loading processes for data warehouses, data marts, or analytical platforms.

Program

1 – Data Concepts, Data-driven Culture and ETL
1.1 – Data Analysis Cycle
1.2 – Location and types of Data Sources: Data Extraction
1.3 – Data Preparation and Transformation Processes
1.4 – Data warehouses vs Data Lakes: Data loading
1.5 – Data Governance Principles and associated professional profiles
1.6 – Data Curation
1.7 – Ethical Aspects of Data Use and General Data Protection Regulation
1.8 – Open Repositories
1.9 – Tools for ETL and comparative study
2 – Data Visualization
2.1 – Rules for preparing data visualization
2.2 – Tools for data visualization: identification and comparative study
2.3 – Planning, monitoring and discussion of the data visualization process
3 – ETL design and data visualization

Curricular Unit Teachers

Isabel Maria Mendes Pedrosa

Grading Methods

The assessment consists of 2 practical assignments (TP1 and TP2) and the preparation of a Meet Up event (PE), if possible in collaboration with a company in the area.

The TP1 and PE components will be carried out in groups and the TP2 component will be mandatory individual in nature.

 NOTE: Continuous assessment requires attendance at least 75% of the classes during the semester. 

 

Continuous Assessment

Group

TP1 – presentation in a room with pitch (max 10m) – 40% of the final grade

          Includes delivery after presentation of Project Report

NOTE TP1: Project*30% + Report*30% + Presentation*20% + Discussion*20%

PE – preparation of an event (PE) Meet Up – 20% of the final grade

Individual

TP2 – presentation in a room with pitch (max 10m) – 40% of the final grade

           Includes delivery after presentation of Project Report

NOTE TP2: Project*30% + Report*30% + Presentation*20% + Discussion*20%

NOTE: Continuous assessment requires attendance at least 75% of the classes during the semester. 

 

Assessment by Exam – always carried out individually

Normal Season

TP1 – presentation in a classroom with pitch (max 10m) – 50% of the final grade

          Includes delivery after presentation of Project Report

NOTE TP1: Project*30% + Report*30% + Presentation*20% + Discussion*20%

TP2 – presentation in a room with pitch (max 10m) – 50% of the final grade

          Includes delivery after presentation of Project Report

NOTE TP2: Project*30% + Report*30% + Presentation*20% + Discussion*20%

Appeal Exam Season

TP1 – presentation in a classroom with pitch (max 10m) – 50% of the final grade

          Includes delivery after presentation of Project Report

NOTE TP1: Project*30% + Report*30% + Presentation*20% + Discussion*20%

TP2 – presentation in a room with pitch (max 10m) – 50% of the final grade

          Includes delivery after presentation of Project Report

NOTE TP2: Project*30% + Report*30% + Presentation*20% + Discussion*20%

Special Season

TP1 – presentation in a room with pitch (max 10m) – 50% of the final grade

          Includes delivery after presentation of Project Report

NOTE TP1: Project*30% + Report*30% + Presentation*20% + Discussion*20%

TP2 – presentation in a room with pitch (max 10m) – 50% of the final grade

          Includes delivery after presentation of Project Report

NOTE TP2: Project*30% + Report*30% + Presentation*20% + Discussion*20%


    Internship(s)

    NAO

    Bibliography

    Aspin, A. (2022). Pro Data Mashup for Power BI: Powering Up with Power Query and the M Language to Find, Load, and Transform Data, Apress Editions
    Deckler, G. (2025). Learn Power BI: A comprehensive, step-by-step guide for beginners to learn real-world business intelligence, 2rd Edition, Packt Publishing
    Deckler, G., Powell, B. (2022). Mastering Microsoft Power BI: Expert techniques to create interactive insights for effective data analytics and business intelligence, 2nd Edition, Packt Publishing
    Kimbal, R., (2004). The Data Warehouse ETL Toolkit: Practical Techniques for Extracting, Cleaning, Conforming, and Delivering Data
    Laursen, G. H., & Thorlund, J. (2016). Business analytics for managers: Taking business intelligence beyond reporting. John Wiley & Sons.
    Few, S. (2021). Now You See It: Simple Visualization Techniques for Quantiative Analysis, Analytics Press.
    Few, S. (2019). The Data Loom: Weaving Understanding by Thinking Critically and Scientifically with Data, Analytics Press.