X
Search results for:
No result found! Try with different keywords!
Users
Organizers
Events
Advertisement

Monitoring the Data Lake: Detecting Data Anomalies in ETL pipelines

Advertisement

Monitoring the Data Lake: Detecting Data Anomalies in ETL pipelines


Intermix.io presents a special 1-hour online training that introduces you to strategies and best practices for adding data validation tests and data quality monitoring to the tables in your data lake and data warehouse.
Why this training
The data in your data warehouse is mission critical. Reports are being used to make crucial company decisions. Your reputation is on the line to be sure of the accuracy of the data being reported. It is critical that you have confidence in your data pipeline execution.
Your data pipelines are complex and changing often. Regressions are possible from minor changes to DAGs and tasks. This may have unintended impacts on tables, and data flow which may not be discovered until much later and data has been already used in reports. Also, failure and DAG outages occur. You need confidence that when failures are fixed, that data is ‘flowing’ again and things are back to normal. Finally, complex ETL demands accuracy when data is mission critical. How do you know that your ETL is copying all rows accurately? That your joins are not dropping any data?
This training gives you practical, real-world examples of tests that can be added to any data pipeline to provide you with the confidence that things are working as expected.
What will I learn?
The class will include hands-on activities and provide pseudo-code examples of tests that can be run against your tables and data models. You will learn about the different classes of tests, how to set them up, and the important metrics to monitor.
How will this help my company and me?
Accurate business decisions, confidence that data quality is valid, and no more guesswork or surprises about data quality. Sounds great, right? Once you’re freed from worrying and fighting fires, you can refocus on your being creative with your data and have the confidence to make changes.



You may also like the following events from Eventbrite:

Also check out other Workshops in Bengaluru.

Liked this event? Spread the word :

Map Bangalore, Bengaluru, India
Loading venue map..
Event details from Report a problem

Are you going to this event?

Advertisement
Create an Event