Udemy – Writing production-ready ETL pipelines in Python / Pandas 2022-04

Published on: 2022-06-28 11:22:39

Categories: 28

Description

Writing production-ready ETL pipelines in Python / Pandas is the name of the course that will show each step to write an ETL pipeline in Python from scratch to production using the necessary tools such as Python 3.9, Jupyter Notebook, Git and Github, Visual Studio Code, Docker and Docker Hub and the Python packages Pandas, boto3, pyyaml, awscli, jupyter, pylint, moto, coverage and the memory-profiler. Two different approaches how to code in the Data Engineering field have been introduced and applied – functional and object oriented programming.

Best practices in developing Python code have been introduced and applied including design principles, clean coding, virtual environments, project/folder setup, configuration, logging, exception handling, linting, dependency management, performance tuning with profiling, unit testing, integration testing and dockerization.

What you’ll learn in this course:

How to write professional ETL pipelines in Python.
Steps to write production level Python code.
How to apply functional programming in Data Engineering.
How to do a proper object oriented code design.
How to use a meta file for job control.
Coding best practices for Python in ETL/Data Engineering.
How to implement a pipeline in Python extracting data from an AWS S3 source, transforming and loading the data to another AWS S3 target.

Who this course is for:

Data engineers, scientists and developers who want to write professional production-ready data pipelines in Python.
Everyone who is interested in writing data pipelines in Python that are ready for production.