Udemy – Prophecy Data Transformation Copilot for Data Engineering 2024-5

Published on: 2024-10-10 10:57:49

Categories: 28

Description

Prophecy Data Transformation Copilot for Data Engineering course. This course is designed to help data engineers and analysts build and deploy a cloud data lake architecture using Prophecy’s Data Transformation Copilot. This course is designed to help you start your data engineering journey with Spark and Prophecy. We’ll start by staging incoming data from application platforms like Salesforce, operational databases with CDC transactional data, and machine-generated data like logs and metrics. We are going to clean and normalize the input tables to prepare a complete, clean and efficient data model. From that data model, we will create four projects to create consumer applications for different real-world use cases.

What you will learn

Learning and designing a data lake model for an e-commerce company
A practical testing environment is provided with this course
Implemented and deployed Medallion architecture using Prophecy running on Databricks
Understanding Apache Spark and its best practices with real use cases
Share and expand pipeline components with data experts and analysts
Deployment pipelines in production and CI/CD and best practices
Use of version control and change management in data engineering
Deploy data quality checks and unit tests