Udemy – Data Engineering using AWS Data Analytics 2024-11

Published on: 2024-11-19 18:04:17

Categories: 28

Description

Data Engineering using AWS Data Analytics course. This course will help you design and implement data pipelines using the AWS Data Analytics platform. During this course, you will learn about various services such as Glue, EMR, Lambda, Athena, Kinesis and many others.

What you will learn

AWS Essentials: Familiarity with basic AWS services such as s3, IAM and EC2
AWS Glue: Design and implement data stacks using Glue Jobs and Glue Workflows
AWS Athena: Run SQL queries on data stored in S3
AWS EMR: Using EMR clusters to process big data using Apache Spark and Hadoop frameworks
AWS Lambda: Executing serverless functions to process events and data
AWS Kinesis: Design and Implementation of Streaming Data Pipelines
AWS Redshift: Create analytical databases and run complex queries on massive data
Data Engineering Using AWS Data Analytics Services: In this course, you will be introduced to a wide range of AWS data analytics services such as Glue, EMR, Athena, Kinesis, and Redshift and learn how to use them to create data pipelines.
AWS Basics: You will learn the basic concepts of AWS such as S3 (cloud storage), IAM (identity access management), EC2 (virtual machines) and other related services.
In-depth understanding of AWS S3: You will learn how to use S3 to store data in the cloud.
Understanding EC2 Details: You will learn about EC2 virtual machines and how to manage them in AWS.
Managing IAM Access: You’ll learn how to manage users, groups, roles, and IAM policies to control access to AWS resources.
Managing Tables with AWS Glue Catalog: You will learn how to create and manage tables in the Glue Catalog.
Engineering Data Pipelines with AWS Glue Jobs: You’ll learn how to create and manage data pipelines using Jobs in Glue.
Coordinate Data Pipelines with AWS Glue Workflows: You will learn how to coordinate and manage the workflow of data pipelines using Workflows in Glue.
Running Queries with AWS Athena: You’ll learn how to run queries on data using the Athena serverless query engine.
Using AWS EMR Clusters: You’ll learn how to use EMR clusters to build data pipelines, create reports, and dashboards.
Get data using AWS Lambda functions: You will learn how to use Lambda functions to get data.
Scheduling with AWS Events Bridge: You will learn how to schedule task executions using Events Bridge.
Engineering Data Flow Pipelines with AWS Kinesis: You will learn how to create and manage data flow pipelines using Kinesis.
Streaming Web Server Logs with AWS Kinesis Firehose: You will learn how to stream web server logs to Kinesis Firehose.
Overview of Data Processing with AWS Athena: You will learn the basics of data processing using Athena.
Running Athena queries with CLI and Python: You will learn how to run Athena queries using the command line and the boto3 Python library.
Creating an AWS Redshift cluster: You will learn how to create a Redshift cluster, create tables, and perform CRUD operations.
Copy data from S3 to Redshift tables: You will learn how to copy data from S3 to Redshift tables.
Understanding distribution styles and creating tables with Distkeys: You will learn about distribution styles in Redshift and how to create tables using Distkeys.
Querying External RDBMS Tables Using Redshift Federated Queries: You will learn how to query external tables in relational databases using Redshift Federated Queries.
Querying Glue or Athena Catalog Tables Using Redshift Spectrum: You will learn how to query tables in the Glue or Athena catalog using Redshift Spectrum.
Familiarity with AWS: setting up the development environment, creating S3 buckets, managing users and roles with IAM.
Data storage: working with S3, understanding different storage classes, using Glacier.
Data Entry: Using Lambda Functions, Examining Glue Components, Setting Up Spark History Server, Working with Glue Catalog and Job APIs.
Data processing: working with EMR, deploying Spark applications, creating streaming pipelines with Kinesis.
Storage of processed data: using Redshift, copying data from S3 to Redshift, developing Redshift applications, using Federated Queries and Spectrum.
Data analysis: working with Athena, using Athena with CLI and boto3.

This course is suitable for people who:

Beginner or intermediate data engineers who want to learn about AWS Analytics Services
Intermediate software engineers who want to learn data engineering using AWS Analytics Services
Data and analytics engineers who want to learn about data engineering using AWS Analytics Services
Testers who want to learn the skills needed to test AWS-based data engineering applications

Data Engineering using AWS Data Analytics course specifications

Publisher: Udemy
Teacher: Durga Viswanatha Raju Gadiraju
Training level: beginner to advanced
Training duration: 26 hours and 15 minutes
Number of courses: 434

Course headings

Prerequisites of Data Engineering using AWS Data Analytics course

A Computer with at least 8 GB RAM
Programming Experience using Python is highly desired as some of the topics are demonstrated using Python
SQL Experience is highly desired as some of the topics are demonstrated using SQL
Nice to have Data Engineering Experience using Pandas or Pyspark
This course is ideal for experienced data engineers to add AWS Analytics Services as key skills to their profile