This video covers a hands-on example in setting up Batch Processing example using Amazon Data Pipeline which leverages S3 and DynamoDB along with Amazon EMR.
The following is the agenda for the video:
⏱ Chapter Timestamps
0:00 - Introduction
00:28 - Architecture Flow
2:31 - Pre-requisite: Amazon S3 Buckets Creation
3:44 - Pre-requisite: Amazon DynamoDB Table Creation
5:15 - Pre-requisite: Amazon S3 file format Creation
8:20 - Pre-requisite: Amazon Data Pipeline Creation
12:40 - Data Format Configuration in Data Pipeline
14:14 - Activate Data Pipeline
17:52 - Data Loaded into DynamoDB
19:03 - Serverless Data Pipeline Trigger using EventBridge and Lambda
20:06 - Summary
📌 Links
🔗 Amazon Data Pipeline: https://aws.amazon.com/datapipeline/
📌 Related Videos
🔗AWS Primer: https://www.youtube.com/playlist?list=PLTyWtrsGknYdyi0JR7c1wsuv-Plrwrzqj
🔗 System Design Primer: https://www.youtube.com/playlist?list=PLTyWtrsGknYeX_wV9ysDuuAxvxfSBfVSI
🔗 Containers Primer: https://www.youtube.com/watch?v=yo2g_CZZWL4&list=PLTyWtrsGknYf_Oee7YOvom5Ev0F3-E2zB
🔗 Kubernetes Primer: https://www.youtube.com/playlist?list=PLTyWtrsGknYfanKF33E12LdJvl5q5PZGp
💥 Join TechPrimers Slack Community: https://bit.ly/JoinTechPrimers
💥 Telegram: https://t.me/TechPrimers
💥 TechPrimer HindSight (Blog): https://medium.com/TechPrimers
💥 Website: http://techprimers.com
💥 Slack Community: https://techprimers.slack.com
💥 Twitter: https://twitter.com/TechPrimers
💥 Facebook: http://fb.me/TechPrimers
💥 GitHub: https://github.com/TechPrimers or https://techprimers.github.io/
🎬Video Editing: FCP
🔥 Disclaimer/Policy:
The content/views/opinions posted here are solely mine and the code samples created by me are open sourced.
You are free to use the code samples in Github after forking and you can modify it for your own use.
All the videos posted here are copyrighted. You cannot re-distribute videos on this channel in other channels or platforms.
#AmazonDataPipeline #AmazonEMR #TechPrimers
#amazon data pipeline #batch processing example #amazon #dynamodb #amazon emr
