Data-lake on AWS.

The Problem

Addressing the Demand for Efficient Data Management: Building a Comprehensive Data Lake Solution from Diverse Input Sources, Streamlining Transformations, and Enabling Seamless SQL Querying.

The solution

Harnessing the Power of AWS: Crafting a Robust Data Lake Utilizing Lambda Functions, S3 Storage, AWS Glue, and Athena for Seamless Data Management.

Scope of work.

  • Developed a series of Lambda functions for ingesting input sources.
  • Constructed a suite of AWS Glue ETL Jobs for data transformation tasks.
  • Established a collection of AWS Athena queries tailored for the reporting team’s needs.