Seeking an experienced AWS Lead Data Engineer to design, build, and manage scalable data pipelines and platforms on AWS, with expertise in ETL, data modeling, and AWS services like Athena, Glue, and Step Functions.
Responsibilities
Design and implement ETL/ELT pipelines using AWS Glue, Spark, and Step Functions
Work with structured and semi-structured data using Athena, S3, and Lake Formation
Develop and deploy serverless data processing solutions using AWS Lambda
Perform SQL and PL/SQL development for data transformation and analysis
Build data lakes and data warehouses using S3, Aurora, and Athena
Implement data governance and security strategies using AWS tools
Participate in data modeling exercises
Work with stakeholders to understand data requirements and deliver insights
Monitor, debug, and optimize data pipelines
Requirements
Strong experience with AWS data services
Proficient in PySpark, Python, SQL, and PL/SQL
Solid understanding of ETL/ELT processes and data warehousing concepts
Familiarity with modern data platform fundamentals
Experience in data modeling
Experience with orchestration and workflow management tools within AWS