Key Responsibilities

Data Ingestion & Processing

1. Design, build, and maintain data ingestion pipelines using AWS Glue, AWS Lambda, and S3
2. Ingest data from multiple sources including databases, files, APIs, and SFTP
3. Handle batch and near-real-time data processing use cases

Data Warehousing & Snowflake

1. Load, transform, and optimize data in Snowflake
2. Develop and maintain Snowflake objects including tables, views, stages, file formats, and tasks
3. Implement performance optimization techniques (clustering, partitioning, query tuning)

Data Transformation & Quality

1. Implement data transformations using SQL and/or ETL frameworks
2. Apply basic data quality checks and validation rules
3. Troubleshoot and resolve data pipeline failures and performance issues

Cloud & Automation

1. Develop serverless solutions using AWS Lambda for orchestration and lightweight transformations
2. Work with IAM roles and policies to ensure secure access to AWS resources
3. Monitor pipelines using CloudWatch and logs

Collaboration & Delivery

1. Collaborate with senior data engineers, architects, and analysts on solution design
2. Participate in code reviews, testing, and deployment activities
3. Support production issues and contribute to continuous improvement initiatives

Required Skills & Qualifications

1. 3-4 years of experience as a Data Engineer
2. Hands-on experience with AWS services: Glue, Lambda, S3, CloudWatch
3. Strong experience with Snowflake data warehouse
4. Proficiency in SQL (Snowflake SQL preferred)
5. Experience working with Python for ETL and Lambda development
6. Understanding of data modeling concepts (fact/dimension tables, normalization)
7. Familiarity with Git-based version control and CI/CD basics