Data Engineer – Python, Airflow, AWS & Snowflake

About the role

We are looking for a skilled Data Engineer to design, build, and maintain robust data pipelines that support our analytics and reporting needs. The role focuses on leveraging Python, Airflow, and AWS services to deliver reliable, scalable data solutions.

Key responsibilities

• Develop reusable Python packages and implement multi‑threaded, functional programming solutions.
• Design, schedule, and monitor ETL workflows using Airflow DAGs.
• Orchestrate multiple pipelines to run sequentially or in parallel and handle error reprocessing.
• Export and import data in formats such as CSV, JSON, and XML, primarily from S3 or other cloud storage.
• Write, optimize, and tune complex SQL queries for Snowflake data warehouses.
• Create and maintain DBT tests for data validation and quality assurance.
• Implement CI/CD pipelines on GitHub and manage version control.
• Apply AWS best practices for S3, ECS, IAM, and cloud security, including encryption and role‑based access.
• Optional: Build serverless data processing tasks using AWS Lambda.

Required profile

• Advanced proficiency in Python, including OOP, multi‑threading, and functional programming.
• Strong experience with Airflow for workflow orchestration.
• Expertise in SQL, especially complex joins, CTEs, and query optimization for Snowflake.
• Hands‑on experience with AWS services (S3, ECS, IAM) and security best practices.
• Proven ability to write unit, integration, and system tests in Python.
• Experience implementing CI/CD pipelines and using GitHub for version control.

Required skills

• Python
• Airflow
• SQL
• Snowflake
• DBT
• GitHub
• CI/CD
• AWS S3
• AWS ECS
• AWS IAM
• AWS Lambda (nice to have)
• Jinja templates