Senior Data Engineer - Airflow Developer
Posted 2 months ago Expired
This job has expired
Looking for a job like Senior Data Engineer - Airflow Developer in or near Seattle, WA? Upload your resume and we'll notify you when similar positions become available.
Upload Your ResumeAbout This Role
The Senior Data Engineer will design, develop, and deploy Apache Airflow DAGs to orchestrate Databricks jobs and build and maintain ETL workflows. This role involves optimizing Airflow DAGs for performance and collaborating with other engineering teams.
Responsibilities
- Design, develop, and deploy Apache Airflow DAGs to orchestrate Databricks jobs
- Build and maintain ETL workflows to extract, transform, and load data
- Optimize Airflow DAGs for performance, scalability, and reliability
- Monitor Airflow and Databricks job runs, and troubleshoot execution failures
- Refactor and maintain reusable workflow components in python
- Collaborate with platform, DevOps, and other data engineering teams
Requirements
- Hands-on experience with Apache Airflow 2.0, Databricks, PySpark, and SparkSQL
- Strong Python programming skills
- Experience designing and developing ETL processes and data pipelines in Databricks
- Solid understanding of scheduling, dependencies, and error handling in Airflow
Skills
* Required skills
Benefits
About Capgemini
Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world.