Senior Data Engineer - Airflow Developer

Capgemini Seattle, WA $140,000 - $150,000
Full Time Senior Level

Posted 2 weeks ago

Interested in this position?

Upload your resume and we'll match you with this and other relevant opportunities.

Upload Your Resume

About This Role

The Senior Data Engineer will design, develop, and deploy Apache Airflow DAGs to orchestrate Databricks jobs and build and maintain ETL workflows. This role involves optimizing Airflow DAGs for performance and collaborating with other engineering teams.

Responsibilities

  • Design, develop, and deploy Apache Airflow DAGs to orchestrate Databricks jobs
  • Build and maintain ETL workflows to extract, transform, and load data
  • Optimize Airflow DAGs for performance, scalability, and reliability
  • Monitor Airflow and Databricks job runs, and troubleshoot execution failures
  • Refactor and maintain reusable workflow components in python
  • Collaborate with platform, DevOps, and other data engineering teams

Requirements

  • Hands-on experience with Apache Airflow 2.0, Databricks, PySpark, and SparkSQL
  • Strong Python programming skills
  • Experience designing and developing ETL processes and data pipelines in Databricks
  • Solid understanding of scheduling, dependencies, and error handling in Airflow

Skills

Python * DevOps * ETL * Databricks * PySpark * Data Pipelines * SparkSQL * Apache Airflow 2.0 *

* Required skills

Benefits

Life Insurance
Employee assistance programs
Medical coverage
Disability Insurance
Dental coverage
Personal days
401(k)
Paid Time Off
Sick leave
Vision coverage
Company Paid Holidays

About Capgemini

Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world.

Professional Services
View all jobs at Capgemini →

Related Searches