Senior Data Engineer

About This Role

Design, build, and maintain scalable data pipelines and ingestion frameworks for a leading P&C insurance carrier. This role involves hands-on coding, mentoring, and collaborating with cross-functional teams to shape the company's next-generation data environment.

Responsibilities

• Design, build, and optimize scalable data pipelines for batch and real-time processing
• Implement data ingestion frameworks including CDC from core systems, APIs, and third-party platforms
• Develop and optimize Apache Spark jobs on Databricks, leveraging Delta Lake, DLT pipelines, and lakehouse architectures
• Ensure data quality, lineage, and governance using Unity Catalog, CI/CD, and role-based access/security controls
• Partner with analytics engineers (dbt) to deliver clean, structured upstream data
• Mentor peers, contribute to architecture decisions, and foster a culture of craftsmanship
• Leverage AWS services (S3, Glue, Lambda, etc.) and DevOps tools (GitHub, CI/CD) for scalable, production-grade deployments

Requirements

• 3-5+ years of professional experience in data engineering
• Strong proficiency in Python, SQL, and Spark
• Hands-on expertise with Databricks (Unity Catalog, Delta Lake, DLT pipelines) and/or Azure Data Services
• Strong knowledge of AWS data services
• Familiarity with modern data architectures (medallion, lakehouse, streaming)
• Experience with GitHub, CI/CD pipelines, and testing frameworks

Qualifications

• 3-5+ years of professional experience in data engineering, ideally within insurance or financial services

Nice to Have

• Experience within insurance or financial services

Skills

* Required skills

Benefits