Databricks Test Engineer - Project Delivery Specialist

About This Role

Act as the primary QA engineer for the Finance data domain, validating Databricks Medallion pipelines and ensuring data products meet functional and data-quality expectations. Design and maintain automated test frameworks using Python to expand coverage across PySpark and Databricks workflows.

Responsibilities

• Act as the primary QA engineer for the Finance data domain
• Validate Databricks Medallion (Bronze/Silver/Gold) pipelines
• Ensure delivered data products meet functional and data-quality expectations
• Design, build, and maintain automated test frameworks (unit/integration/data-quality/regression) using Python
• Expand test coverage across PySpark and Databricks workflows
• Execute performance benchmarking and scalability testing for PySpark jobs (runtime, cost, and stability)
• Partner with data engineers to diagnose bottlenecks and verify improvements
• Communicate regularly with Engagement Managers (Directors), project team members, and client representatives
• Escalate matters requiring additional attention from engagement management
• Independently and collaboratively lead client engagement workstreams focused on improvement, optimization, and transformation of processes

Requirements

• 5+ years of QA/testing experience for Databricks data platforms
• Strong working knowledge of Medallion architecture and PySpark-based pipelines
• Strong Python development skills
• Experience building automated tests and test harnesses for data pipelines (including assertions, fixtures/test data management, and CI integration)
• Deep understanding of data products and data-quality dimensions (completeness, accuracy, timeliness, reconciliation)
• Ability to translate requirements into executable test cases
• Experience performing performance testing/benchmarking for distributed data processing (profiling, metrics/SLAs, and result reporting)

Qualifications

• Bachelor's degree, preferably in Computer Science, Information Technology, Computer Engineering, or related IT discipline; or equivalent experience
• 5+ years of QA/testing experience for Databricks data platforms

Skills

* Required skills