Lead Data Engineer
Posted 2 months ago Expired
This job has expired
Looking for a job like Lead Data Engineer in or near O'Fallon, MO? Upload your resume and we'll notify you when similar positions become available.
Upload Your ResumeAbout This Role
As a Lead AI Data Engineer at Mastercard Security Services, you will design scalable data pipelines, build feature engineering workflows, and create shared tools that empower data scientists. You will also lead efforts to integrate data pipelines with model APIs and support production-grade AI systems, contributing to AI products that help fight fraud.
Responsibilities
- Lead the development of scalable, high-performance data pipelines for structured and unstructured data used in AI/ML models
- Design and manage feature stores to ensure consistent feature computation across training and inference environments
- Collaborate with ML Engineering and ML Ops teams to build and maintain data quality and drift monitoring systems
- Develop AI microservices that integrate data pipelines with model APIs for real-time and batch inference
- Recommend and implement improvements to data quality, efficiency, and pipeline resilience
- Partner with cross-functional teams to deliver robust analytical solutions using Mastercards global data assets
- Organize and maintain tools for data transfer, mining, cleaning, processing, and validation
- Contribute to the development of scalable AI infrastructure and reusable components for enterprise-wide use
Requirements
- Proven experience in Python and SQL for scripting, automation, and data integration
- Hands-on experience with big data technologies such as Spark, Kafka, Hive, Impala, Airflow, and NiFi
- Familiarity with relational databases (PostgreSQL, Oracle) and cloud platforms (preferably AWS)
- Experience building and managing feature engineering workflows and feature stores
- Understanding of ML workflows, including model training, inference, and monitoring
- Experience with CI/CD tools (GitHub Actions, Jenkins) and infrastructure automation (Terraform)
- Strong knowledge of Linux/Unix systems and shell scripting
- Excellent communication and collaboration skills, with the ability to work across technical and non-technical teams
- Subject matter expertise in payments, fraud, or cybersecurity or a demonstrated ability to quickly learn new domains
- Creative, curious, and innovative mindset with a passion for solving complex problems
Nice to Have
- Experience with AWS cloud services for data engineering and ML workflows
- Hands-on experience with Databricks, including Medallion Architecture and Delta Lake
Skills
* Required skills
Benefits
About Mastercard
Mastercard powers economies and empowers people in 200+ countries and territories worldwide, helping build a sustainable economy with secure, simple, smart and accessible digital payments.