Data Engineer

About This Role

Lead the design and development of scalable and high-performance data solutions using AWS services and Databricks. Implement and manage Elastic Search for efficient data retrieval and analysis.

Responsibilities

• Lead the design and development of scalable and high-performance solutions using AWS services
• Experience with Databricks, Elastic search, Kibanna, S3
• Experience with Extract, Transform, and Load (ETL) processes and data pipelines
• Write clean, maintainable, and efficient code in Python/Scala
• Implement and manage Elastic Search engine for efficient data retrieval and analysis
• Experience with data warehousing, data visualization tools, data integrity
• Execute full software development life cycle (SDLC) including gathering requirements and writing functional/technical specifications
• Design both logical and physical database models
• Develop database objects including stored procedures and functions
• Develop software design documents and work with stakeholders for review and approval

Requirements

• 12 years developing complex database systems
• 8 years with Databricks
• 8 years using Elastic search, Kibanna
• 8 years using Python/Scala
• 8 years with Oracle
• 5 years experience with Extract, Transform, and Load (ETL) processes and developing data pipelines
• 5 years experience with AWS
• Over 5 years experience with data warehousing, data visualization tools, data integrity
• Over 5 years using CMM/CMMI Level 3 methods and practices
• Over 5 years implemented agile development processes including test-driven development

Qualifications

• 12 years developing complex database systems; 8 years with Databricks, Elastic search, Kibanna, Python/Scala, Oracle; 5 years with ETL processes, data pipelines, AWS, data warehousing, data visualization, data integrity; 5 years with CMM/CMMI Level 3 methods and agile development.

Nice to Have

• Over 3 years experience or knowledge on creating CI/CD pipelines using Azure DevOps

Skills

* Required skills