Software Engineer III

About This Role

The Software Engineer III will design, implement, and optimize data ingress and egress pathways, ensuring efficient and scalable processing for the organization's analytics workloads. This role requires expertise in dataflow design, data transport mechanisms, and Apache Spark based processing.

Responsibilities

• Design data ingress and egress pathways
• Implement data ingress and egress pathways
• Optimize data ingress and egress pathways
• Ensure efficient processing of analytics workloads
• Ensure scalable and reliable processing of analytics workloads

Requirements

• Extensive expertise in dataflow design
• Extensive expertise in data transport mechanisms
• Extensive expertise in Apache Spark based distributed processing
• Experience using the Linux CLI and Linux tools
• Experience developing Bash scripts to automate manual processes
• Recent software development experience using Python
• Recent software development experience using Java
• Experience with Distributed Big Data processing engines including Apache Spark
• Experience using Jupyter Notebook
• Experience with data wrangling and preprocessing using tools such as pandas, NumPy
• Experience working with structured, semi-structured, and unstructured data such as Parquet, JSON, CSV, XML
• Familiarity with data quality concepts, data validation, and anomaly detection
• Experience with Git Source Control System

Qualifications

• Master’s degree in computer science or related discipline from an accredited college or university OR Bachelor’s degree in computer science or related discipline from an accredited college or university
• 5+ years of experience as a Software Engineer in programs and contracts of similar scope, type, and complexity.

Nice to Have

• Experience using Apache Airflow (DAG design, scheduling, operators, sensors) to orchestrate, schedule, and monitor complex workflows
• Experience with SQL technologies such as MySQL, MariaDB, and PostgreSQL for querying, joining, and aggregating large datasets
• Familiar with HPC Job Scheduling tools including Slurm
• Experience using the Atlassian Tool Suite (JIRA, Confluence)

Skills

* Required skills