Software Engineering PMTS

About This Role

Design, develop, and maintain information retrieval infrastructure for Slack Search and other product experiences, focusing on reliability, performance, scalability, and security. Evolve systems to incorporate cutting-edge search technology and generative AI capabilities.

Responsibilities

• Design and develop scalable, resilient offline indexing pipelines that process and transform data for Slack's search infrastructure
• Work with product engineering to conceptualize and build new features for a large user base
• Optimize data processing pipelines for performance, cost-efficiency, and reliability at massive scale
• Collaborate with team members to ensure high availability and data freshness across distributed indexing systems
• Actively own indexing pipeline features and define their long-term health, while improving surrounding data infrastructure
• Monitor, troubleshoot, and optimize pipeline performance, addressing data quality issues and processing bottlenecks
• Assist skilled support and operations teams in triaging and resolving production issues in on-call rotation
• Mentor other engineers and deeply review code
• Improve engineering standards, tooling, and processes

Requirements

• 7+ years building web applications and operating large scale distributed systems
• Extensive experience with distributed data processing frameworks such as Spark, Hadoop, Map/Reduce, or EMR
• Deep experience with workflow orchestration tools like Airflow for managing complex data pipelines
• Proficient in Python, Java, Scala for data engineering
• Track record of developing high-availability, performant data pipelines that process large volumes of data across multiple clusters and regions
• Ability to contribute to technical architecture discussions and drive technical decisions
• Write understandable, testable code with an eye towards maintainability
• Strong communication skills for explaining complex technical concepts
• Possess computer science fundamentals: data structures, algorithms, programming languages, distributed systems, and information retrieval

Qualifications

• Bachelor's degree in Computer Science, Engineering or related field, or equivalent
• 7+ years building web applications and operating large-scale distributed systems professionally

Nice to Have

• Contributed to open-source projects related to Airflow, Spark, Lucene, Solr, or ElasticSearch
• Knowledge of search indexing optimization techniques and index construction
• Experience with real-time streaming data processing (e.g., Kafka, Flink, Kinesis)
• Experience with data quality frameworks and monitoring for large-scale pipelines
• Experience around text mining, parsing, and NLP preprocessing for search

Skills

* Required skills

Benefits