We are looking for a Senior Software Engineer to join our Data Engineering Infrastructure team, which builds and operates the foundational platforms that power data ingestion, transformation, and analytics across the organization. You will design and develop high-performance, reliable, and scalable systems that enable data engineers, analysts, and ML practitioners to move, process, and govern data efficiently and securely.
As a Senior Software Engineer in the Data Engineering Infrastructure team, you will design and build distributed systems and frameworks that automate the lifecycle of data - from ingestion to transformation to serving. You'll work at the intersection of software engineering, distributed data processing, and cloud infrastructure, helping to define the standards, abstractions, and tools that enable our data platform to operate at scale.\n\nYou will collaborate closely with teams across data engineering, analytics, ML, and platform engineering to deliver resilient infrastructure components such as data ingestion pipelines, metadata and schema management services, workflow orchestration, and monitoring frameworks. This is a hands-on role where you will influence architecture, write production-grade code, and drive engineering excellence across the data platform.
12+ years of experience in software engineering, with at least 5 years focused on data systems or platform infrastructure\nProven track record of leading design discussions, mentoring engineers, and driving cross-team technical initiatives\nStrong programming skills in Java, Python; Scala - nice to have\nHands-on experience designing, developing, and operating high-performance backend services\nHands-on experience with distributed data frameworks such as Spark, Flink, or Kafka\nSolid understanding of data modeling, storage formats (Parquet/Avro/ORC), and partitioning strategies\nExperience with data governance, cataloging, and schema management systems (e.g., Hive Metastore, Glue, Iceberg, Delta Lake)\nExperience working with cloud-based data platforms (AWS, Google Cloud Platform, or Azure)\nExperience building data infrastructure frameworks, SDKs, or shared libraries used by multiple data teams\nFamiliarity with CI/CD, container orchestration (Kubernetes), and infrastructure-as-code tools (Terraform, CloudFormation)\nExcellent problem-solving, debugging, and communication skills\nBachelor's degree in Computer Science, Engineering, or related field (or equivalent practical experience)
Deep expertise in optimizing and troubleshooting distributed data frameworks (e.g., Apache Spark internals, Flink stateful streaming, Kafka Connect)\nAdvanced experience with workflow orchestration tools (e.g., Airflow, Prefect, Dagster), including custom development\nProven track record in performance tuning, cost optimization, and designing comprehensive observability solutions for large-scale cloud data platforms\nFamiliarity with data security, privacy, and compliance principles within data infrastructure\nExperience integrating data platforms with MLOps and machine learning workflows\nActive participation or contributions to relevant open-source data technologies or industry communities\nMaster's or Ph.D. degree in Computer Science, Data Engineering, or a related quantitative field
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
- Dice Id: 90733111
- Position Id: d69f89038c3baccb58267228382dd3a2
- Posted 3 hours ago
Never miss an opportunity! Create an alert based on the job you applied for.
