Senior Data Engineer (Machine Learning)

Senior Data Engineer (ML/ ETL)

Full Time

Remote

Responsibilities:

• bold">Collaborate with machine learning practitioners to understand algorithm feature and data requirements

• bold">Work with Engineering teams to collect required data from internal and external systems

• bold">Develop and maintain production-grade batch and streaming pipelines

• bold">Setup orchestration and automation of ETL jobs using tools such as Airflow and Jenkins

• bold">Improve and extend existing data infrastructure services

• bold">Conduct data exploration, analysis and provide data strategy consultancy

• bold">Drive and maintain a culture of quality, innovation and experimentation

• bold">Work in an Agile environment that focuses on collaboration and teamwork

• bold">Mentor colleagues on best practices and technical concepts of building large scale solutions

Qualifications:

• bold">10+ years of data engineering experience

• bold">Experience deploying and running services in AWS and in engineering big-data solutions using technologies like Databricks, EMR, S3, and Spark

• bold">Experience building streaming pipelines using Kafka, Spark, Flink, or Samza

• bold">Experience loading and querying cloud-hosted databases such as Redshift and Snowflake

• bold">Experience designing and developing backend microservices for large scale distributed systems using gRPC or REST

• bold">Experience with graph-based data workflows such as Apache Airflow, Meson

• bold">Excellent communication and people engagement skills