Software Engineer - Distributed Systems

Overview

Remote
$60 - $80
Contract - W2
Contract - 12 Month(s)

Skills

Python
Rust
PHP
Hack
Cloud
DAG
RAG
Graph
Machine Learning
Tensorflow
Data Processing
PyTorch
Machine Learning (ML)

Job Details

Summary:

We are seeking a skilled and experienced engineer to join our team as a Distributed Systems Specialist. As a generalist with expertise in distributed systems, you will play a key role in designing, developing, and optimizing our data generation pipelines for maximum efficiency. Your experience with DAGs/Graphs, large fan-out loads, and distributed workers/jobs will be instrumental in helping us scale our systems to meet the growing demands of our users.

Responsibilities:

Design, develop, and optimize data processing pipelines using DAGs/Graphs and distributed workers/jobs.
Collaborate with cross-functional teams to identify and prioritize system improvements.
Develop and implement solutions to handle large fan-out loads and ensure system scalability.
Work closely with data scientists and engineers to integrate and scale solutions.
Identify opportunities for optimization and implement changes to improve system performance and reduce resource utilization.

Requirements:

5+ years of experience in software development, with a focus on distributed systems.
Experience with large fan-out loads and distributed workers/jobs.

Proficiency in at least one of the following programming languages:
Python (primary language).
Rust.
C++.
Hack or PHP
Experience with cloud-based infrastructure, such as AWS or Google Cloud Platform.
Strong problem-solving skills and ability to work independently.
Excellent communication and collaboration skills.

Must-Have Skills:

Experience running compute clusters or building data infrastructure.
Proficiency in at least one of the following programming languages: Python (ideally), Rust, C++, Hack (PHP).
Experience with cloud-based infrastructure such as AWS .

Nice-to-have Skills:

Experience with companies using large data scales.
Experience with Digital Content Creation (DCC) tools such as blender
Experience with machine learning frameworks, such as TensorFlow or PyTorch

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.