Title: Big Data Engineer
Location: Washington, DC / Rockville, MD / McLean, VA (Hybrid 3 days onsite with 2 days remote)
Duration: 6 Months with possible extension
Interview process: Phone, Onsite panel
Job Summary:
We are seeking an experienced Big Data Engineer to design, develop, and optimize large-scale data processing systems. This is an engineering-focused role supporting data-intensive applications and data science workflows. The ideal candidate has strong experience with distributed systems, cloud platforms, and technologies such as Spark, Hadoop, Python, and Scala.
Responsibilities
Design, develop, and maintain scalable data processing pipelines using Spark, Hadoop, Python, Scala, and related technologies.
Build reliable solutions for data ingestion, storage, transformation, and processing.
Translate business and analytical requirements into scalable engineering solutions.
Optimize data pipelines for performance, scalability, and reliability.
Build datasets, features, and pipelines that support data science, analytics, and model development.
Develop automated testing frameworks and implement unit, integration, system, and end-to-end testing.
Implement data quality controls and continuous testing practices.
Monitor and troubleshoot production data pipelines and resolve performance or reliability issues.
Collaborate with data scientists, analysts, and engineering teams on data-driven solutions.
Evaluate emerging technologies and continuously improve data architecture and engineering practices.
Qualifications
Strong hands-on Big Data engineering experience.
Experience with Apache Spark and Hadoop ecosystems.
Strong programming skills in Python and/or Scala.
Experience designing and optimizing large-scale data pipelines and distributed processing systems.
Experience with cloud-based data platforms and services.
Strong understanding of data ingestion, transformation, storage, and processing patterns.
Experience with automated testing and data quality practices.
Experience supporting data science or machine learning workflows.
Strong problem-solving and troubleshooting skills.
Experience with graph data or graph technologies is a plus.
Ideal Background
This role is best suited for a hands-on data engineer who has built large-scale data platforms and pipelines while working closely with data science teams. Direct data science experience is not required, but experience building the data infrastructure that supports analytics and machine learning is highly valued.
Never miss an opportunity! Create an alert based on the job you applied for.