Hi,
We are seeking a highly skilled Big Data Engineer to join our consulting team for a remote engagement with a client headquartered in Boston, MA. The ideal candidate will have extensive experience working with large-scale data pipelines, real-time streaming technologies, and cloud-based big data platforms. This role requires strong expertise in Apache Spark, Apache Kafka, and data ingestion frameworks, along with hands-on experience in Azure, SQL, and Snowflake.
Title: Big Data Engineer (Mid-Level Consultant)
Location: Remote (Client based in Boston, MA)
Duration: 6 months (possibility of extension)
Experience Required: 5–8 years
About the Role
Key Responsibilities
- Design, build, and optimize big data ingestion pipelines to handle large volumes of structured and unstructured data.
- Develop and maintain real-time streaming solutions using Kafka and Spark Streaming.
- Collaborate with data architects and business stakeholders to define data integration strategies.
- Implement scalable solutions on Azure cloud services and integrate with Snowflake for analytics and reporting.
- Ensure data quality, reliability, and performance across distributed systems.
- Troubleshoot and optimize existing pipelines to improve efficiency and reduce latency.
- Document technical designs, workflows, and best practices for long-term maintainability.
Required Skills & Qualifications
- 5–8 years of professional experience as a Big Data Engineer or similar role.
- Strong expertise in Apache Spark (batch and streaming) and Apache Kafka.
- Proven experience with large-scale data ingestion and processing.
- Hands-on experience with Azure Data Services (e.g., Data Lake, Data Factory, Synapse).
- Proficiency in SQL and experience with Snowflake for data warehousing.
- Solid understanding of distributed systems and scalable architectures.
- Strong problem-solving skills and ability to work independently in a remote environment.
Preferred Qualifications
- Experience with performance tuning of big data pipelines.
- Familiarity with CI/CD pipelines and DevOps practices for data engineering.
- Exposure to data governance and security best practices.
We are an Equal Opportunity Employer
Never miss an opportunity! Create an alert based on the job you applied for.
