Data Engineer

We are seeking a highly skilled Data Engineer to join our consulting team for a remote engagement with a client headquartered in Boston, MA. The ideal candidate will have extensive experience working with large-scale data pipelines, real-time streaming technologies, and cloud-based big data platforms. This role requires strong expertise in Apache Spark, Apache Kafka, Python, Machine learning and data ingestion frameworks, along with hands-on experience in Azure, SQL, and Snowflake.

Title: Data Engineer

Location: Remote (Client based in Boston, MA)

Duration: 6 months (possibility of extension)

Experience: 12 +Years

 About the Role

Key Responsibilities

• Design, build, and optimize big data ingestion pipelines to handle large volumes of structured and unstructured data.
• Develop and maintain real-time streaming solutions using Kafka and Spark Streaming.
• Collaborate with data architects and business stakeholders to define data integration strategies.
• Implement scalable solutions on Azure cloud services and integrate with Snowflake for analytics and reporting.
• Ensure data quality, reliability, and performance across distributed systems.
• Troubleshoot and optimize existing pipelines to improve efficiency and reduce latency.
• Document technical designs, workflows, and best practices for long-term maintainability.

Required Skills & Qualifications

• 12+ years of professional experience as abig Data Engineer or similar role.
• Strong expertise in Apache Spark (batch and streaming) and Apache Kafka.
• Proven experience with large-scale data ingestion and processing.
• Hands-on experience with Azure Data Services (e.g., Data Lake, Data Factory, Synapse).
• Proficiency in SQL and experience with Snowflake for data warehousing.
• Solid understanding of distributed systems and scalable architectures.
• Strong problem-solving skills and ability to work independently in a remote environment.

Preferred Qualifications

• Experience with performance tuning of big data pipelines.
• Familiarity with CI/CD pipelines and DevOps practices for data engineering.
• Exposure to data governance and security best practices.

We are an Equal Opportunity Employer