Role: Data Engineer
Location: Boston, MA (Onsite) - Only locals
Duties and Responsibilities:
As a senior member, collaborate with a small team of data engineers, driving excellence in data platform evolution. Promote a culture of innovation, accour improvement.
Design, develop, and maintain scalable data pipelines using Azure Data Factory, Databricks, and SQL Server (T-SQL).
Implement configuration-driven ingestion processes for batch and near real-time data.
Ingest, transform, and integrate data from diverse sources (APIS, JSON, CSV, XLS, etc.) using industry best practices.
Analyze, design, and document data models (ERDs), create data dictionaries and data flow diagrams as part of pipeline development.
Design and implement Lakehouse solutions leveraging Medallion architecture.
Develop and maintain SSRS and Power BI datasets and reports.
Oversee the health, scalability, and security of the Azure-based data platform.
Support and enhance data governance, cataloging, and MDM initiatives.
Ensure data security, privacy, and compliance best practices are followed.
Manage access controls, data lineage, and metadata to support data discoverability and compliance.
Continuously evaluate and improve data engineering processes for efficiency, automation, and reliability.
Develop and maintain CI/CD pipelines using GitHub Actions or CircleCl.
Implement and refine monitoring, alerting, and observability for data pipelines and platform health.
Collaborate with cross-functional teams to identify and eliminate process bottlenecks.
Optimize data pipelines and platform components for performance, scalability, and cost-effectiveness.
Evaluate and implement data quality and observability tools to ensure data reliability and accessibility.
Work closely with the data product manager, program manager, engineering manager, and scrum master to define and deliver data products based on priorit
Participate in sprint planning, backlog refinement, and Agile ceremonies.
Collaborate with offshore and cross-functional teams to deliver high-quality solutions.
Refine user stories and requirements to ensure clarity and actionability.
Participate in new teammate onboarding, provide guidance and mentorship to junior engineers and facilitate regular feedback sessions, career development p upskilling.
Never miss an opportunity! Create an alert based on the job you applied for.
