Data Engineer

Key Responsibilities

Data Platform & Pipeline Development

• Design, develop, and maintain cloud-native data pipelines using Databricks, Microsoft Azure Data Factory, and Microsoft Fabric to support data integration and analytics solutions.
• Implement incremental and real-time data ingestion strategies using a medallion architecture for data lake storage.
• Write, optimize, and maintain complex SQL queries to transform, integrate, and analyze data across enterprise systems.
• Develop solutions with a focus on scalability, maintainability, testability, and long-term operability within a continuous delivery mindset.

Data Operations & Reliability

• Support and troubleshoot legacy data platforms built on SSIS and SQL Server, ensuring high availability and performance of critical data processes.
• Identify, troubleshoot, and resolve data integration and data quality issues to ensure reliable production data delivery.
• Contribute to observability, automated validation, and CI/CD pipelines to support fast feedback and safe releases.

Agile Collaboration & Engineering Practices

• Collaborate daily with product owners, QA, and engineers through Scrum ceremonies including standups, backlog refinement, sprint planning, sprint reviews, and retrospectives.
• Participate in proof-of-concept efforts, technical spikes, and design discussions, providing thoughtful technical analysis and pragmatic recommendations.
• Apply XP engineering practices such as pairing, incremental delivery, continuous refactoring, and shared code ownership to maintain high-quality, evolvable systems.
• Clearly communicate technical concepts, risks, tradeoffs, and progress to both technical and non-technical stakeholders.

Requirements

Required Qualifications

• 5+ years of experience designing and building data solutions.
• Strong proficiency in SQL and Python for data analytics and transformation.
• Hands-on experience with ETL pipeline development and automation.
• Solid understanding of data lake architecture and design principles.
• Experience working on Agile teams (Scrum, XP, or similar), with regular participation in standups, sprint planning, refinement, reviews, and retrospectives.
• Comfort operating in highly collaborative environments with frequent verbal and written communication across engineering, product, and QA teams.
• Ability to break work into small, iterative deliverables and adapt quickly based on feedback and changing priorities.
• Strong ownership mindset, including accountability for the quality, reliability, and maintainability of delivered solutions.

Preferred Qualifications

• Experience with Azure cloud services and cloud-based ETL tools.
• Familiarity with data visualization tools such as Power BI or Tableau.
• Understanding of event-driven architectures, including queues, batch processing, and pub/sub models.
• Exposure to NoSQL databases such as MongoDB or Cassandra.
• Experience working on product engineering teams delivering customer-facing or operationally critical systems.
• Familiarity with modern engineering practices inspired by XP, including automated testing, pairing, refactoring, and continuous integration.
• Experience operating systems in production environments with uptime, reliability, and observability expectations.

Bonus Points For

• Experience in Data Science or Machine Learning, particularly in model deployment or feature engineering.
• Experience contributing to engineering standards, documentation, or continuous improvement initiatives within an Agile team.