Senior Data Engineer (SAS to Databricks Migration)
Location: Remote (USA)
Employment Type: Contract Role
Work Schedule: Flexible hours with core collaboration hours aligned to U.S. time zones
About the Role
Are you an experienced Data Engineer looking to make a massive impact in a remote setting? We are looking for a Senior Data Engineer with specialized expertise in migrating legacy systems to modern cloud architectures. In this role, you will lead the modernization of legacy workloads by migrating SAS-based processes to Databricks on Azure, delivering robust, high-performance data pipelines for advanced analytics and reporting.
Employment Type: Contract Role
Work Schedule: Flexible hours with core collaboration hours aligned to U.S. time zones
Key Responsibilities
Cloud Data Engineering: Design, build, and optimize scalable data pipelines on Azure utilizing Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Synapse, and Azure SQL.
Legacy Migration: Interpret, convert, and modernize legacy SAS workloads (including Base SAS and SAS macros) into modern, optimized Databricks environments.
Databricks & Spark Optimization: Leverage advanced Databricks capabilities, including Delta Lake and Apache Spark optimization, to enhance performance.
Query & Performance Tuning: Optimize the performance of complex Spark jobs and SQL queries.
Data Governance & Quality: Implement rigorous data quality, validation, and monitoring practices across all pipelines.
Version Control & CI/CD: Utilize Git and follow CI/CD best practices for seamless deployment and code management.
Required Qualifications
Experience: Minimum of 5+ years of hands-on data engineering experience.
Azure Expertise: Strong hands-on experience with Azure data services (ADF, ADLS, Synapse, Azure SQL).
Databricks & Spark: Advanced Databricks experience, including Delta Lake architecture and Spark optimization.
Programming: Proficiency in Python and PySpark.
SQL & Modeling: Expert-level SQL skills and solid experience with data modeling principles.
SAS Knowledge: Working knowledge of SAS (Base SAS, SAS macros) with a proven ability to interpret and translate legacy SAS code into modern frameworks.
Engineering Best Practices: Experience implementing data quality frameworks, monitoring practices, Git workflows, and CI/CD pipelines.
