Databricks Developer

Location – New Brunswick , NJ

Senior Data Engineer to lead the design, development, and optimization of data solutions on Azure Databricks

• Lead the design and development of end-to-end data pipelines on Azure, leveraging Databricks, Azure Data Lake, and ADLS Gen 2.
• Implement scalable ETL/ELT pipelines for structured, semi-structured, APIs, and unstructured data.
• Collaborate with Data Scientists, Business Analysts, and other stakeholders to translate business requirements into robust data solutions.
• Optimize data workflows for performance, reliability, and cost-efficiency in cloud environments.
• Establish and enforce data quality, governance, and validation standards.
• Provide guidance on data modeling, schema design, and metadata management to support analytics and AI workloads.
• Monitor and troubleshoot production data pipelines and recommend performance improvements.
• Evaluate emerging Azure and Databricks technologies and propose enhancements to our data architecture.