Databricks Data Engineer Architect

Job Title: Databricks Data Engineer Architect

Location: Remote Role

Contract Duration: -6 -12 + months

Job Summary

We are seeking an experienced Databricks Data Engineer Architect to lead the architecture, design, and implementation of enterprise data engineering solutions on the Databricks platform. The ideal candidate will have extensive hands-on experience with Databricks, a proven track record of architecting scalable data solutions, and demonstrated experience migrating and integrating data from Salesforce and NetSuite into Databricks.

The candidate will be responsible for defining the data architecture, establishing data integration patterns, and ensuring the development of reliable, scalable, secure, and high-performing data pipelines. This role will work closely with business stakeholders, technical teams, and offshore data engineers to deliver the project's data engineering objectives.

Key Responsibilities

• Architect, design, and implement end-to-end data engineering solutions using Databricks.
• Lead the design and development of scalable data ingestion and transformation pipelines.
• Design and implement data migration solutions to ingest data from Salesforce and NetSuite into Databricks.
• Define data architecture, data models, integration patterns, and best practices for the Databricks platform.
• Develop and optimize ETL/ELT pipelines using Python, SQL, PySpark, and other relevant technologies.
• Establish data quality, validation, reconciliation, monitoring, and error-handling frameworks.
• Design secure and efficient data processing solutions, including appropriate access controls and governance practices.
• Optimize data pipelines for performance, scalability, reliability, and cost efficiency.
• Collaborate with business stakeholders to understand data requirements and translate them into technical architecture and implementation plans.
• Provide technical leadership and guidance to offshore data engineering resources.
• Review technical designs, code, and implementation approaches to ensure adherence to architectural standards.
• Support testing, deployment, troubleshooting, and production readiness activities.
• Document architecture decisions, data flows, technical specifications, and operational procedures.

Required Qualifications and Experience

• Extensive hands-on experience with Databricks and its data engineering capabilities.
• Proven experience architecting and delivering end-to-end data engineering solutions.
• Demonstrated experience migrating and integrating data from both Salesforce and NetSuite into Databricks.
• Strong proficiency in Python, SQL, and PySpark.
• Extensive experience building and managing ETL/ELT pipelines and complex data integration workflows.
• Strong understanding of data lakehouse architecture, data modeling, and modern cloud data platforms.
• Experience with Delta Lake and related Databricks capabilities.
• Knowledge of data governance, security, data quality, and pipeline monitoring.
• Experience with API-based integrations, incremental data loads, and change data capture where applicable.
• Strong analytical, problem-solving, communication, and technical leadership skills.
• Ability to work collaboratively with onshore stakeholders and offshore delivery teams.

Preferred Qualifications

• Databricks certifications in data engineering or solution architecture.
• Experience with cloud platforms such as Microsoft Azure, AWS, or Google Cloud.
• Experience with orchestration tools, CI/CD pipelines, and automated deployment practices.
• Familiarity with Salesforce APIs, NetSuite APIs, and enterprise application integration patterns.
• Experience delivering enterprise-scale data migration and modernization projects.

Key Deliverables

• Databricks solution architecture and technical design documentation.
• Data ingestion and migration architecture for Salesforce and NetSuite.
• Scalable and production-ready data pipelines.
• Data quality, validation, and reconciliation mechanisms.
• Technical standards and implementation guidelines for the data engineering team.
• Technical leadership, design reviews, and support for project delivery.