Job Description
We are seeking a Senior Databricks Engineer to join our Data Engineering team and support our enterprise data modernization initiatives. This role will work closely with Application Leads, Data Engineers, Architects, and business stakeholders to design, implement, optimize, and govern solutions on the Databricks Lakehouse Platform.
The ideal candidate will be a hands-on Databricks SME who can guide teams on implementation best practices, governance, security, performance optimization, cost management, and CI/CD enablement while supporting the migration of Oracle-based data warehouse workloads to Databricks.
Key Responsibilities
-
Design and implement scalable data solutions using the Databricks Lakehouse Platform.
-
Partner with Application Leads and Engineering teams to define implementation approaches and best practices.
-
Optimize Databricks environments for performance, scalability, reliability, and cost efficiency.
-
Implement and support Unity Catalog, data governance, lineage, and security controls.
-
Design and manage data access controls.
-
Implement Delta Sharing for secure internal and external data sharing.
-
Support Oracle-to-Databricks modernization and migration initiatives.
-
Establish and enhance CI/CD processes, deployment automation, and DevOps best practices.
-
Design, build, and manage semantic data models and business layers to support Databricks SQL, AI/BI Dashboards, Genie, Power BI, and enterprise analytics.
-
Develop standards, reusable patterns, and implementation guidance for application teams.
Required Skills Experience
-
5+ years of Data Engineering experience.
-
3+ years of hands-on Databricks implementation experience.
-
Strong knowledge of Databricks Lakehouse Platform, Delta Lake, Unity Catalog, and Delta Sharing.
-
Experience implementing RBAC, security, governance, and data lineage solutions.
-
Proven experience with Databricks performance tuning and cost optimization.
-
Experience implementing CI/CD pipelines using Git, Azure DevOps, GitHub, Jenkins, or similar tools.
-
Experience supporting cloud-based data platforms (Azure, AWS, or Google Cloud Platform).
-
Strong understanding of data warehousing concepts and modern data architectures.
-
Excellent communication and stakeholder collaboration skills.
Soft Skills
-
Strong problem-solving and analytical skills.
-
Ability to influence and guide technical teams through expertise and collaboration.
-
Passion for mentoring teams and promoting engineering best practices.
-
Self-motivated with a continuous improvement mindset.
-
Strong teamwork and communication skills.
Focus Areas: Databricks Platform, Governance, Delta Sharing, CI/CD, Cost Optimization, Analytics Enablement, and Oracle Modernization.
Preferred Skills
-
Experience supporting Oracle to Databricks migration and modernization programs.
-
Familiarity with Informatica PowerCenter and/or Informatica IDMC environments.
-
Experience with Databricks Genie, AI/BI Dashboards, and Databricks SQL.
-
Experience integrating Databricks with Power BI.
-
Knowledge of Databricks Workflows, Auto Loader, Delta Live Tables (DLT), Lakeflow, and MLflow.
-
Experience with Infrastructure as Code tools such as Terraform.
-
Databricks certifications are a plus.
