Databricks admin/platform engineer with FinOps experience
Remote
Strong exp. in FinOps end to end tagging, dashboards, finops data model.
Need to understand connectivity, compute, resources at granular level.
Data engineering background is a plus.
Key Responsibilities
Design, implementation, and optimization of end-to-end solutions on Databricks, including Unity Catalog, Delta Lake, Databricks SQL, Workflows, Jobs, and Mosaic AI.
Build, and maintain scalable data ingestion, transformation, and orchestration pipelines using Databricks tools (Spark, Delta Live Tables, Auto Loader, etc.).
Identify and resolve performance bottlenecks in Spark jobs, clusters, and SQL queries. Implement cost optimization strategies, auto-scaling, and efficient resource management.
Establish and enforce Databricks best practices for security, data governance, compliance (e.g., GDPR, HIPAA), and Unity Catalog implementation across the organization.
Support data science and machine learning teams by implementing MLflow, Feature Store, Vector Search, and generative AI capabilities on Databricks.
Act as the escalation point for complex technical issues, providing root cause analysis and rapid resolution for production workloads.
Mentor data engineers, analysts, and developers on Databricks capabilities. Conduct training sessions, create documentation, and build internal centers of excellence.
Design seamless integrations between Databricks and other technologies (cloud platforms like AWS/Azure/Google Cloud Platform, ETL tools, BI tools, data catalogs, and orchestration systems).
Implement robust monitoring, logging, alerting, and dashboarding for Databricks workloads using Databricks features and external tools.
Stay current with the latest Databricks features and roadmap. Evaluate new capabilities and recommend adoption strategies to drive business value.
Work closely with stakeholders to translate business requirements into technical solutions and deliver high-impact data & AI projects.
