Senior Databricks Architect (100% Remote)

Company Overview
Our client is a leading provider of large-scale infrastructure and engineering services that support essential industries across North America. They are committed to investing in modern data platforms and fostering a culture of continuous learning, collaboration, and technical excellence.

Role Summary
The Intersect Group is seeking a highly skilled Senior Databricks Architect to lead the design, development, and optimization of a modern data platform leveraging the Databricks Lakehouse architecture. This role combines deep technical expertise with leadership responsibilities, driving data engineering, analytics, and AI/ML initiatives across multiple concurrent projects.
You will serve as a hands-on architect and trusted advisor, partnering with internal teams and external resources to implement scalable, high-performance data solutions. This role requires someone who can own delivery end-to-end, guide junior resources, and ensure consistent execution across a complex, multi-project environment.

Key Responsibilities

• Architect, design, and implement scalable data solutions using Databricks Lakehouse architecture.
• Lead hands-on development, including building and optimizing data pipelines, notebooks, and workflows in Python and SQL.
• Serve as the primary technical escalation point for Databricks-related challenges across development and production environments.
• Collaborate with cross-functional teams—including data modelers, engineers, and external consultants—to deliver projects on time.
• Oversee multiple initiatives (6–10 concurrently), ensuring alignment across varying project stages and stakeholders.
• Implement and optimize Databricks features such as Auto Loader, Delta Live Tables (DLT), and streaming data pipelines.
• Establish and enforce best practices for performance, scalability, security, and data governance (e.g., row-level security, column masking, encryption).
• Mentor junior team members and contribute to overall team development through knowledge sharing and technical guidance.
• Provide regular updates on progress, risks, and blockers to leadership.

Key Requirements

• 5+ years of experience in data architecture, data engineering, or AI/ML solutions with ownership of technical delivery.
• Extensive hands-on expertise with Databricks, including Lakehouse architecture and Medallion design patterns.
• Strong experience implementing Databricks in Azure environments.
• Proficiency in Python and SQL for building data pipelines and analytics solutions.
• Experience with large-scale distributed systems and big data technologies (e.g., Apache Spark, Hadoop, Cassandra).
• Expertise in building and optimizing data ingestion frameworks, including Auto Loader, Delta Live Tables, and streaming solutions.
• Strong understanding of data security and governance practices within Databricks environments.
• Proven ability to lead projects end-to-end and manage multiple priorities simultaneously.
• Excellent communication skills and a collaborative mindset, with the ability to work across internal teams and external partners.

Preferred Qualifications

• Databricks certifications in data engineering or related disciplines.
• Experience with Azure networking or security configurations.
• Exposure to AI/ML workloads and advanced analytics use cases.

Call to Action
If you are a hands-on Databricks expert who thrives in complex environments and enjoys leading high-impact data initiatives, we encourage you to apply. Please submit your resume and contact information to The Intersect Group for immediate consideration.