Data Architect (Databricks Focused)

Job Title: Data Architect (Databricks Focused)

Location: Remote

Job Type: Fulltime

Job Description:-

We are seeking a seasoned Databricks Architect (15+ years total experience) to lead the architecture, design, and development of scalable ETL/ELT pipelines across distributed data environments. This role owns Databricks workspace administration, cluster policies, performance optimization, data governance with Unity Catalog, and DevOps automation-partnering closely with platform, infrastructure, and application teams to deliver secure, cost?effective, and high?performing data solutions.

Qualifications

Required:

• 15+ years overall industry experience
• 7+ years in data engineering with strong background in cloud?native data architecture.
• Deep hands?on experience with Databricks architecture, workspace administration, and cluster/pool management.
• Experience defining and managing cluster policies, pools, autoscaling strategies, and job compute routing.
• Strong knowledge of Spark performance tuning and job optimization (shuffle, skew, AQE, partitioning, caching, broadcast).
• Proven expertise in Databricks SQL, PySpark, Delta Lake, and large?scale batch/streaming data pipelines.
• Skilled in building reusable Python libraries with Pandas, Openpyxl, XlsxWriter, and PySpark.
• Practical experience with Unity Catalog for security, governance, lineage, and data discovery.
• Strong collaboration experience with front?end/back?end development teams and backend integration via REST APIs.
• Strong SQL expertise; hands?on experience with PostgreSQL, SQL Server, or similar RDBMS.
• DevOps expertise with Azure DevOps, Git, and pipeline automation (CI/CD for jobs, notebooks, libraries, infra).
• Excellent communication skills; ability to lead technical discussions with cross?functional teams and stakeholders.

• Cloud Platforms: Azure
• Big Data & Analytics: Databricks, PySpark, Delta Lake, Databricks SQL
• Programming & Frameworks: Python, Pandas, PySpark, Flask
• Visualization & BI: Power BI
• App Integration: React, .NET, REST APIs
• DevOps & CI/CD: Azure DevOps, Git
• Databases: Databricks SQL, Azure SQL DB, SQL Server or similar