Sr. Databricks Architect (Data Governance & AI)

Position: Sr. Databricks Architect (Data Governance & AI)
Location: Houston, TX (4 days hybrid onsite)

Job Summary:
We are looking for a Senior Databricks Architect with strong expertise in Data Governance and exposure to AI/ML tools. The ideal candidate will design and implement scalable data platforms, ensure data quality and compliance, and support advanced analytics and AI initiatives.

Key Responsibilities:

• Design and architect scalable data solutions using Databricks (Lakehouse architecture)
• Implement and manage data governance frameworks (data quality, lineage, catalog, security)
• Define and enforce data standards, policies, and best practices
• Work with stakeholders to ensure compliance with data privacy and regulatory requirements
• Build and optimize ETL/ELT pipelines using Spark and Databricks
• Integrate AI/ML tools and frameworks for advanced analytics and automation
• Collaborate with data engineers, analysts, and data scientists
• Monitor performance, cost optimization, and reliability of data platforms

Required Skills:

• Strong experience with Databricks, Apache Spark, and Delta Lake
• Hands-on experience in Data Governance tools (Unity Catalog, Collibra, Alation, etc.)
• Proficiency in Python, SQL, and data engineering concepts
• Experience with cloud platforms (Azure, AWS, or Google Cloud Platform)
• Knowledge of data security, data privacy, and compliance frameworks
• Exposure to AI/ML tools (MLflow, Azure ML, or similar)

Preferred Qualifications:

• Experience with Generative AI or advanced analytics use cases
• Strong understanding of data modeling and architecture patterns
• Certifications in Databricks or cloud platforms

Soft Skills:

• Strong communication and stakeholder management skills
• Problem-solving and analytical thinking
• Ability to lead and mentor teams

Nice to Have:

• Experience with real-time data processing (Kafka, streaming)
• Familiarity with DevOps and CI/CD for data pipelines