Role: Azure Databricks Architect with AI
Location: Chicago, IL Hybrid
Job Type: Contract
Job Summary
We are looking for an experienced Azure Databricks Architect with AI to design and implement scalable data and AI solutions on Microsoft Azure. The ideal candidate will have strong experience with Azure Databricks, data engineering, cloud architecture, and AI/ML technologies.
Key Responsibilities
Design and implement scalable data platforms using Azure Databricks and Microsoft Azure.
Develop data pipelines using Databricks, PySpark, SQL, and Azure Data Factory.
Design and optimize data lakes, data warehouses, and lakehouse architectures.
Integrate AI/ML solutions into enterprise data platforms.
Work with Generative AI, Azure OpenAI, ML models, and LLM-based solutions.
Establish best practices for data security, governance, monitoring, and performance.
Collaborate with data engineers, data scientists, application teams, and business stakeholders.
Provide technical leadership and guide development teams on architecture and implementation.
Troubleshoot performance, scalability, and data integration issues.
Required Skills
8+ years of experience in data engineering, cloud architecture, or related roles.
Strong hands-on experience with Azure Databricks and PySpark.
Strong knowledge of Azure Data Lake Storage (ADLS), Azure Data Factory, and Azure Synapse.
Experience with Python, SQL, and data modeling.
Experience designing Lakehouse architectures.
Good understanding of AI/ML and Generative AI concepts.
Experience with Azure OpenAI, LLMs, RAG, or AI-powered applications is preferred.
Knowledge of data governance, security, and cloud best practices.
Strong communication and problem-solving skills.
Preferred Qualifications
Microsoft Azure certifications such as Azure Solutions Architect Expert or Azure Data Engineer Associate.
Experience with Unity Catalog and Databricks governance.
Experience implementing enterprise-scale AI and analytics solutions.
Experience working in Agile development environments.
