Azure Databricks Senior Cloud Architect

Overview

On Site
Full Time

Skills

Airflow
Azure
Generative AI
Kafka
Data Warehouse
OpenAI
Spark
Azure Databricks
Python

Job Details


Required Qualifications:

  • Bachelor s degree or foreign equivalent required from an accredited institution. Will also consider three years of progressive experience in the specialty in lieu of every year of education.

  • At least 11 years of experience with Information Technology.

  • Proven years of hands-on experience in architecting solutions on Azure, Databricks and Spark

  • Proficiency in data engineering tools: Spark, Kafka, Airflow, SQL, Python.

  • Strong understanding of retail or CPG data domains

  • Familiarity with vector search and semantic retrieval for retail use cases.

  • Expertise in data privacy and compliance in consumer-facing environments

  • Good experience in end-to-end implementation of data warehouse and data marts

  • Strong understanding of Data Warehousing concepts of Data Vault, Dimensional modelling, OLAP design

  • Extensive experience with CI/CD platforms such as GitLab CI, GitHub Actions, AWS CodeBuild, Azure Pipelines, and Jenkins

  • Experience designing and implementing complex solutions for distributed systems

  • Experience in leading and mentoring teams

Preferred Qualifications:

  • Experience in Relational Modeling, Dimensional Modeling and Modeling of Unstructured Data

  • Familiarity with log and monitoring solutions

  • Experience in utilizing development containers, unit testing/code quality review/linting - standard best practice

  • Good understanding of Agile software development frameworks

  • Strong communication and Analytical skills

  • Ability to work in teams in a diverse, multi-stakeholder environment comprising of Business and Technology teams

  • Experience and desire to work in a global delivery environment

  • Experience with Generative AI frameworks: OpenAI, Hugging Face, LangChain

  • Knowledge of Model Context Protocol (MCP) or similar AI integration standards.

  • Experience and detailed knowledge with Master Data Management, ETL, Data Quality, metadata management, data profiling, micro-batches, streaming data loads

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.