Azure Cloud Engineer

Azure Cloud Engineer

Location: Bay Area, CA (or) Portland, OR (or) Denver, CO

Key Skills: Cloud Engineer (Azure) with experience in Databricks and data platform architecture.

Responsibilities:

Design and deploy scalable and secure Azure infrastructure solutions for our data platform, including Databricks, Azure Data Lake Storage, Azure Synapse Analytics, and other related services.

Collaborate with data engineers and data scientists to understand their requirements and ensure the cloud infrastructure meets their needs.

Implement best practices for infrastructure as code (IaC) using tools like Azure Resource Manager templates, Terraform, or Bicep.

Configure and manage Azure networking components, such as Virtual Networks, Subnets, Network Security Groups, and Azure Firewalls.

Implement and maintain security and compliance measures, including Azure Active Directory integration, role-based access control (RBAC), and data encryption.

Automate deployment and maintenance processes using Azure DevOps or other CI/CD tools.

Monitor and optimize the performance and cost of Azure resources, including Databricks clusters and Azure Data Lake Storage.

Stay up-to-date with the latest Azure services, features, and best practices, and provide guidance to the team.

Participate in on-call rotations and respond to incidents and alerts related to the Azure infrastructure.

Requirements:

Bachelor's degree in Computer Science, Information Technology, or a related field, or equivalent work experience.

3+ years of experience working with Azure cloud services, with a focus on data and analytics workloads.

Strong understanding of Azure infrastructure services, such as Virtual Machines, Storage Accounts, Azure Data Lake Storage, Azure Synapse Analytics, and Azure Databricks.

Experience with infrastructure as code (IaC) tools like ARM templates, Terraform, or Bicep.

Proficiency in scripting languages like PowerShell, Bash, or Python for automation and management tasks.

Knowledge of networking concepts and experience with Azure networking services (Virtual Networks, Network Security Groups, Azure Firewalls).

Familiarity with security and compliance best practices in Azure, including Azure Active Directory, RBAC, and data encryption.

Experience with CI/CD pipelines and automation tools like Azure DevOps or GitHub Actions.

Strong problem-solving and analytical skills, with the ability to troubleshoot and resolve complex issues.

Excellent communication and collaboration skills, with the ability to work effectively with cross-functional teams.

Preferred Qualifications:

Experience working with Apache Spark and Databricks environments.

Knowledge of data engineering and data science concepts and workflows.

Familiarity with containers and container orchestration platforms like Azure Kubernetes Service (AKS).

Understanding of cloud cost optimization strategies and resource management best practices.

Relevant cloud certifications, such as Azure Solutions Architect or Azure Data Engineer Associate.