Databricks Architect

Overview

$DOE
Full Time

Skills

Python
AWS
Spark
Scala
DataBricks

Job Details

HMG America LLC is the best Business Solutions focused Information Technology Company with IT consulting and services, software and web development, staff augmentation and other professional services. One of our direct clients is looking for Databricks Architect in Remote. Below is the detailed job description.

Job Title: Databricks Architect
Location: Remote
Employment Type: Full-Time
Cloud Platforms: AWS and Azure

Job Description:

We are seeking an experienced Databricks Architect to join our growing cloud engineering team. The ideal candidate will have strong expertise in designing, implementing, and managing large-scale data solutions using Databricks on AWS and Azure cloud platforms. Key Responsibilities:

  • Design and implement end-to-end data solutions using Databricks, integrating with AWS and Azure cloud services.

  • Collaborate with data engineers, data scientists, and business stakeholders to understand data requirements and translate them into scalable solutions.

  • Develop data pipelines, data models, and advanced analytics solutions leveraging Databricks and cloud services.

  • Optimize performance of Databricks clusters, jobs, and Spark applications for large datasets.

  • Ensure data security, governance, and compliance standards are maintained across data platforms.

  • Provide architectural guidance and best practices around data lakehouse strategy.

  • Troubleshoot and resolve technical issues related to Databricks environments.

  • Automate deployment processes and manage infrastructure as code using tools like Terraform or CloudFormation.

  • Mentor and provide technical leadership to engineering teams on Databricks and cloud technologies.

Required Qualifications:

  • Proven experience as a Databricks Architect or similar role (15+ years of relevant experience).

  • Strong expertise in Databricks platform (notebooks, clusters, jobs, Delta Lake).

  • Hands-on experience with Spark, Python, Scala, or SQL in data processing workflows.

  • In-depth knowledge of AWS services (S3, EC2, IAM, Redshift, Glue) and Azure services (Data Factory, Data Lake Storage, Synapse).

  • Experience implementing data lake and data warehouse solutions in cloud environments.

  • Familiarity with CI/CD pipelines and Infrastructure as Code (Terraform, CloudFormation).

  • Strong understanding of data security, compliance, and governance frameworks.

  • Excellent problem-solving, communication, and architectural design skills.

Preferred Qualifications:

  • Databricks Certified Professional (Databricks Certified Data Engineer or Architect).

  • Experience with ML integration in Databricks.

  • Knowledge of real-time streaming technologies (e.g., Kafka, Spark Structured Streaming).

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.