Databricks Architect AWS Cloud

Overview

Contract - W2
50% Travel

Skills

databricks
Jenkins
GitHub
Data Quality
Apache Spark
Data Management
Governance
PySpark
Microsoft Azure
Amazon Web Services
Identity and Access Management
Information Engineering
Architecture
Continuous Integration
Stakeholder Management
Communication Skills
Extract Transform Load (ETL)
Amazon S3
Data Lakes
Electronic Medical Records
Business Analytics Applications
Data Ingestion
Kinesiology

Job Details

Databricks Architect AWS Cloud

Location: Minnesota (Local candidates only)
Duration: Long-term Contract

Job Summary:

We are seeking a highly experienced Databricks Architect with strong AWS expertise to lead the architecture, design, and optimization of large-scale data engineering and analytics solutions. The ideal candidate will have hands-on experience building enterprise-grade data platforms using Databricks, Delta Lake, and AWS cloud services.

Key Responsibilities:

  • Lead end-to-end architecture of Databricks-based data and analytics platforms on AWS.
  • Design scalable ELT/ETL pipelines using Databricks, Spark, and Delta Lake.
  • Architect data ingestion, data quality, governance, and lineage frameworks.
  • Implement CI/CD pipelines for Databricks notebooks and jobs.
  • Optimize Spark workloads and cluster configurations for performance and cost.
  • Integrate Databricks with AWS services (S3, Glue, Lambda, Redshift, EMR, Kinesis, IAM).
  • Provide architectural guidance and governance across engineering teams.
  • Collaborate with product owners, data engineers, and business stakeholders.

Required Skills:

  • 10+ years in data engineering/architecture; 4+ years in Databricks.
  • Strong hands-on Spark (PySpark/Scala) experience.
  • Expert knowledge of AWS cloud architecture-S3, IAM, Glue, Lambda, Step Functions, EMR, Redshift.
  • Experience implementing Delta Lakehouse architecture.
  • Strong knowledge of CI/CD (Azure DevOps, GitHub Actions, Jenkins, etc.).
  • Excellent communication and stakeholder management skills.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.