Databricks Architect

Overview

On Site
$80 - $85
Contract - Independent
Contract - W2
Contract - 6 Month(s)

Skills

Databricks

Job Details

About Aspire:

Aspire Systems is a $170 million global technology services firm serving as a trusted technology partner for our customers. We work with some of the world's most innovative enterprises and independent software vendors, helping them leverage technology and outsourcing in our specific areas of expertise. It is a responsible and diverse organization of over 4,600 people across globe.

Our core philosophy of "Attention. Always." communicates our belief in lavishing care and attention on our customers and employees.

Job Title: Databricks Architect

Job Type: W2 or C2H

Location: St. Louis, MO

Working Model: Onsite 5 days

Years of experience: 8-12 years

Salary: $80/Hr. to $85/Hr.

Job Summary:

We are seeking a highly skilled Databricks Architect with hands-on experience in migrating Azure Data Factory (ADF) pipelines to Databricks, and a strong foundation in data governance and compliance. The ideal candidate will play a key role in modernizing our data platform, optimizing performance, and ensuring data integrity, security, and regulatory compliance.

Key Responsibilities:

  • Lead the migration of data pipelines and workflows from Azure Data Factory to Databricks, ensuring minimal downtime and performance optimization.
  • Architect and implement scalable data processing frameworks using Apache Spark, PySpark, Delta Lake, and Databricks notebooks.
  • Define and implement best practices for data governance, including data lineage, quality, cataloging, classification, and access control.
  • Collaborate with stakeholders to understand data needs and translate them into efficient Databricks-based solutions.
  • Optimize data pipelines for cost-efficiency, scalability, and performance.
  • Integrate Databricks with Azure services (e.g., Azure Data Lake Storage, Synapse, Purview, Key Vault).
  • Establish and maintain CI/CD pipelines, DevOps processes, and version control for Databricks environments.
  • Work with data stewards, compliance teams, and security architects to align with data governance and regulatory standards (e.g., GDPR, HIPAA).
  • Document architecture decisions, migration plans, and support knowledge transition.

Required Skills & Qualifications:

  • Bachelor s or Master s degree in Computer Science, Data Engineering, or related field.
  • 8+ years of experience in data engineering/architecture, with 2+ years in Databricks.
  • Proven experience in migrating ADF pipelines to Databricks, including orchestration, transformation, and dependency management.
  • Strong expertise in Apache Spark, PySpark, SQL, and Delta Lake.
  • Good understanding of data governance frameworks including metadata management, data cataloging , data masking, and RBAC.
  • Experience with Azure data ecosystem ADF, ADLS Gen2, Synapse, Key Vault, and Azure DevOps.
  • Familiarity with CI/CD practices in Databricks using tools like Databricks CLI, Repos, Terraform, or ARM templates.
  • Experience with performance tuning and cost optimization in Databricks environments.
  • Strong problem-solving skills, documentation discipline, and communication ability.

Preferred Skills:

  • Experience working with Unity Catalog in Databricks for fine-grained access control.
  • Knowledge of data quality tools.
  • Familiarity with enterprise data warehouse concepts and modern data stack.
  • Certifications such as Databricks Certified Data Engineer, Azure Data Engineer Associate, or Azure Solutions Architect.

Disclaimer: The salary, other compensation, and benefits information is accurate as of the date of this posting. Aspire reserves the right to modify this information at any time, subject to applicable law.

Our commitment to diversity and inclusion:

At Aspire Systems, we foster a work culture that appreciates diversity and inclusiveness. We understand that our multigenerational workforce belongs to different regions, cultures, economic backgrounds, race, gender, ethnicity, education, personality, and religion.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.