Director of Data Engineering (Databricks Platform Lead)

Remote • Posted 1 hour ago • Updated 1 hour ago
Full Time
No Travel Required
Remote
$200/yr
Company Branding Image
Fitment

Dice Job Match Score™

📊 Calculating match score...

Job Details

Skills

  • Databricks
  • Data Engineering
  • Artificial Intelligence

Summary

Location: Charlotte, NC (Hybrid preferred; open to remote for the right candidate; likely out of the services business of use to connecting many disparate systems via Databricks / P.E. Roll up situation)
Compensation: Low 200''s + base + bonus + equity

Company Overview

Our client is a rapidly scaling, private equity-backed services platform growing through acquisition. With 50+ operating companies and a highly decentralized environment, the organization is undergoing a major transformation to unify data, enable AI, and drive operational excellence across the enterprise.

Today, data exists across dozens of disconnected ERP systems and environments. The mandate is clear: centralize, govern, and activate data at scale using Databricks as the core platform.


The Opportunity

This is a hands-on Director of Data Engineering role for a builder—someone who can operate as both architect and player-coach.

You will own the design and execution of the enterprise data platform, bringing together fragmented systems into a scalable, governed, and AI-ready ecosystem. This is not a pure leadership role—you will be expected to write code, build pipelines, and lead from the front.

You will play a critical role in establishing the foundation for:

  • Enterprise data consolidation
  • Data product development
  • AI/ML enablement and LLM integration

What You’ll Own

Data Platform Architecture & Build (Hands-On)

  • Design and implement a Databricks-first data platform on AWS
  • Build and operationalize medallion architecture (bronze, silver, gold layers)
  • Develop scalable, reusable data pipelines across 50+ disconnected systems
  • Standardize ingestion patterns across ERP systems (e.g., Great Plains/WinSoft and others)
  • Write production-grade code in Python/SQL within Databricks

Data Integration & Connectivity

  • Solve for highly fragmented, non-standard environments across acquired companies
  • Build frameworks for integrating data across isolated networks and systems
  • Establish scalable ingestion strategies (batch + streaming where applicable)

Data Products & Business Enablement

  • Partner with business stakeholders to create trusted, governed data products
  • Enable operational reporting and real-time insights
  • Support downstream tools (Power BI, analytics layers, etc.)

AI & Advanced Analytics Enablement

  • Lay the foundation for AI/ML and LLM use cases within Databricks
  • Support development of an LLM gateway and AI consumption layer
  • Collaborate with AI leadership on use-case prioritization and execution

Governance & Operating Model

  • Help establish and participate in a data governance committee
  • Define standards for:
    • Data quality
    • Data ownership
    • Access controls & compliance
  • Align platform decisions to business outcomes, not just technical elegance

Leadership (Player-Coach)

  • Lead and mentor a small but growing team of:
    • Data engineers
    • Integration specialists
    • Analytics/BI resources
  • Set engineering standards and best practices
  • Remain deeply involved in hands-on delivery

What You Bring

Core Technical Experience

  • Deep, hands-on experience with Databricks (non-negotiable)
  • Strong expertise in:
    • Python
    • SQL
    • Spark / distributed data processing
  • Experience building end-to-end data platforms in AWS
  • Proven implementation of medallion architecture / lakehouse design

Integration & Complexity Experience

  • Experience integrating multiple ERP systems or fragmented environments
  • Background in M&A-heavy or decentralized organizations is a strong plus
  • Ability to operate in environments with little standardization and high ambiguity

Data Product Mindset

  • Experience building data products, not just pipelines
  • Ability to translate business needs into scalable data solutions

Leadership + Hands-On Balance

  • Prior experience at Lead / Principal / Director level
  • Comfortable being both architect and executor
  • Proven ability to build from 0 ? 1

What Success Looks Like

In the first 6–12 months, you will:

  • Stand up a scalable Databricks data foundation
  • Begin consolidating data from dozens of siloed systems
  • Deliver initial data products to the business
  • Establish governance, standards, and engineering discipline
  • Enable the first wave of AI-driven use cases

Why This Role is Compelling

  • True greenfield build with executive support
  • Direct impact on enterprise transformation + AI strategy
  • High visibility across business and leadership
  • Opportunity to shape long-term data and AI architecture

Context from Current State

  • 50+ systems across acquired entities, many disconnected
  • Databricks already in place and positioned as the future core platform
  • Snowflake being deprecated for limited use cases
  • Strong executive alignment around data + AI as strategic priorities

 

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: PTPyNBxOPFsfRK6
  • Position Id: 8968220
  • Posted 1 hour ago

Company Info

About Zfinity Global

Zfinity Global, established in 2024 and headquartered in Nashville, TN, extends its global presence with operations in India. Specializing in delivering targeted talent solutions, Zfinity Global drives business success by implementing expert recruitment strategies and producing outstanding results for global clients.

Service Areas

With expertise in Staffing Services, Offshore Delivery, Managed Services and Recruitment Process Outsourcing (RPO), Zfinity Global thrives on being locally rooted yet globally connected, ensuring efficient, scalable, and high-quality workforce solutions tailored to business needs.

About_Company_OneAbout_Company_Two
Create job alert
Set job alertNever miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Remote

Today

Easy Apply

Full-time

140,000 - 180,000

Remote

Yesterday

Easy Apply

Contract

60 - 65

Search all similar jobs