Title: Databricks Architect/Lead/Engineer
Location:Houston, TX — candidate must be based in Houston (local candidates only)
Work Model:Onsite, 3 days per week in office (hybrid)
Engagement Type: Contract
Duration: 2 years (with potential to extend up to 5 years)
Description:
•Design the overall Databricks Lakehouse architecture, including ingestion, storage (Delta Lake), processing, and consumption layers.
•Define reference architectures, design patterns, and best practices for data engineering, analytics, and ML workloads.
•Lead decisions on workspace organization, compute strategy (clusters, SQL warehouses, serverless), and cost optimization.
•Architect data governance and security using Unity Catalog, including access controls, lineage, and data classification.
•Establish CI/CD, DevOps, and Infrastructure-as-Code standards for the platform (e.g., Terraform, Databricks Asset Bundles).
•Guide integration with cloud services (AWS / Azure / Google Cloud Platform) and source systems.
•Partner with stakeholders, security, and leadership to translate business requirements into technical roadmaps.
•Provide technical oversight and mentorship to tech leads and engineers.
Required Qualifications
•10+ years in data architecture / data engineering, with 4+ years hands-on with Databricks.
•Deep expertise in Delta Lake, Spark (PySpark/Scala), and the Lakehouse architecture.
•Strong knowledge of Unity Catalog, data governance, and security models.
•Experience architecting on at least one major cloud (AWS, Azure, or Google Cloud Platform).
•Proven track record designing large-scale, production data platforms.
•Strong stakeholder management and the ability to communicate architecture to both technical and business audiences.
Preferred
•Databricks certifications (e.g., Databricks Certified Data Engineer Professional, Solutions Architect).
•Experience with streaming (Structured Streaming, Kafka), MLflow, and lakehouse ML workflows.
•Background in oil & gas, energy, or other large enterprise environments.
Never miss an opportunity! Create an alert based on the job you applied for.
