Databricks Engineer

Job Title: Databricks Engineer

Location: Austin, TX Onsite

Key Responsibilities

·        Analyze business requirements and recommend effective technology solutions.

·        Design, develop, and optimize scalable data pipelines using Databricks and Apache Spark.

·        Build and maintain ETL/ELT pipelines for enterprise data integration.

·        Develop and manage Lakeflow Declarative Pipelines (formerly Delta Live Tables/DLT).

·        Implement Medallion Architecture (Bronze, Silver, Gold) and Delta Lake for data lakehouse solutions.

·        Design and maintain dimensional data models, including star and snowflake schemas.

·        Create Databricks SQL dashboards, Databricks Apps, and analytical solutions for business users.

·        Implement data governance, data quality, validation, and security best practices.

·        Monitor, troubleshoot, and optimize data workloads for performance, scalability, and cost efficiency.

·        Create and schedule data processing jobs using Lakeflow Jobs (Databricks Workflows) or similar orchestration tools.

·        Collaborate with business and technical teams to gather requirements and deliver data-driven solutions.

·        Document system requirements, technical designs, and implementation processes.

·        Present technical solutions and business insights to stakeholders.

Required Qualifications

·        8+ years of experience in IT supporting the design, development, deployment, or delivery of technology solutions.

·        8+ years of hands-on experience with Databricks and Apache Spark for building ETL/ELT pipelines.

·        8+ years of experience in data warehousing and dimensional data modeling (star and snowflake schemas).

·        Strong proficiency in SQL and Python (or Scala) for large-scale data processing.

·        Experience developing Databricks SQL dashboards and Databricks Apps.

·        Experience implementing data governance, data quality, and data security practices.

·        Experience with Lakeflow Declarative Pipelines (formerly Delta Live Tables/DLT).

·        Strong knowledge of Delta Lake, Medallion Architecture, data lakehouse design, and Lakeflow Jobs or similar orchestration tools (e.g., Airflow).

·        Excellent verbal and written communication skills with the ability to interact with both technical and business stakeholders.

Preferred Qualifications

·        Experience working in public sector or state government environments.

·        Databricks Certified Data Engineer Associate or Professional certification.

·        Experience with CI/CD practices, DevOps, and Git-based workflows for data pipelines.