Data Engineer with Databricks Genie

Need Consultants local to Plano, TX.

Databricks Genie experience is mandatory

Job Title: Data Engineer with Databricks Genie

Location: Plano, TX Onsite

Duration: 12+ Months

Experienced Data Engineer with strong expertise in Databricks, Databricks Genie (AI-powered data assistant), and modern data pipeline architecture.

Key Responsibilities

• Design and build scalable batch and streaming data pipelines using Databricks.
• Implement and optimize ETL/ELT workflows using Spark and Delta Lake.
• Configure and enable Databricks Genie for natural language-based data exploration and analytics.
• Develop and maintain data models, curated layers (Bronze/Silver/Gold), and semantic layers for analytics use cases.
• Build real-time ingestion pipelines using streaming frameworks (Kafka/Event Hubs).
• Implement data governance, lineage, cataloging, and security controls.
• Optimize performance, cost, and reliability of data workloads.
• Collaborate with data analysts, data scientists, and business stakeholders to enable AI-driven insights.
• Implement CI/CD for data engineering workflows using Infrastructure as Code (Terraform, ARM, CloudFormation).

Required Skills & Experience

• 8+ years of experience in Data Engineering.
• Strong hands-on experience with Databricks (Workflows, Delta Lake, Unity Catalog).
• Experience implementing or enabling Databricks Genie for conversational analytics.
• Proficiency in PySpark, SQL, and Python.
• Experience with Delta Lake architecture and medallion design patterns.
• Experience with cloud platforms (AWS/Azure/Google Cloud Platform).
• Strong knowledge of data pipeline orchestration tools (Airflow, ADF, etc.).
• Hands-on experience with streaming platforms (Kafka, Spark Streaming, Structured Streaming).
• Understanding of data modeling (Star schema, Snowflake schema).
• Experience in data quality frameworks and monitoring tools