Senior Data Engineer- Data Genie, Databricks
Contract W2
12 Months
On-site
$60 - $70/hr
ConnectedX, Inc.
Fitment
Dice Job Match Score™
🔗 Matching skills to job...
Job Details
Skills
- Databricks Genie
- ELT
- Databricks
- unity catalog
- Delta Lake
Summary
We are seeking a highly experienced Databricks Genie Engineer with data engineering experience and deep expertise in the Databricks Lakehouse platform. The ideal candidate will design, build, and optimize scalable data pipelines, implement AI-powered data interactions using Databricks Genie, and drive enterprise-grade data solutions across cloud environments.
You will play a key role in architecting modern data platforms, enabling self-service analytics through natural language querying (Genie), and ensuring high performance, security, and cost optimization across large-scale workloads.
Key Responsibilities
Databricks & Lakehouse Engineering
- Design and implement scalable data pipelines using Databricks (PySpark, SQL).
- Architect and manage Delta Lake tables (Bronze, Silver, Gold layers).
- Optimize Spark jobs for performance, scalability, and cost efficiency.
- Implement Unity Catalog for data governance and access control.
- Manage cluster configuration, autoscaling, and job orchestration.
Databricks Genie Implementation
- Configure and optimize Databricks Genie for natural language-to-SQL analytics.
- Build semantic models and curated datasets to support Genie use cases.
- Improve AI-driven query accuracy and optimize prompt engineering.
- Collaborate with business users to enable self-service analytics.
- Troubleshoot and tune Genie-generated SQL for performance and correctness.
Data Architecture & ELT
- Design modern ELT architectures leveraging Delta Lake.
- Implement incremental processing (CDC, streaming with Structured Streaming).
- Develop data quality frameworks and validation pipelines.
- Handle schema evolution and medallion architecture best practices.
Cloud & Integration
- Work across AWS / Azure / Google Cloud Platform cloud platforms.
- Integrate with S3 / ADLS / GCS object storage.
- Implement CI/CD pipelines for Databricks deployments.
- Manage secrets, tokens, and secure connectivity.
- Integrate BI tools (Power BI, Tableau) and AI/ML workflows.
Required Qualifications
- 10+ years of experience in Data Engineering.
- 5+ years of hands-on experience with Databricks.
- Strong expertise in PySpark and Spark SQL.
- Deep understanding of Delta Lake and Lakehouse architecture.
- Experience implementing Databricks Genie or AI-driven analytics solutions.
- Strong experience with cloud platforms (AWS/Azure/Google Cloud Platform).
- Expertise in building scalable ELT pipelines.
- Solid understanding of data governance and security frameworks.
- Experience with CI/CD (Azure DevOps, GitHub Actions, Terraform, etc.).
- Strong troubleshooting and debugging skills.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
- Dice Id: 91015520
- Position Id: 9034403
- Posted 13 hours ago
Company Info
About ConnectedX, Inc.
Our customer focused approach makes - Our clients needs our top priority.
Our emphasis is on people. We know that it is seasoned professionals with industry knowledge and experience that really make an impact.
Our values are simple. We believe in long-term relationships and building high-performance teams that deliver topnotch service.
We provide our clients with exceptional value by being flexible in our services and guarantee a high level of personal attention and satisfaction. We help our clients think and act like a software company in the Application Economy . Designing and developing micro-services through software driven innovation.
Continuous delivery through continuous improvements at the convergence of design, data and devices is the foundation of our business. We pride ourselves on the wide range of personalized services we create for our clients.
Our mission
Our mission is to deliver next generation consumer and enterprise experiences for our clients in a connected world. Our services help clients operationalize to be in perpetual Beta so that our clients can focus on what they do best. Our people are trusted to innovate, integrate and create value from connected services.
Vision and Values
Seasoned people working together with passion, enthusiasm and commitment offering laser like focus delivering innovative services and disruptive solutions to the Automotive space. We aspire to be a company that retains and nurtures its strong beliefs and values as it grows by
1. Serving our clients to act fast, and use Apps to drive growth, innovation and differentiation
2. Empower our employees to deliver amazing outcomes
3. Investing time and money into the communities we are connected where we live, work and play in a connected world.
Our emphasis is on people. We know that it is seasoned professionals with industry knowledge and experience that really make an impact.
Our values are simple. We believe in long-term relationships and building high-performance teams that deliver topnotch service.
We provide our clients with exceptional value by being flexible in our services and guarantee a high level of personal attention and satisfaction. We help our clients think and act like a software company in the Application Economy . Designing and developing micro-services through software driven innovation.
Continuous delivery through continuous improvements at the convergence of design, data and devices is the foundation of our business. We pride ourselves on the wide range of personalized services we create for our clients.
Our mission
Our mission is to deliver next generation consumer and enterprise experiences for our clients in a connected world. Our services help clients operationalize to be in perpetual Beta so that our clients can focus on what they do best. Our people are trusted to innovate, integrate and create value from connected services.
Vision and Values
Seasoned people working together with passion, enthusiasm and commitment offering laser like focus delivering innovative services and disruptive solutions to the Automotive space. We aspire to be a company that retains and nurtures its strong beliefs and values as it grows by
1. Serving our clients to act fast, and use Apps to drive growth, innovation and differentiation
2. Empower our employees to deliver amazing outcomes
3. Investing time and money into the communities we are connected where we live, work and play in a connected world.
Create job alert
Never miss an opportunity! Create an alert based on the job you applied for.Similar Jobs
It looks like there aren't any Similar Jobs for this job yet.
Search all similar jobs