Data Engineer with Databricks

Role: Data Engineer with Databricks/Spark and Gen AI/LLM Experience
Location: Santa Clara, CA (Onsite from Day 1)
Experience: 12-14 year max

About the Role
We are seeking a skilled Data Engineer with hands-on experience in Databricks to join our team in Santa Clara. The ideal candidate will have a strong background in data engineering, excellent coding skills, and the ability to work collaboratively in a fast-paced environment.

Responsibilities:
- Design, develop, and maintain scalable data pipelines and ETL processes using Databricks.
- Collaborate with data scientists and analysts to understand data requirements and deliver solutions.
- Optimize and troubleshoot data workflows to ensure high performance and reliability.
- Implement data quality checks and ensure data integrity.
- Prior experience in GEN AI Models or custom solutions using databricks might be preferred. Extensive use of Genie Models might be added advantage.
- Participate in code reviews and contribute to best practices for data engineering.

Role Expectations
Build and operate production-grade Databricks/Spark pipelines (batch and streaming) using Delta/Iceberg.
Apply robust data quality checks and data lineage (using tooling or catalog-driven approaches).
Understand and design agentic workflows for data and GenAI-driven use cases (orchestration of multiple tools/agents, reliability, and observability).
Implement CI/CD for data (Git-based workflows, automated tests, promotion/rollback between environments).

Nice to have
Practical exposure to RAG/LLM-based data pipelines and GenAI/LLM integration in data engineering solutions, including agentic workflow patterns.

**Requirements:**
- Bachelor's degree in Computer Science, Engineering, or a related field.
- Proven experience as a Data Engineer with hands-on coding experience.
- Proficiency in Databricks and Spark.
- Strong programming skills in Python, Scala, or Java.
- Experience with SQL and relational databases.
- Familiarity with cloud platforms such as AWS, Azure, or Google Cloud Platform.
- Excellent problem-solving skills and attention to detail.
- Strong communication and teamwork abilities.