AWS Databricks Data Engineer

Our client is looking AWS Databricks Data Engineer project Los Angeles CA (Hybrid) below is the detailed requirements.

Job Title : AWS Databricks Data Engineer

Location : Los Angeles CA (Hybrid)

Duration : Full Time

Job Description:

Summary : We are seeking a highly skilled AWS Data Engineer with strong expertise in SQL, Python, PySpark, Data Warehousing, and Cloud-based ETL to join our data engineering team. The ideal candidate will design, implement, and optimize large-scale data pipelines, ensuring scalability, reliability, and high performance. This role requires close collaboration with cross-functional teams and business stakeholders to deliver modern, efficient data solutions.

Required Skills & Experience

• Strong expertise in Databricks (Delta Lake, Unity Catalog, Lakehouse Architecture, Table Triggers, Workflows, Delta Live Pipelines, Databricks Runtime, etc.).
• Proven ability to implement robust PySpark solutions.
• Hands on experience with Databricks Workflows & orchestration.
• Solid knowledge of Medallion Architecture (Bronze/Silver/Gold).
• Significant experience designing or rebuilding batch heavy data pipelines.
• Strong background in query optimization, performance tuning, and Spark shuffle optimization.
• Ability to handle and process tens of millions of records efficiently.
• Familiarity with Genie enablement concepts (understanding required; deep experience optional).
• Experience with CI/CD, environment setup, and Git-based development workflows.
• Solid understanding of AWS cloud, including:
  • IAM
  • Networking fundamentals
  • Storage integration (S3, Glue Catalog, etc.)

Certifications

• Databricks Certified Data Engineer Associate / Professional
• AWS Data Engineer or AWS Solutions Architect certification

Preferred Experience

• Experience with Databricks Runtime configurations and advanced features.
• Knowledge of streaming frameworks such as Spark Structured Streaming.
• Experience developing real-time or near real-time data solutions.
• Exposure to GitLab pipelines or similar CI/CD systems.