Data Engineer Databricks & Amazon Lakehouse Architectures

Job Position : Data Engineer Databricks & Amazon Lakehouse Architectures
Job Duration : 6-12 Months
Location : Bridgewater, NJ (Onsite)

Job Description:

• Need to work 5 days onsite a week
• Design, implement, and optimize scalable and reliable data lakehouse architectures using Databricks (on Azure/AWS) and Amazon Lake House (AWS native stack including S3, Glue, Redshift, Athena, Lake Formation, etc.)
• Develop and maintain robust ETL/ELT pipelines using Databricks notebooks (PySpark/Scala), AWS Glue, or other modern tools
• Architect solutions that ensure high data quality, data governance, and security across cloud platforms
• Lead the migration, integration, and transformation of large datasets between on-premise and cloud (Databricks and AWS) environments
• Collaborate with data scientists, analysts, and business stakeholders to understand data needs and deliver data models for BI/AI/ML workloads
• Implement best practices for data lake organization, storage optimization (Delta Lake, Parquet), metadata cataloging, and versioning
• Monitor, troubleshoot, and optimize pipeline performance and cost efficiency in multi-cloud environments
• Document technical solutions, data flow diagrams, and operational processes
• Mentor junior engineers and promote a culture of innovation and continuous improvement
  

Required Skills & Qualifications:

• Bachelor s or Master s degree in Computer Science, Engineering, Information Systems, or a related field
• 8+ years of experience as a Data Engineer or similar role with hands-on experience in both Databricks Lakehouse and Amazon Lake House architectures
• Deep expertise in Databricks (Delta Lake, Databricks Workflows, Unity Catalog, Databricks SQL, and integrations with Azure or AWS)
• Advanced knowledge of the AWS Lake House stack: S3, Glue, Redshift, Athena, Lake Formation, Kinesis, and related services
• Strong proficiency in SQL, PySpark, and/or Scala
• Experience with ETL/ELT pipeline development, orchestration (Airflow, Databricks Jobs, Step Functions, etc.)
• Solid understanding of data warehousing concepts, schema design, and performance optimization
• Experience with data governance, access controls, lineage, and cataloging (Unity Catalog, AWS Glue Data Catalog, Lake Formation)
• Working knowledge of CI/CD, DevOps practices for data pipelines, and version control (Git)
• Excellent problem-solving skills, communication, and ability to work in cross-functional teams

Regards,

C. Mageshwari