Sr DataBricks Engineer _ NY/NJ Locals _ Healthcare Insurance _ 12+ Years

Sr Databricks Engineer

Location: NY/NJ (Hybrid)

Responsibilities:

• Develop, optimize, and maintain Databricks notebooks for ETL and data transformation.
• Design and implement scalable data pipelines using Databricks and Azure Data Factory.
• Apply strong knowledge of distributed computing concepts to large-scale data processing.
• Work with data lake architectures and Delta Lake for reliable and performant data storage.
• Automate data quality checks, monitoring, and logging.
• Integrate data from multiple cloud sources such as Azure Blob, Data Lake Storage, and SQL/NoSQL databases.
• Perform data modeling and schema design for analytical workloads.
• Implement CI/CD pipelines for Databricks notebooks and jobs.
• Manage workspace administration, cluster management, and job orchestration in Databricks.

Qualifications:

• Proven experience with Databricks for large-scale data processing and analytics..
• Familiarity with cloud data platforms, preferably Azure.
• Understanding of data engineering best practices and distributed systems.
• Experience with data pipeline orchestration and automation.

Best Regards

Naveen Pulletikurthi

Sr Recruiter

Sunray Informatics Inc

Email: naveen @ sunrayinformatics dot com