Job Title: Sr. Databricks Developer – 12+ years exp
Location: Wilmington, DE - 5 Days onsite Role
Long Term project: 2 Years
$56/hr on H1B Transfer and $59/hr on W2.
Client Interview Process:
Round 1: zoom interview with 2-3 people panel
Round 2: In-Person at any JPMC location
Skillset – Databricks Internal, AWS Internal, unity catalog, metadata, and Data Governanace.
Sr Databricks Engineer
As a Sr Databricks Engineer, you will play a crucial role in our data engineering team, focusing on building, maintaining, and optimizing our data pipelines and analytics platforms. You will be responsible for designing and implementing scalable, efficient, and reliable data solutions using Databricks and AWS. Your work will directly impact our ability to derive insights from data and drive data-driven decision-making across the organization.
Responsibilities
- Design, develop, and maintain high-performance data pipelines using Databricks and AWS services.
- Implement ELT/ETL processes to ensure data is efficiently and accurately extracted, transformed, and loaded into Databricks.
- Leverage Delta Lake and Lakehouse architecture to build robust data storage and processing solutions.
- Collaborate with data scientists and analysts to support their needs and provide data engineering expertise.
- Develop and manage data workflows using PySpark/Scala Spark, ensuring they are optimized for performance and cost.
- Implement ML Ops practices on Databricks to streamline machine learning model deployment and management.
- Ensure data governance and metadata management best practices are followed within the Databricks environment.
- Optimize data processing and storage costs while maintaining performance and reliability.
- Mentor and guide junior data engineers, fostering a culture of knowledge sharing and continuous improvement.
Requirements
<>Required Skills>
- 12+ Years of Total IT Experience needed
- Deep knowledge in Databricks & AWS ecosystems.
- Proficiency in Delta Lake and Lakehouse architecture (Lake flow, Lake Base).
- Strong experience with PySpark/Scala Spark for data engineering tasks.
- High volume production-grade Databricks ELT/ETL engineering experience.
- Experience with streaming data and DLT (Delta Live Tables).
- Knowledge of ML Ops on Databricks.
- Understanding of Data Mesh architecture.
- Advanced cost/performance engineering skills.
- Experience with data engineer roles on Databricks & AWS.
<>Preferred Skills>
- Experience with Unity Catalog and metadata management.
- Familiarity with data governance best practices.
- Strong problem-solving skills and ability to work in a fast-paced environment.
- Excellent communication and collaboration skills.