Overview
Remote
Depends on Experience
Accepts corp to corp applications
Contract - W2
Contract - Independent
Contract - 12 Month(s)
Skills
AWS
Databricks Lakehouse
Data Pipeline
Scala
or Java.
Python
Data Modelling and ETL
Data Governance and Quality
Job Details
Title: Databricks Developer
Location: New York- Remote
Type: Contract
Duration: 12+ Months
Job Description
AWS Proficiency:
- Strong understanding of AWS services relevant to data engineering, particularly:
- AWS Glue: Experience in building and managing ETL pipelines.
- Amazon S3: Knowledge of data storage and management.
- AWS Lambda/Step Functions: Familiarity with serverless computing for data processing.
- Understanding of data warehousing solutions.
Proficient in Databricks Lakehouse concepts, including:
- Delta Tables: Experience in using Delta Lake for data storage and management.
- Delta Sharing: Knowledge of sharing data across platforms securely.
- Unity Catalog: Familiarity with data governance and access control in Databricks.
- Medallion Architecture: Understanding of the Medallion architecture (Bronze, Silver, Gold layers).
Data Pipeline Development:
- Experience in designing and implementing data pipelines using AWS and Databricks.
- Proficient in orchestrating data workflows and managing data ingestion processes.
Programming Skills:
- Strong programming skills in languages such as Python, Scala, or Java.
- Proficient in SQL for querying and manipulating data.
Data Modelling and ETL:
- Experience in data modelling for Lakehouse architecture.
- Knowledge of ETL best practices and data transformation techniques.
Data Governance and Quality:
- Understanding of data governance practices and frameworks.
- Experience implementing data quality checks and monitoring processes.
Project Experience:
- Proven track record of delivering data engineering projects, particularly in cloud environments.
- Experience with large-scale data processing and analytics projects.
Collaboration and Agile Methodologies:
- Experience working in Agile teams and collaborating with data scientists, analysts, and other stakeholders.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.