Databricks Developer

Overview

On Site
Depends on Experience
Contract - W2
Contract - 12 Month(s)

Skills

Databricks
Apache Spark
Python
Hadoop
Kafka
Datawarehousing concepts
GIT
Developing data pipelines

Job Details

JOB DESCRIPTION:

Position: Databricks Developer
Location: NYC, NY
Duration: 12 months

Responsibilities:
The Databricks Developer will be responsible for designing, developing, and maintaining scalable data pipelines and solutions using Databricks. The ideal candidate will have a strong background in data engineering, experience with big data technologies, and a deep understanding of Databricks and Apache Spark.
Key Responsibilities:
Design and develop scalable data pipelines and ETL processes using Databricks and Apache Spark.
Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
Optimize and tune data pipelines for performance and scalability.
Implement data quality checks and validations to ensure data accuracy and consistency.
Monitor and troubleshoot data pipelines to ensure reliable and timely data delivery.
Develop and maintain documentation for data pipelines, processes, and solutions.
Implement best practices for data security, governance, and compliance.
Participate in code reviews
Required skills:

Experience in data engineering or a related field.
Strong experience with Databricks and Apache Spark.
Proficiency in programming languages such as Python, Scala, or Java.
Experience with big data technologies such as Hadoop, Hive, and Kafka.
Strong SQL skills and experience with relational databases.
Experience with cloud platforms such as AWS, Azure, or Google Cloud.
Knowledge of data warehousing concepts and technologies.
Experience with version control systems such as Git.
Strong problem-solving skills and attention to detail.
Excellent communication and collaboration skills.

Desired skills:
Experience with Delta Lake and Databricks Delta.
Experience with data visualization tools such as Power BI, Tableau, or Looker.
Knowledge of machine learning and data science concepts.
Experience with CI/CD pipelines and DevOps practices.
Certification in Databricks, AWS, Azure, or Google Cloud.
Required Education:
At least a Bachelor s Degree (or equivalent experience) in Computer Science, Software/Electronics/Electrical Engineering, Information Systems or closely related field is required.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

About RiseIT Solutions