Databricks Engineer - ONLY FULLTIME - NO C2C

Overview

On Site
Full Time

Skills

Python
Azure
Databricks

Job Details

NO C2C PLEASE
Role: Databricks Engineer/ Lead
Location : Cincinnati OH
Full time
About the Role
We are seeking a highly motivated and skilled Sr. Databricks Engineer to join our growing Data Engineering team. In this role, you will play a critical part in designing, developing, and maintaining data pipelines and solutions on the Databricks platform. You will work closely with cross-functional teams to deliver high-quality data solutions that drive business insights and support key initiatives.
Responsibilities:
  • Design, develop, and deploy data pipelines using Databricks, including data ingestion, transformation, and loading (ETL) processes.
  • Develop and maintain high-quality, scalable, and maintainable Databricks notebooks using Python.
  • Work with Delta Lake and other advanced features.
  • Leverage Unity Catalog for data governance, access control, and data discovery.
  • Develop and optimize data pipelines for performance and cost-effectiveness.
  • Integrate with various data sources, including but not limited to databases and cloud storage (Azure Blob Storage, ADLS, Synapse), and APIs.
  • Experience working with Parquet files for data storage and processing.
  • Experience with data integration from Azure Data Factory, Azure Data Lake, and other relevant Azure services.
  • Perform data quality checks and validation to ensure data accuracy and integrity.
  • Troubleshoot and resolve data pipeline issues effectively.
  • Collaborate with data analysts, business analysts, and business stakeholders to understand their data needs and translate them into technical solutions.
  • Participate in code reviews and contribute to best practices within the team.
Qualifications:
  • 10+ years of experience as a Data Engineer with a strong focus on Databricks.
  • Strong proficiency in Python and SQL.
  • Hands-on experience with Databricks features such as delta lake, clusters, notebooks, jobs, and workspaces.
  • Experience with data warehousing concepts and best practices.
  • Strong understanding of data modeling and data quality principles.
  • Experience with Agile development methodologies.
  • Excellent communication and collaboration skills.
  • Ability to work independently and as part of a team.
Preferred Qualifications:
  • Experience with Azure cloud services (e.g., Azure Data Factory, Azure Synapse Analytics)
  • Experience with CI/CD pipelines for Databricks workloads.
  • Experience with data visualization tools (e.g., Power BI, Tableau)
  • Experience with data security and privacy best practices
  • Knowledge of Insurance domain
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.