Databricks Data Engineer

Overview

On Site
Full Time
Accepts corp to corp applications
Contract - Independent
Contract - W2
Contract - 6+ Month(s)

Skills

GCP
SQL Server
Python
Docker
Scala
Data Engineering
Kubernetes
Mentoring
SQl
ETL
Azure
Databricks
AWS
data migration
Data Modeling
CI/CD
Apache Spark
Delta Lake
Communication Skills
cloud platforms
Agile/Scrum
problem-solving
data governance
data warehousing
data pipelines
Cross-Functional Collaboration
Technical Leadership
Infrastructure as Code
Performance Optimization
data security
Data Quality
Feature Engineering
Data Wrangling
RDBMS to Databricks
MLflow
Spark MLlib
Machine Learning Pipelines
Big Data Technologies
DevOps for Data
Scalable Data Architecture
DataOps
End-to-End Data Pipeline Development
Migration from On-Prem to Cloud
Productionizing ML Models
Real-Time Data Processing
Secure and Compliant Data Systems
Cost-Effective Data Infrastructure

Job Details

Solution IT Inc. is looking for an Databricks Data Engineer for one of its clients in Houston, TX (Hybrid)

Job Title: Databricks Data Engineer

Responsibilities:

  • Design, develop, and implement robust data solutions using modern data platforms like Databricks.
  • Ensure scalable, reliable, and secure data environments that support advanced analytics and meet business needs.
  • Lead the migration of data from traditional RDBMS systems to Databricks environments.
  • Design scalable data pipelines and infrastructure to meet evolving data demands.
  • Develop and manage ETL processes using Databricks for efficient data extraction, transformation, and loading.
  • Optimize ETL workflows to improve performance and ensure data integrity.
  • Ensure seamless data transitions with minimal disruption to ongoing operations.
  • Monitor and optimize data system performance, reliability, scalability, and cost-effectiveness within Databricks environments.
  • Collaborate with data engineers, scientists, analysts, and product managers to define and deliver data solutions.
  • Establish and enforce best practices and standards for data engineering and data warehouse processes.
  • Evaluate and implement tools and technologies to improve data pipeline efficiency and effectiveness.
  • Mentor junior data engineers and provide technical leadership.
  • Design and operationalize end-to-end machine learning pipelines on Databricks, including data wrangling, feature engineering, training, and tuning.
  • Work with DevOps and infrastructure teams to deploy and manage production data systems.
  • Ensure compliance with organizational, industry, and regulatory data standards.
  • Collaborate with enterprise architects and IT leaders to align data architecture with overall IT strategy.
Required Skills:
  • BS or MS in Computer Science or a related field.
  • Extensive experience with Databricks, including ETL and data migration.
  • Databricks certifications are a strong plus.
  • Experience with cloud platforms like AWS, Azure, or Google Cloud Platform.
  • Familiarity with big data technologies and frameworks.
  • Experience implementing and optimizing ML workflows using Databricks ML, MLflow, scikit-learn, Spark MLlib, etc.
  • Strong knowledge of data warehousing, modeling, and SQL.
  • Experience transitioning from RDBMS systems to cloud-native platforms.
  • Proficiency in Python, SQL, and scripting languages.
  • Understanding of data governance, data quality, and data security best practices.
  • Expertise in database technologies, particularly SQL Server.
  • Familiarity with Docker and orchestration tools like Kubernetes.
  • Excellent analytical and problem-solving skills.
  • Strong communication and collaboration skills, especially in cross-functional environments

Work Site: Houston, TX (Hybrid)

Duration: 6+ Months

Expected Start Date: 2 Weeks

Number of Positions: 01

Please send your responses back to

About Solution IT

Solution IT is a national IT consulting company specializing in: Technology Staffing and Oracle E-Business Solutions based in Boston, Massachusetts.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.