Sr Databricks Engineer

Remote • Posted 1 day ago • Updated 3 hours ago

Contract Independent

Contract W2

Contract Corp To Corp

Remote

Depends on Experience

Dia Software Solutions

Fitment

Dice Job Match Score™

🤯 Applying directly to the forehead...

Job Details

Skills

Implement ETL/ELT workflows for both structured and unstructured data
Automate deployments using CI/CD tools
data scientists
analysts
stakeholders
Design and maintain data models
schemas
database structures to support analytical and operational use cases
data lakes (Azure Data Lake Storage) and data warehouses
Evaluate and implement appropriate data storage solutions
Implement data validation and quality checks to ensure accuracy and consistency
Contribute to data governance initiatives
metadata management
data lineage
data cataloging
Implement data security measures
encryption
access controls
Proficiency in Python and R programming languages
Strong SQL querying and data manipulation skills
Azure cloud platform
DevOps
CI/CD pipelines
and version control systems
agile
multicultural environments
Strong troubleshooting and debugging capabilities
Design and develop scalable data pipelines using Apache Spark on Databricks
Optimize Spark jobs for performance and cost-efficiency
Integrate Databricks solutions with cloud services (Azure Data Factory)
Ensure data quality
governance
and security using Unity Catalog or Delta Lake
Deep understanding of Apache Spark architecture
RDDs
DataFrames
and Spark SQL
Databricks notebooks
clusters
jobs
and Delta Lake
ML libraries (MLflow Scikit-learn TensorFlow)
Databricks Certified Associate Developer for Apache Spark
Azure Data Engineer Associate

Summary

Hi,

Greetings from DIA SOFTWARE SOLUTIONS LLC!

We reaching out about an exciting Direct client opportunity with one of our clients. Please review the requirements and let me know if you are interested in this position?

Direct client Req:: Need Sr Databricks Engineer Remote, TX

PLEASE SEND THE RESUMES TO SKUMAR AT DIASOFTWARESOLUTIONS DOT COM !

Job Description:

The Worker is responsible for developing, maintaining, and optimizing big data solutions using the Databricks Unified Analytics Platform.

This role supports data engineering, machine learning, and analytics initiatives within this organization that relies on large-scale data processing.

Duties include:

Designing and developing scalable data pipelines
Implementing ETL/ELT workflows
Optimizing Spark jobs
Integrating with Azure Data Factory
Automating deployments
Collaborating with cross-functional teams
Ensuring data quality, governance, and security.

SKILLS MATRIX

Minimum Requirements: Candidates that do not meet or exceed the minimum stated requirements (skills/experience) will be displayed to customers but may not be chosen for this opportunity.
Actual Years Experience	Years Experience Needed	Required/ Preferred	Skills/Experience
	4	Required	Implement ETL/ELT workflows for both structured and unstructured data
	4	Required	Automate deployments using CI/CD tools
	4	Required	Collaborate with cross-functional teams including data scientists, analysts, and stakeholders
	4	Required	Design and maintain data models, schemas, and database structures to support analytical and operational use cases
	4	Required	Evaluate and implement appropriate data storage solutions, including data lakes (Azure Data Lake Storage) and data warehouses
	4	Required	Implement data validation and quality checks to ensure accuracy and consistency
	4	Required	Contribute to data governance initiatives, including metadata management, data lineage, and data cataloging
	4	Required	Implement data security measures, including encryption, access controls, and auditing; ensure compliance with regulations and best practices
	4	Required	Proficiency in Python and R programming languages
	4	Required	Strong SQL querying and data manipulation skills
	4	Required	Experience with Azure cloud platform
	4	Required	Experience with DevOps, CI/CD pipelines, and version control systems
	4	Required	Working in agile, multicultural environments
	4	Required	Strong troubleshooting and debugging capabilities
	3	Required	Design and develop scalable data pipelines using Apache Spark on Databricks
	3	Required	Optimize Spark jobs for performance and cost-efficiency
	3	Required	Integrate Databricks solutions with cloud services (Azure Data Factory)
	3	Required	Ensure data quality, governance, and security using Unity Catalog or Delta Lake
	3	Required	Deep understanding of Apache Spark architecture, RDDs, DataFrames, and Spark SQL
	3	Required	Hands-on experience with Databricks notebooks, clusters, jobs, and Delta Lake
	1	Preferred	Knowledge of ML libraries (MLflow, Scikit-learn, TensorFlow)
	1	Preferred	Databricks Certified Associate Developer for Apache Spark
	1	Preferred	Azure Data Engineer Associate

DIA SOFTWARE SOLUTIONS LLC.

Austin, TX 78727| Direct:

DIA SOFTWARE SOLUTIONS is an Affirmative Action/Equal Opportunity Employer that supports workplace diversity. All employment decisions are made without regard to race, color, religion, sex, national origin, age, disability, veteran status, marital or family status, sexual orientation, gender identity, or genetic information. All Dia soft staff must be able to demonstrate the legal right to work in the United States. DIA SOFTWARE SOLUTIONS is an E-Verify employer

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 91162472
Position Id: 8900700
Posted 1 day ago

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

It looks like there aren't any Similar Jobs for this job yet.

Search all similar jobs