Big Data Engineer

Hybrid in Austin, TX, US • Posted 3 hours ago • Updated 3 hours ago
Contract W2
Hybrid
$60,000 - $80,000/yr
Fitment

Dice Job Match Score™

🔢 Crunching numbers...

Job Details

Skills

  • SQL

Summary

Seeking a Big Data Engineer to develop, maintain, and optimize scalable data solutions using Databricks, Apache Spark, Azure Data Factory, Delta Lake, and Azure Data Lake Storage. The consultant will build ETL/ELT pipelines, optimize Spark workloads, automate deployments, and maintain data quality, governance, and security.

Required Experience

  • 4+ years with ETL/ELT, Python, R, SQL, Azure, CI/CD, DevOps, data modeling, data lakes, and data warehouses.
  • 3+ years with Databricks, Apache Spark, Spark SQL, Delta Lake, Azure Data Factory, and Unity Catalog.
  • Strong understanding of Spark architecture, RDDs, DataFrames, clusters, notebooks, and jobs.
  • Experience implementing data validation, metadata management, lineage, cataloging, encryption, access controls, and auditing.
  • Strong troubleshooting, Agile delivery, and cross-functional collaboration skills.

Preferred

  • MLflow, Scikit-learn, or TensorFlow experience.
  • Databricks Certified Associate Developer for Apache Spark.
  • Microsoft Azure Data Engineer Associate certification.

Important Requirements

Selected candidates must pick up equipment in Austin on the first day and may be required to attend onsite meetings with minimal notice.

--

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 10500016
  • Position Id: 9089505
  • Posted 3 hours ago
Contact the job poster
Rakesh Vangala

Rakesh Vangala

Recruiter @ BURGEON IT SERVICES LLC
Create job alert
Set job alertNever miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Hybrid in Austin, Texas

Today

Easy Apply

Contract

Depends on Experience

No location provided

Today

Full-time

Remote

Today

Easy Apply

Contract

Depends on Experience

Remote

22d ago

Full-time

Search all similar jobs