Big Data Engineer(with DevOps)

Overview

Accepts corp to corp applications
Contract - Independent

Skills

Python
DevOps
ETL
bigdata
hdfs

Job Details


We are looking for BigData Engineer with DevOps Exp. .

Location: Pittsburgh, PA

Position Overview

We are seeking a highly experienced Senior Big Data DevOps Engineer with at least 8 years of professional experience in managing large-scale data platforms and automation frameworks. The ideal candidate will have deep expertise in HDFS, Hive, Impala, PySpark, Python, and DevOps automation tools such as uDeploy and Jenkins. This role is critical in ensuring smooth, reliable, and scalable data operations across multiple environments.

Key Responsibilities

Manage end-to-end data operations, including HDFS table management and data lifecycle governance

Design, build, and optimize ETL pipelines using PySpark and Python

Oversee multi-environment codebase governance, ensuring consistency and compliance across dev, test, and production environments

Implement and maintain DevOps automation using tools such as uDeploy and Jenkins

Lead platform upgrades, patching, and performance tuning for big data ecosystems

Provide production support, troubleshooting issues, and ensuring high availability of data services

Collaborate with data engineers, analysts, and business stakeholders to deliver reliable and scalable data solutions

Drive best practices in CI/CD, monitoring, and automation for big data platforms

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

About United Technology