Data Engineer with DevOps

Farmers Branch, TX, US • Posted 13 hours ago • Updated 13 hours ago
Contract W2
100 Weeks
On-site
$120000/yr
Company Branding Image
Fitment

Dice Job Match Score™

🧠 Analyzing your skills...

Job Details

Skills

Summary

Job Title: Senior Data Engineer with DevOps

Duration : Full Time
Location : Pittsburgh, PA, Cleveland, OH, or Dallas, TX

Work Mode : 5 Days Onsite
Years Of Exp : 8+ Yrs

Job Description:
We are seeking a Data Engineer with 5 years of experience to design and maintain scalable data pipeline supporting analytics, reporting, and operational needs. The role involves collaborating with cross functional teams to ensure data alignment with business requirements and enterprise standards.

Duties and Responsibilities:

  • Design and build scalable data pipelines aligned with business needs
  • Process large dataset (batch + sometimes near Realtime)
  • Ensure data quality, consistency, and governance standards across systems
  • Support data integration and transformation efforts for analytics and reporting platforms
  • Maintain data dictionaries, metadata, and documentation
  • Participate in data architecture reviews and model validation processes
  • Support analytics reporting and risk platforms.

Required Qualifications
  • 5+ years of experience in data engineering and big data processing
  • Strong expertise in Apache Spark (Spark Core, Spark SQL) and PySpark for large scale batch processing
  • Experience working with structured and semi structured data, including complex transformations and performance tuning
  • Proficiency in data ingestion and integration from sources like Oracle, SQL Server, Hive, HDFS, and S3; transform data into ‘curated data models''
  • Experience writing data to Hive tables, Data Lakes (Iceberg), and downstream reporting systems
  • Strong knowledge of SQL and data modeling concepts
  • Hands on experience with Apache Airflow for workflow orchestration (DAG design, scheduling expectations, monitoring)
  • Proficiency in shell scripting for job automation, file validation, dependency handling, and logging. Trigger Spark Jobs, perform file checks and validation; Archive & purge data; mange job dependency, logging & error handling
  • Strong understanding of batch processing and batch job scheduling frameworks
  • Experience migrating from CA7/Control M Airflow (daily, hourly, weekly schedules) CI/CD for data pipelines
  • Fundamentals in Linux and Networking
  • Docker, OCP containerization / Kubernetes
 
  • Knowledge of CI/CD pipeline tools: Tools commonly include Jenkins, GitHub Actions, Azure DevOps, GitLab Cl, Maven, and Gradle
  • Automate operational tasks using Python, Bash/Shell, and PowerShell
  • Implement monitoring and alerting, Application Insights. Enable centralized logging with tools such as ELK.
  • Experience ensuring data quality, reliability, and compliance in regulated environments
  • Good communication and documentation skillsGood communication and collaboration skills in cross-functional teams
  • Agile/Safe methodologies
Must Have Skills
  • Airflow
  • Containerization
  • DevOps
  • Elastic Stack & Elasticsearch
  • GitHub
  • Hadoop Hive
  • Jenkins
  • JSON Web Token (JWT)
  • Kubernetes
  • OpenShift
  • Oracle
  • Python
  • Shell Script
  • SQLite


Ref: #404-IT Pittsburgh
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 10295162
  • Position Id: 376113
  • Posted 13 hours ago

Company Info

About System One

System One is a leading provider of specialized, highly technical outsourced services, workforce solutions, staffing, and recruiting to critical infrastructure, technology, life sciences, and government sectors As an essential partner to private and public organizations of various needs and sizes, we offer our expertise to help them bring their most complex and mission-critical programs to fruition.

With our highly specialized services, in-demand technical skills, and vast operational expertise, we can deliver accelerated results that benefit our clients and the job seekers who partner with us. We staff contract, contract-to-hire, and direct-hire professionals across our network to work with our clients, some of whom are industry leaders and nationally branded organizations.

Our dedicated team of recruiters work tirelessly to match jobseekers with careers that align with their skills, aspirations, and goals.

We understand that finding the right job can be a transformative experience, and we are committed to supporting jobseekers throughout their career journeys. Our recruitment services are designed to connect individuals with fulfilling jobs that offer growth and development opportunities.

Whether you're an organization seeking customized workforce solutions, outsourced services, or if you're a jobseeker searching for your next opportunity,

System One is the perfect staffing partner for you.

About_Company_One
Contact the job poster
Shafique Mohammed

Shafique Mohammed

Recruiting Manager @ System One
Create job alert
Set job alertNever miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Hybrid in Plano, Texas

4d ago

Easy Apply

Contract, Third Party

$42

Remote

14d ago

Easy Apply

Contract, Third Party

$46.11

Remote

6d ago

Easy Apply

Contract, Third Party

Depends on Experience

Remote

14d ago

Easy Apply

Contract, Third Party

$48.95

Search all similar jobs