Senior ETL Pipeline Engineer (Docker & Kubernetes)

Overview

On Site
Depends on Experience
Contract - Independent
Contract - W2
Contract - 6 Month(s)

Skills

ETL
Pipeline
Docker
Kubernetes
AWS

Job Details

Senior ETL Pipeline Engineer (Docker & Kubernetes)
Location: Dallas, TX (Onsite role)
Duration: 6+ Months contract

Interview process:

1 video

2 onsite


Overview:
  • We are looking for a highly skilled ETL Pipeline Engineer to design, develop, and manage ETL (Extract, Transform, Load) pipelines using Docker and Kubernetes.
  • The ideal candidate will have experience with containerization and orchestration technologies, ensuring efficient and scalable data workflows for our organization.
Key Responsibilities:
  • Build and manage ETL pipelines to process and transform large-scale datasets.
  • Containerize ETL workflows using Docker for modular and portable data processing.
  • Deploy, manage, and scale ETL pipelines in Kubernetes clusters.
  • Collaborate with data engineering teams to integrate ETL pipelines with the broader data architecture.
  • Monitor pipeline performance and troubleshoot issues to ensure reliability and high availability.
  • Automate pipeline deployment and updates using CI/CD tools.
  • Optimize data workflows to meet performance requirements and reduce latency.
  • Document pipeline designs, configurations, and best practices.
Requirements:
  • Strong experience with ETL tools and processes.
  • Proficiency in containerization using Docker.
  • Hands-on experience with Kubernetes for deploying and managing applications.
  • Knowledge of CI/CD tools and practices.
  • Familiarity with cloud platforms such as AWS, Azure, or Google Cloud.
  • Programming skills in Python, Java, or other scripting languages used in ETL workflows.
  • Understanding of database technologies (e.g., SQL, NoSQL) and data modeling concepts.
  • Strong problem-solving skills and attention to detail.
Preferred Qualifications:
  • Experience with Kubernetes operators for data processing.
  • Familiarity with distributed data processing frameworks like Apache Spark or Hadoop.
  • Cloud certifications in container orchestration or data engineering.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.