DevOps Engineer

Key Responsibilities

• Deploy, configure, and manage Apache Airflow environments (Scheduler, Webserver, Workers, Metadata DB).

• Design, implement, and optimize Airflow DAGs for data pipelines and workflow orchestration.

• Manage Airflow in cloud and containerized environments (Kubernetes/Docker).

• Implement CI/CD pipelines for Airflow deployments and infrastructure automation.

• Monitor system performance, ensure high availability, and implement logging & alerting solutions.

• Troubleshoot production issues and perform root cause analysis.

• Collaborate with Data Engineering, Platform, and Application teams to support pipeline reliability.

• Implement infrastructure as code (IaC) using tools like Terraform or CloudFormation.

• Ensure security best practices, role-based access, and secrets management.

Required Skills & Qualifications

• 4 8 years of DevOps / Platform Engineering experience.

• Strong hands-on expertise with Apache Airflow (production deployments).

• Experience with containerization tools (Docker) and orchestration platforms (Kubernetes).

• Strong knowledge of CI/CD tools (Jenkins, GitHub Actions, GitLab CI, etc.).

• Experience with cloud platforms (AWS / Azure / Google Cloud Platform).

• Familiarity with monitoring tools (Prometheus, Grafana, ELK stack, etc.).

• Strong scripting skills (Python, Bash).

• Experience with infrastructure automation (Terraform preferred).

• Solid understanding of Linux systems and networking fundamentals.

Preferred Qualifications

• Experience managing Airflow in Kubernetes (Helm charts, autoscaling).

• Knowledge of data engineering concepts and ETL pipelines.

• Exposure to distributed systems and high-availability architectures.

• Certification in AWS/Azure/Google Cloud Platform (optional but preferred).