Job Title: Airflow DevOps Engineer
Optimize mission-critical data workflows as a senior Python/Airflow specialist designing, maintaining, and scaling Apache Airflow pipelines. Drive operational efficiency through dynamic DAG development, cluster administration, and application team support in a high-impact hybrid role.
Location: St. Louis, MO (3 days/week onsite required)
Experience Level: Senior (5+ years Airflow/Python)
Role Summary
Lead the design, optimization, and administration of Apache Airflow data orchestration platforms powering key business operations. Combine deep Python development expertise with Airflow mastery to build reliable, scalable workflows supporting strategic initiatives and day-to-day operations.
Key Responsibilities
Advanced Airflow Development
- Architect and optimize Apache Airflow workflows using dynamic DAG generation, custom operators, and advanced scheduling patterns.
- Develop production-grade Python code for DAGs, sensors, hooks, executors, and Airflow plugins/extensions.
- Identify and resolve workflow inefficiencies through performance profiling, query optimization, and resource scaling.
Cluster Administration & Support
- Manage Airflow cluster administration including CeleryExecutor, KubernetesExecutor, and multi-node deployments.
- Support application teams with DAG development best practices, troubleshooting, and optimization recommendations.
- Implement monitoring, alerting, and health checks for production Airflow environments.
DevOps & Automation
- Design CI/CD integration for Airflow DAG deployment using GitHub Actions, Jenkins, or GitLab CI.
- Implement infrastructure as code for Airflow environments (Docker Compose, Helm charts, Terraform).
- Automate DAG testing frameworks including unit tests, integration tests, and end-to-end validation.
Required Technical Expertise
Core Technology | Expertise Required |
Apache Airflow | Dynamic DAGs, Custom Operators, XComs, Sensors, Task Groups, Variables/Connections |
Python | Advanced development (DAG authoring, plugins, type hints, pytest, FastAPI) |
Executors | CeleryExecutor, KubernetesExecutor, LocalExecutor, scaling strategies |
DevOps | Docker, Kubernetes, Helm, Terraform, CI/CD pipelines |
Monitoring | Prometheus, Grafana, Airflow Metrics, Sentry, custom alerting |
Certifications (MANDATORY)
- CBAP Certified Business Analysis Professional
- Apache Airflow Certification (Astronomer or official)
Experience Profile
- 5+ years hands-on Airflow development (DAG authoring, optimization, cluster management)
- Advanced Python expertise (production-grade DAG development, custom operators)
- Proven Airflow cluster administration (multi-node, production environments)
- Experience supporting application teams (DAG reviews, troubleshooting, best practices)
- Hybrid work experience (3 days/week onsite capability)
Keywords: Airflow DevOps Engineer, Apache Airflow, Python Developer, DAG development, dynamic DAG generation, Airflow cluster administration, CeleryExecutor, KubernetesExecutor, custom operators, XComs, Airflow sensors, task groups, Airflow Variables, Airflow Connections, Python pytest, Airflow plugins, Docker Kubernetes Helm Terraform, CI/CD GitHub Actions Jenkins GitLab CI, Airflow monitoring Prometheus Grafana Sentry, workflow optimization, production Airflow, CBAP certification, Apache Airflow Certification, St. Louis MO, hybrid onsite, data orchestration, business process automation
About VDart Group
VDart Group is a global leader in technology, product, and talent solutions, serving Fortune 500 clients in 13 countries. With over 4,000 professionals worldwide, we deliver innovation, operational excellence, and measurable outcomes across industries. Guided by our commitment to People, Purpose, and Planet, VDart is recognized with an EcoVadis Bronze Medal and as a UN Global Compact member, reflecting our dedication to sustainable practices.
Never miss an opportunity! Create an alert based on the job you applied for.