Join a dynamic enterprise technology team as a Senior Data Orchestration Engineer, where you will help modernize workload management by migrating legacy scheduling from AutoSys to Apache Airflow. This role is ideal for someone who enjoys building reliable, scalable orchestration solutions that improve automation, visibility, and consistency across data and application workflows.
Responsibilities
- Lead the migration of scheduled enterprise workloads from AutoSys to Apache Airflow, including discovery, dependency analysis, conversion, testing, and production cutover.
- Design, build, and maintain Airflow DAGs that support data pipelines and business-critical workflows.
- Review legacy scheduling patterns and identify ways to simplify, consolidate, or retire outdated jobs.
- Coordinate orchestration across Databricks, Microsoft Fabric, cloud services, APIs, and enterprise systems.
- Define practical standards for scheduling, dependencies, retries, error handling, monitoring, and recovery procedures.
- Support CI/CD, version control, and infrastructure-as-code practices for orchestration development and deployment.
- Partner with engineering, application, and platform teams to ensure a smooth migration with minimal disruption to operations.
- Help improve reliability, observability, and operational consistency for enterprise workflows.
Skills
- 5+ years of experience in data engineering, platform engineering, or enterprise workload orchestration.
- Hands-on experience with AutoSys and Apache Airflow, including migrating legacy workloads to modern orchestration platforms.
- Strong Airflow knowledge, including DAG development, scheduling, operators, sensors, and production support.
- Proficiency in Python, SQL, and REST APIs.
- Experience working with cloud platforms, preferably Azure or AWS.
- Exposure to Databricks and modern data architecture patterns.
- Working knowledge of CI/CD, containerization, observability, and support for production environments.
- Ability to collaborate effectively across technical teams and communicate clearly.
Preferred Skills
- Experience simplifying complex legacy job schedules and improving orchestration standards.
- Familiarity with enterprise application integration and cross-platform workflow coordination.
- Experience establishing reusable orchestration patterns that can be applied across multiple teams.
- Comfort working in fast-moving environments with a focus on operational stability and continuous improvement.
Horizontal is committed to fostering a welcoming, inclusive environment where diverse perspectives are valued and everyone has the opportunity to contribute and grow. We encourage candidates from all backgrounds to apply and join us in building equitable, innovative solutions.
By applying for this position, you acknowledge and agree that Horizontal Talent may contact you regarding your application using automated technology, including phone calls, SMS/text messages, or email, which may be delivered by our virtual AI recruiter, Alex.
Please apply through this online posting or by visiting our Job Board at Applications will be accepted for 4 weeks. For those that join the team, we offer competitive compensation and benefits including medical, dental, vision, and retirement. Check out all we have to offer and how you can become part of the Horizontal Talent Team. The pay range for this role is $62 - $100 per hour based on qualifications and experience.
