Job Description:
Required Core Skills:
Python Engineering: 5+ years of hands-on experience building modular, reusable, production-grade Python applications, data pipelines, and processing frameworks.
Advanced BigQuery & SQL: Advanced proficiency in BigQuery SQL, including complex multi-table joins, analytical window functions, CTEs, incremental MERGE statements, and query execution plan tuning.
Airflow / Google Cloud Platform Cloud Composer: Deep expertise in developing production Airflow DAGs, runtime/dynamic configurations, task dependencies, failure handling, and production backfill management.
Google Cloud Platform Infrastructure: Strong operational experience across Google Cloud Platform core services: BigQuery, Cloud Storage (GCS), Pub/Sub messaging, and IAM role management.
Testing & Release Management: Proven experience with pytest, unit testing, mocking, Git version control, CI/CD pipeline integration, and automated deployment validation across multi-tier enterprise environments.
Preferred:
Lumi / ERIRP Framework: JSON-driven pipeline architecture, runtime configuration, reusable hooks, orchestration patterns, safety gates, validation controls, and operational notifications.
Data Reliability & Financial Processing: Idempotent reruns, late-arriving data handling, deduplication strategies, audit and reconciliation checks, data-quality controls, and financial precision/rounding.
