Core Skills
Python: Modular and reusable application development, configuration-driven design, robust exception handling, logging, debugging, data processing, and production troubleshooting.
BigQuery SQL: Complex joins, CTEs, window functions, MERGE operations, advanced deduplication patterns, incremental data processing, performance tuning, and query optimization.
Airflow / Cloud Composer: Production DAG development, task orchestration and dependencies, XCom, dynamic/runtime configuration, retry strategies, failure handling, backfills, reruns, and production troubleshooting.
Google Cloud Platform: BigQuery, Cloud Storage, IAM and service accounts, Pub/Sub integrations, permissions management, and cloud-based data pipeline operations.
Testing & Delivery: pytest, unit and integration testing, mocking, Git workflows, CI/CD pipelines, deployment validation, and E1 E2 E3 environment promotion.
Develop During Onboarding
Lumi / ERIRP Framework: JSON-driven pipeline architecture, runtime configuration, reusable hooks, orchestration patterns, safety gates, validation controls, and operational notifications.
Data Reliability & Financial Processing: Idempotent reruns, late-arriving data handling, deduplication strategies, audit and reconciliation checks, data-quality controls, and financial precision/rounding.
Never miss an opportunity! Create an alert based on the job you applied for.
