Data Engineer (Billing & Compensation Data Pipelines)

Role                       : Data Engineer (Billing & Compensation Data Pipelines)

Location               : Remote

Role Summary
We are hiring a developer to build and maintain automated pipelines that extract revenue and expense data from internal systems and Snowflake, transform/reconcile that data, and feed calculation logic used in compensation and billing computations. This role owns orchestration via Apache Airflow and must ensure accuracy, auditability, and traceability of financial data flows.

Key Responsibilities

• Design and build ETL/ELT pipelines pulling revenue and expense data from on-premise source systems into Snowflake.
• Develop Airflow DAGs for scheduling, dependency management, retries, and monitoring of data pipeline jobs.
• Implement calculation logic across multiple data points, such as gross vs. net revenue and expense allocations, in SQL (Snowflake SQL/T-SQL) or Python.
• Perform data validation, reconciliation, and deduplication to ensure billing/revenue figures are accurate before downstream consumption.
• Debug and maintain existing SQL views, including identifying column mismatches, join logic errors, and typos.
• Optimize queries/views to avoid full-table scans and unfiltered CTEs that degrade production performance.
• Build monitoring/alerting for pipeline failures, data quality issues, and SLA breaches within Airflow.
• Document data lineage, transformation logic, and calculation rules to support audit and traceability requirements.
• Collaborate with technical leads and business stakeholders to translate revenue/expense business rules into pipeline logic.

Skills

• Minimum 5 years of relevant professional experience.
• Strong knowledge of Apache Airflow for job scheduling, orchestration, and DAG development.
• Strong knowledge of Snowflake SQL and/or T-SQL, including CTEs, window functions, date-based transformations, and multi-source joins.
• Experience with ETL/ELT development pulling data from on-premise systems into cloud data warehouses.
• Experience with Python for pipeline scripting, custom operators, and data transformation logic.
• Strong understanding of revenue and expense data structures.
• Experience in writing and executing unit/system test cases for data pipelines.
• Financial industry experience, preferably in investment management or billing/compensation systems.
• Ability to manage time independently and work independently.
• Experience with CI/CD pipelines, preferably Azure DevOps.
• Experience in an Agile/Scrum environment is a plus.
• Experience debugging SQL views for column mismatches, join errors, and performance issues is a plus.
• Experience using AI capabilities is a plus.