Data Integration Lead - Retail Wealth Management

Data Integration Lead (Streaming & Warehousing) - Retail Wealth

Location: Remote

Type: 1 year contract role

Role

The role is a hands-on Data Engineering Lead to build streaming and batch data pipelines powering our digital self service roadmap. Design robust ETL/ELT on AWS, Kafka, Databricks, Oracle, and guide the transition to Snowflake. The role blends architecture and delivery-owning data quality, observability, and governance in the Retail Wealth domain.

• Domain Expertise: Deep experience in retail wealth, broker dealers, warehouses, family offices, custodians, trade order management, and digital self-service platforms (e.g., robo advisors).
• Technical Stack: AWS, Kafka, Databricks, Oracle, Snowflake (future), ETL processes.
• Hands-On Leadership: Integration lead must design/develop APIs and coordinate with architects; data lead must be strong in ETL/data integration.

Responsibilities

• Design and build streaming & batch pipelines for ingestion, curation, and consumption (real time + micro batch).
• Engineer scalable ELT/ETL on Databricks (PySpark/Spark SQL), integrating sources including APEX, custodians, broker dealers, and market/reference data.
• Optimize workloads on AWS (S3, Glue, EMR/Databricks, Lakehouse patterns); manage Oracle sources; drive Snowflake migration strategy and execution.
• Enforce data quality (DQ), lineage, metadata, and governance with best practice frameworks and tooling.
• Partner with analytics, product, and integration teams to support dashboards, operational reporting, advanced analytics, and ODS.
• Establish DevSecOps for data (versioned transformations, CI/CD, IaC patterns, secrets mgmt).
• Define and track SLAs/SLOs, cost controls, and performance baselines.

Must-Have Qualifications

• 5+ years (7+ preferred) in data engineering with lead-level ownership delivering production pipelines.
• Retail Wealth expertise: custodians, broker dealers, warehouses/family offices; order/trade and position/transaction data.
• Hands-on with Kafka (topics, partitions, schema/registry), Databricks (PySpark/Spark SQL), AWS data stack, and Oracle sources.
• Strong SQL/performance tuning; ELT/ETL design patterns; batch orchestration (e.g., Airflow/Databricks Jobs).
• Practical data governance: lineage, DQ, PII controls, encryption, RBAC, and regulatory awareness (FINRA/SEC).
• Experience planning/executing Snowflake migrations (data modeling, performance, cost/pricing levers).
  
  
  Thanks & regards,
  Rohit Gupta
  VBeyond Corporation
  
  Note VBeyond is fully committed to Diversity and Equal Employment Opportunity.