Job Title: Senior Snowflake / Python Data Engineer (Hadoop Migration Lead)
Experience Level: 8–15+ Years
Location: Face-to-Face / Hybrid (As required by client standards)
Duration: Long-Term Contract
In-person Interview
Position Summary
client is seeking a seasoned, hands-on Senior Snowflake & Python Data Engineer with 8–15+ years of enterprise data engineering experience to lead our large-scale cloud modernization initiative. The core objective of this role is to architect, design, and execute the end-to-end migration of legacy Hadoop data pipelines, Hive tables, and HDFS storage into a modern, high-performance Snowflake Data Cloud.
The ideal candidate will combine deep technical expertise in Python, Spark/Hive, advanced Snowflake architecture, and data migration strategies with strong financial services domain knowledge and elite communication skills for face-to-face client interactions.
Key Responsibilities
Hadoop-to-Snowflake Migration: Lead the migration of legacy big data workloads (HDFS, Hive, Spark, MapReduce) to Snowflake, translating legacy scripts, queries, and orchestration flows into optimized Snowflake-native structures.
Python & Cloud Data Engineering: Design and develop scalable automated ELT/ETL data pipelines using Python, SQL, and modern cloud orchestration frameworks.
Snowflake Architecture & Optimization: Leverage advanced Snowflake features including Snowpipe, Streams, Tasks, Multi-Cluster Warehouses, Zero-Copy Cloning, and Time Travel to maximize performance and minimize compute costs.
Performance Tuning: Optimize complex analytical SQL queries, tune data warehouse storage, and re-architect batch data flows to handle massive financial datasets efficiently.
Data Quality & Governance: Implement robust data validation frameworks, access controls (RBAC), masking policies, and data lineage tracking during and post-migration.
Stakeholder & Team Collaboration: Work closely with enterprise architects, data governance teams, and business stakeholders at Morgan Stanley to define migration standards, resolve technical bottlenecks, and provide technical leadership to junior engineers.
Required Technical Qualifications
8–15+ years of total IT experience, with at least 5+ years dedicated to modern cloud data warehousing and big data ecosystems.
Proven, hands-on experience leading Hadoop-to-Snowflake migrations (migrating Hive, HDFS, HBase, or Spark environments into Snowflake).
Expert-level proficiency in Python and SQL for data processing, scripting, and pipeline automation.
Deep technical expertise in Snowflake cloud-native architecture (Snowpipe, Streams, Tasks, Secure Data Sharing, Resource Monitors, and performance tuning).
Solid background in big data frameworks including Apache Spark, Hive, or Kafka.
Experience with CI/CD deployment pipelines (GitHub Actions, Jenkins, Terraform, or similar infrastructure-as-code tools).
Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related quantitative field.
Preferred Qualifications / Soft Skills
Financial Services Industry Experience: Prior experience working within investment banking, wealth management, or financial services environments (e.g., Morgan Stanley, similar tier-1 banks).
Client-Facing Presence: Outstanding verbal and written communication skills, with a polished professional demeanor suited for face-to-face technical discussions and executive walkthroughs.
Certifications: Snowflake SnowPro Core or Advanced Architect certification is a strong plus.
