Data Engineer

Data Engineer – PySpark, Snowflake & AWS 
Location – Mclean, VA 

About the Role 
We’re looking for a hands-on, delivery-focused Data Engineer to help build and scale our cloud data platform. You’ll design and develop modern data pipelines using PySpark, Snowflake, and AWS to optimize cloud data workloads. 
This is a contractor, hands-on role requiring the candidate to be local to McLean, VA and available for on-site engagement. 
The ideal candidate combines strong engineering fundamentals with cloud-native data expertise — able to translate business needs into robust, performant, and well-documented data solutions. Experience with Fannie Mae, Freddie Mac, or equivalent GSE/mortgage enterprise environments is strongly preferred. 
What You’ll Do 
Design & Build Scalable Data Pipelines using PySpark for large-scale batch and streaming data processing. 
Develop PySpark Solutions — write production-grade PySpark code to read data from S3 (Parquet/Delta files), perform complex transformations, and process large-scale datasets efficiently. 
Implement Deduplication Logic — design and implement robust deduplication strategies for large datasets using PySpark. 
Performance Tuning (PySpark) — optimize PySpark jobs for reading and processing very large datasets, including partitioning, caching, broadcast joins, and shuffle optimization. 
Develop Cloud-Native Data Solutions on AWS — leveraging services like S3, Glue, EMR, Lambda, Step Functions, and Redshift. 
Engineer Snowflake Data Platforms — build warehouses, schemas, and data models optimized for analytics and reporting. 
Build Snowflake Iceberg Tables — design and implement Apache Iceberg tables in Snowflake for open lakehouse architectures and interoperability. 
Develop Dynamic Tables & Materialized Views — build and maintain Snowflake Dynamic Tables and Materialized Views to support near real-time analytics and query acceleration. 
Snowflake Performance Tuning — optimize Snowflake workloads through clustering, micro-partition pruning, query profiling, warehouse sizing, result caching, and materialization strategies. 
Modernize Legacy ETL — migrate on-prem ETL workloads (e.g., DataStage, Informatica) to PySpark and Snowflake-based cloud pipelines. 
Optimize Performance — tune Spark jobs, Snowflake queries, and AWS resource utilization for speed and cost. 
Ensure Data Quality & Governance — implement validation, lineage, and monitoring across every pipeline. 
Collaborate Across Teams — partner with data architects, analysts, TPMs, and business stakeholders to deliver trustworthy data products. 
Document Everything — from technical designs to runbooks, ensuring every pipeline is maintainable and audit-ready. 
What We’re Looking For 
Must-Haves 
6+ years of hands-on data engineering experience in enterprise environments. 
Strong expertise in PySpark — building distributed data processing pipelines at scale. 
Hands-on experience writing PySpark code to read data from S3 Parquet/Delta files, perform transformations, and handle large datasets. 
Proven experience in performance tuning of PySpark jobs when reading and processing very large datasets. 
Experience implementing deduplication logic in PySpark for high-volume data pipelines. 
Deep hands-on experience with Snowflake — data modeling, SnowSQL, Snowpipe, Streams, Tasks, and role-based access. 
Hands-on experience creating Snowflake Iceberg Tables for open table format and lakehouse use cases. 
Experience building Snowflake Dynamic Tables and Materialized Views for incremental data processing and query acceleration. 
Strong Snowflake performance tuning skills — clustering keys, micro-partition pruning, query profiling, warehouse right-sizing, caching strategies, and cost optimization. 
Proven AWS experience — S3, Glue, EMR, Lambda, IAM, Step Functions, CloudWatch, and Redshift. 
Advanced SQL skills — complex joins, subqueries, window functions, CTEs, and performance tuning. 
Strong Python programming skills beyond PySpark — for utilities, automation, and orchestration. 
Experience with orchestration tools — Airflow, AWS Step Functions, or equivalent. 
Solid understanding of data warehousing, ELT/ETL patterns, data lakes, and lakehouse architectures. 
Excellent communication skills — able to articulate technical decisions to both engineers and business stakeholders. 
Nice to Have 
Experience with IBM DataStage or other legacy ETL tools (for modernization contexts). 
Familiarity with CI/CD for data pipelines (Git, Jenkins, GitHub Actions, Terraform). 
Exposure to data quality frameworks (Great Expectations, dbt tests). 
Knowledge of streaming platforms (Kafka, Kinesis). 
Experience in regulated environments — financial services, mortgage, or GSE programs (Fannie Mae / Freddie Mac). 
Certifications: AWS Certified Data Analytics / Solutions Architect, SnowPro Core/Advanced. 
Your Toolkit 
Category 
Tools & Skills 
Big Data 
PySpark, Spark SQL, Databricks (plus) 
Cloud Data Warehouse 
Snowflake (SnowSQL, Snowpipe, Streams, Tasks, Iceberg Tables, Dynamic Tables, Materialized Views) 
AWS Services 
S3, Glue, EMR, Lambda, Step Functions, Redshift, IAM, CloudWatch 
Storage Formats 
Parquet, Delta Lake, Apache Iceberg 
Programming 
Python, SQL, Shell Scripting 
Orchestration 
Airflow, Step Functions, Control-M 
DevOps 
Git, Jenkins, Terraform, CI/CD pipelines 
Legacy ETL (plus) 
IBM DataStage, Informatica 
Governance 
Data lineage, quality frameworks, audit readiness