Big Data/Machine Learning Engineer - Sr

Project Overview: goal is to basically bring transaction data from various LOB and make those data available for AML use cases. Use cases involve bunch of customers like Maddie, who runs model to identify any transaction which seems like fraudulent. They also have customer like CTR Scout, which is responsible for meeting the regulatory requirement of anybody transacting more than $10,000 in cash for any given day, and reporting the same. So they have use cases downstream & building data pipeline to bring all these transaction data for all those use cases.

 Then they have Discover personal loan project too, build data pipeline to bring the personal loan into their ecosystem so that all the AML use cases can be supported and subtracted data.

Building data pipeline using spark, Python

Data engineering experience – ideally 5+ years; HM said 5 is the approximate cutoff and 5–10 years is optimal. Not looking for very senior resources, but very hands on DE.

AWS, SQL.

Airflow – orchestration experience.

Data publishing happens in delta lake, snowflake

Databricks, snowflake, database knowledge.

Experience troubleshooting pipeline issues and handling vulnerabilities/ETB/production issues.

DPL role – would be only data pipeline and not majorly API.

Nexus is larger project and DPL is part of that.

Prior experience with Data, not only building pipeline but also supporting pipeline.

Data analysis, production support will be helpful, comes with data engineering experience.

Ex cap one is not a preference; manager’s top priority is skill and talent. If submitting with capital one’s feedback would be helpful.

API development — nice to have, not required.

No ML required.