Data Engineer

SENIOR Expert Data Engineer

Federal Reserve Bank of NY

3 days a. week onsite

Pyspark expertise, Scala, Databricks and AWS are a must have.

Top Skills' Details

Qualifications:
Expert-level understanding of Agile, SDLC, CI/CD, DevOps, GitOps, and ProdOps is required.
Expert-level expertise in SQL, Python, Scala or similar; other object-oriented programming experience a plus
Expert-level experience in data architecture, data types/formats, optimizing code and ETL jobs, data architecture, data modeling, data contracts a key requirement for successful collaboration with data architects and visualization developers.
Expert-level professional hands-on experience in extensible data pipeline development using Databricks, Alteryx, AWS Glue and Lambda functions.
Expert-level professional hands-on experience with RDBMS, ETL scripting tools and IDEs, data pipeline development, and management using orchestration/monitoring tools.
Expert-level knowledge and experience in implementing software engineering security best practices and contributing to design/engineering for DevSecOps.
Expert-level experience with master data management strategies, rules-based engines, and tools for multi-stage data correlation on large data sets.
Must have experience managing deployment and leading the release of code to production.
Expert-level understanding of data visualization tools and requirements for business customers.