Data Engineer (Spark)


Depends on Experience
Full Time


Data Science
Machine Learning

Job Details

Job Tittle: Data Engineer
Duration: Full-time
Location: Mclean, VA/TX/FL/NJ (Hybrid 3 days onsite, Rest days remote)

Must Have: Spark

Your Primary Responsibilities: Work with large, complex data sets and high throughput data pipelines that meet business requirements. Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources. Build data and analytics tools that utilize the data pipeline to provide actionable insights to operational efficiency and other key business performance metrics. Work with internal and external stakeholders to assist with data-related technical issues and support data infrastructure needs. Collaborate with data scientists and architects on several projects. Solve various complex problems. Designing processes to integrate data from multiple sources to facilitate client centric advanced analytics Developing efficient, scalable and repeatable processes to transform data into insight on regular basis

Talent Needed for Success: Degree in an analytical field such as Data Science, Machine Learning, Analytics, Statistics, Computer Science, or highly quantitative engineering Previous experience as a data engineer or in a similar role 5+ years of Python development experience is necessary. Hands-on experience with database technologies (e.g. SQL and noSQL) Experience building high throughput data pipelines. Technical expertise with distributed Spark or other distributed data processing technologies Experience with machine learning techniques Great numerical and analytical skills Ability to write reusable code components. Open-minded to the new technologies, frameworks Thorough business analysis skills

Additional Qualifications Understanding Blockchain system mechanism Knowledge of data preparation techniques to aid statistical analysis Results-oriented self-starter who is confident in defending his/her critical thinking abilities.

About The Depository Trust & Clearing Corporation