Remote/ Providence, RI /Boston, MA
6 months CTH
Looking for a Senior Data Engineer with a winning track record in building high performing data and analytical solutions that transform, integrate and meet various data demands from business.
Our Senior Data Engineer will be responsible for the development of ETL process on AWS technologies. We are looking for a leader who can mentor other developers along with hands on experience to develop scripts to move data from AWS S3 to downstream systems. You will be responsible for building data products which involves architecture, design, build, test and support data integration projects.
Primary responsibilities include:
- Implementing data integration architecture and solutions.
- Conducting source system/data analysis and data profiling.
- Building robust data pipelines to collect process and compute different metrics from various financial sources, adhering to quality and development standards.
- Designing application architecture / technical design and articulating the data pipeline solutions to team members.
- Data mapping from source to target and define transformation rules that meet the models created by Data Architects. Collaborating with cross functional team members as necessary and coming up with optimal solutions that meet data demands.
- Executing unit test of data populated in target data container, validating expected result and ensuring quality and accuracy. Coordinating with business users for User Acceptance Testing and with Operations team for code deployment to upper environments
- Following change management team stipulations on path to production requirements and strictly adhering to the compliance and regulatory needs.
- Expert in AWS technologies
- Proficient with ETL Development tools like Talend, Data Stage
- Excellent skills in SQL Development
- Python and other scripting language expert
- Ability to work on AWS Glue, AWS S3, Lake formation and other cloud concepts.
IDEAL Candidate will have a combo of: Talend, Redshift, AWS.
- Moving from Datastage to Talend. (Someone with both is ideal)
- They use Redshift on AWS (Experience with this is helpful)
- Extracting data from data lake.
- Diverse experience with Talend
- Python experience is very helpful
Education, Certifications, and/or Other Professional Credentials:
- BS/Engineering in Computer Science or any technology related degree