ETL Engineer

emr, ec2, s3, redshift, rds, lambda, hadoop platform: hdfs, hive, map reduce, sqoop., writing complex SQLs for data integration and manipulation
Full Time
Depends on Experience
Work from home not available Travel not required

Job Description

Position Title: ETL Engineer

Location: Columbia, MD/ Fairmont, WV

Employment Type: Full Time

Responsibilities:

  • Deliver ETL solutions from conception to design, development, and release.
  • Develop ETL processes and Spark jobs for Analytical data loads, XML ingest, flattening, transformation and data load utilities.
  • Establish processes for data reconciliation with source systems.
  • Schedule ETL processes, tune ETL for performance, and resolve ETL production issues.
  • Develop HIVE, REDSHIFT DDLs and DMLs.
  • Collaborate with Data architects.
  • Develop automated test scripts.
  • Work in an agile environment on incremental delivery of ETL products.
  • Participate in Daily Scrum calls as well as spring planning sessions to support development efforts.
  • Effectively use Jira to estimate & track user stories and tasks.

Skill Set:

  • 3-5 years of working experience in ETL development on Hadoop platform: HDFS, HIVE, Map Reduce, Sqoop.
  • 3-5 years of functional programming knowledge preferably with Scala/Spark or Java or Python or R.
  • Experience in AWS services such as EMR, EC2, S3, Redshift, RDS, Lambda.
  • Good Experience writing complex SQLs for data integration and manipulation.
  • Work independently with minimum supervision.
  • Communicates & collaborates effectively with the team.
  • Good Experience on shell and bash scripting.

Company Website: www.sparksoftcorp.com

Founded in 2004, Sparksoft is a Small Business (WOSB), Small Disadvantaged Business (SDB), CMMI Level 3, ISO 9001:2015, ISO 27001:2013, HUBZone, 8(a), Women-Owned, and Minority-owned business (SWaM), MBE/DBE/SBE holding prime contracts with SPARC, GSA and Navy Seaport-e.Lead by a team with over 20 years of industry experience, we have a proven track record in software development and managing multi-year projects. We work in both federal and commercial spaces, providing innovative solutions in Agile Software Testing, Software Engineering, Program Management, Security & Privacy Solutions, Delivery Management and Health IT. We have dedicated Centers of Excellence in software testing, independent verification and validation, and security and privacy, that are designed to fulfill the IT needs of any commercial or government field.

Sparksoft is an Affirmative Action/Equal Opportunity Employer and does not discriminate against any applicant for employment or employee because of race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, protected veteran status, or any other characteristic prohibited under Federal, State, or local laws.

Dice Id : 10197954
Position Id : 001 ETL
Originally Posted : 1 month ago
Have a Job? Post it