Data Engineering Manager

AWS, EC2, EMR, RedShift, Data Pipeline, Airflow, S3, Cloud Formation, CLI, Jenkins, Spark, Hadoop, Presto
Full Time
$40,000 - $140,000
Work from home not available Travel not required

Job Description

Data Engineering Manager


Location: Sterling, VA


Duration: Full-Time


Key Areas of Responsibility:


  • Proactively drive the vision for data lake across a product vertical and define and execute on a plan to achieve that vision.
  • Lead the design, implementation, and continuous delivery of pipelines using distributed AWS based big data technologies supporting data processing initiatives across batch and streaming datasets
  • Identify, evaluate and implement cutting edge big data pipelines and frameworks required to provide requested capabilities to integrate external data sources and APIs

  • Review, analyse and evaluate market requirements, business requirements and project briefs in order to design the most appropriate end-to-end technology solutions

  • Manage and lead the team of 5-8 data engineers
  • Operate in agile environment to meet the department objective
  • Established relationship with key business lines, Infosec and infrastructure team
  • Provide architectural support by building Proof of Concepts & Prototypes
  • Self-Starter to deliver data engineering solutions to optimize both the cost and existing solution
  • Stay current with emerging technologies and industry trends
  • Fostering a culture of continuous improvement
  • Identifying infrastructure and architectural investment needed.

Required Qualifications


  • Required management experience of atleast 2-3 years, leading or managing.
  • Minimum of 7-8 years Software Industry experience.
  • 3+ years of development experience with AWS services Must have EC2, EMR, RedShift, Data Pipeline or Airflow, S3, Cloud Formation, CLI and Jenkins.
  • 3+ years of development experience with big data platform like Spark, Hadoop, Presto etc
  • Proficiency working with structured, semi-structured and unstructured data sets including social, web logs and real time streaming data feeds
  • Able to tune Big Data solutions to improve performance and end-user experience
  • Knowledge on Visualization and Data Science Tools.
  • Ability and eagerness to constantly learn and teach others
  • Experience scaling and managing 5+ person teams.
  • Communication and leadership experience, with experience initiating and driving projects.
  • Project management experience.
  • Data architecture experience.
  • Experience in SQL or similar languages and development experience in at least one scripting language (Python, Perl, etc.)
  • BA/BS in Computer Science, Math, Physics, or another technical field.
Dice Id : 90979346
Position Id : 5989383
Have a Job? Post it

Similar Positions

Data Engineer with Python, SQL, Presto
  • The Consortium Inc
  • Arlington, VA
Data Scientist
  • CyberData Technologies
  • Washington, D.c., DC
Senior Data Engineer
  • Capital One
  • Richmond, VA
Senior Data Engineer
  • Blue Orange Digital
  • Alexandria, VA
Big Data
  • Reliable Software Resources
  • Bethesda, MD
Data Engineer
  • Robert Half Technology
  • Bethesda, MD
Sr. Data Engineer
  • Capital Markets Placement
  • Mclean, VA
Senior Data Configuration & Management Specialist
  • Modern Technology Solutions
  • Alexandria, VA
Sr. Datawarehouse Developer
  • Open Systems Technologies
  • Chevy Chase, MD
Hadoop Data Engineer
  • Metro Systems Inc
  • West Mclean, VA
Data Architect with Secret Clearance
  • Applied Information Sciences
  • Alexandria, VA
Data Scientist
  • IT Resonance Inc.
  • Alexandria, VA
Sr. Big Data Engineer
  • Idexcel Inc.
  • Mclean, VA