Data Engineering Manager
Location: Sterling, VA
The Data Engineering Manager should have strong data infrastructure and data architecture skills, a proven track record of leading and scaling data engineering teams, strong operational skills to drive efficiency and speed, strong project management leadership, and a strong vision for how analytics can proactively improve organizations. You will be part of the Global Data & Analytics engineering technology team and will partner closely with a team of data scientists, business analysts & data engineers leading cloud based Big Data & Analytics strategy.
You'll work on implementing complex AWS based big data projects with a focus on collecting, parsing, managing, analyzing and visualizing large sets of data to turn information into insights using multiple technology platforms. Therefore, this role requires an understanding of how a secure big data cloud environment is architected to gain real insights faster, with less friction and complexity. Ideal candidate should be passionate about working with cutting edge technologies in solving problems and Leading a team of 5-8 data engineers.
You'll need to be an innovative forward-thinker who will help lead end-to-end execution of data engineering initiatives and contribute directly to existing and emerging business strategies and goals. Creativity, Attention to detail and ability to work in a collaborative team environment are essential. The Data engineering Manager will work closely with the Director of Data Engineering to decide on needed infrastructure architecture and software design needs and act according to the decisions.
Key Areas of Responsibility:
Proactively drive the vision for data lake across a product vertical and define and execute on a plan to achieve that vision.
Lead the design, implementation, and continuous delivery of pipelines using distributed AWS based big data technologies supporting data processing initiatives across batch and streaming datasets
Identify, evaluate and implement cutting edge big data pipelines and frameworks required to provide requested capabilities to integrate external data sources and APIs
Review, analyse and evaluate market requirements, business requirements and project briefs in order to design the most appropriate end-to-end technology solutions
Manage and lead the team of 5-8 data engineers
Operate in agile environment to meet the department objective
Established relationship with key business lines, Infosec and infrastructure team
Provide architectural support by building Proof of Concepts & Prototypes
Self-Starter to deliver data engineering solutions to optimize both the cost and existing solution
Stay current with emerging technologies and industry trends
Fostering a culture of continuous improvement
Identifying infrastructure and architectural investment needed.
Minimum of 7-8 years Software Industry experience.
3+ years of development experience with AWS services Must have EC2, EMR, RedShift, Data Pipeline or Airflow, S3, Cloud Formation, CLI and Jenkins.
3+ years of development experience with big data platform like Spark, Hadoop, Presto etc
Proficiency working with structured, semi-structured and unstructured data sets including social, web logs and real time streaming data feeds
Able to tune Big Data solutions to improve performance and end-user experience
Knowledge on Visualization and Data Science Tools.
Ability and eagerness to constantly learn and teach others
Experience scaling and managing 5+ person teams.
Communication and leadership experience, with experience initiating and driving projects.
Project management experience.
Data architecture experience.
Experience in SQL or similar languages and development experience in at least one scripting language (Python, Perl, etc.)
BA/BS in Computer Science, Math, Physics, or another technical field.