Senior Data Engineer

Title: Senior Data Engineer

Location: O'Fallon, MO 63368

Duration: 24 Months

Need Locals to MO Only

Client: Mastercard

Exp: 10+ Years

Ex Mastercard candidate is highly preferred

Top 3 required technical skills:

1. Strong knowledge of Hadoop platforms and other distributed data processing technologies: Proven experience delivering mission critical Big Data projects while collaborating effectively with diverse, highly experienced teams across enterprise operational functions.
2. Linux/Unix and System Administration: Advanced Linux expertise is required, including strong proficiency with shell scripting and system troubleshooting. The candidate must be comfortable navigating Linux environments and resolving issues independently.
3. Programming: Proficiency in at least one programming language is required, with Java strongly preferred. Candidates without Java experience must demonstrate strong expertise in another language and the ability to quickly learn new technologies as needed. Experience with infrastructure management or automation tools (e.g. Python, Ruby, chef, ansible) and Kubernetes familiarity is a significant plus. The candidate should be comfortable tackling complex problems and applying strong analytical and problem solving skills.

Overview
Purpose of the Team -
o Provide world-class support and expertise to all of MasterCard's Big Data environments.
o Provide Big Data thought leadership and technology roadmaps to support MasterCard's vision and evolution.
Purpose of the Job -
o Represent MasterCard's Big Data Engineering team as a subject matter expert.
o Support MasterCard's present Big Data infrastructures.
o Integrate new technologies and platforms into current environments as technologies evolve and demands change.
Who would be the best fit for this role eg:
o What technologies do you have hands-on experience with, and what degree is that experience?
o In your current role, what new Big Data technologies have you integrated into your current environment and how?
o What experience do you have with migrating users and applications from traditional infrastructure and technologies to newer Big Data platforms?

Role
Work directly with application and development teams to completely understand all requirements for project and ad hoc requests and provide accurate estimates.
Participate in a team of technical subject matter experts that provides 24x7 operational support of mission-critical Big Data environments for highly-available and contingent platforms.
Create and maintain documentation and define department best practices in support of proper change management, security and operational requirements.
Investigate and understand new technologies and additional capabilities of current platforms that support MasterCard's vision of Big Data, leading their adoption when appropriate.
Educate peers and cross-train teammates with Big Data concepts and application of technology to serve business needs
Interact with managed services resources and direct maintenance and operational support activities per MasterCard standards and industry best-practices.

All About You
Essential Knowledge, Skills and Experience
o Strong knowledge of Hadoop platforms and other distributed data processing platforms.
o Linux/Unix and system administration: Advanced Linux knowledge is a must. Understanding of shell, debugging things etc. The candidate should be able to get their way around Linux and get things to work.
o Strong background in delivering mission-critical Big Data project work while interacting with diverse and experienced teammates across the spectrum of enterprise operational functions.
o Troubleshooting: The candidate must be able to engage in solving complex problems. Programming problems are a good example.
o Programming: Good in one programming language. Java would be preferable. If not Java, the candidate must have a good handle on one language and display the ability to pick up a new one if required. Experience on programming infrastructure management or automation tools is a big plus.
o Communication skills: Excellent written and verbal communication skills can communicate up to executive level. This comprises of clear and concise articulation of thoughts.
Desirable or Additional Capabilities
o Knowledge and hands-on experience with Kubernetes, as well as automation using tools such as Ansible and Chef, is a plus.
o Understanding of and experience with network engineering in support of large IT organizations.
o Knowledge in statistics and machine learning would be highly beneficial: The candidate should have a strong developed mentality with a focus on measuring and metrics
o Background in ETL. Practical experience dealing with large ETL pipelines is a plus.
o Databases: Basic SQL knowledge a must. Good to have advanced data warehousing and MPP knowledge but it's not a must have.