Cloud Data Engineer

The Cloud Data Engineer works as a key member of our team that leverages data as a service for acquiring, securing, cataloging, processing, and analyzing of internal and external data sets. This role will have the opportunity to shape the product strategy, vision and portfolio of our platform. Once joining the team, you will become a member of a fast-paced engineering team focused on designing and implementing large-scale distributed data processing systems using cutting edge cloud based, open source and proprietary big data technologies. In this role, you will implement a variety of solutions to ingest data into, process data within, and expose data from the platform.

Responsibilities

• Provide technical and/or business application consultation to business partners and team members in the areas of functionality, architecture, operating systems and databases for complex application systems.
• Develop data platform components in a cloud and on-premises environments to ingest data and events from cloud and on-premises environments as well as third parties
• Build automated pipelines and data services to validate, catalog, aggregate and transform ingested data
• Build automated data delivery pipelines and services to integrate data from the data lake to internal and external consuming applications and services
• Build and deliver deployment and monitoring capabilities consistent with DevOps models
• Works with architects in transforming high level architecture designs and assists in technical delivery of large scale enterprise projects to implement optimized end-to-end solutions.
• Develop low-level technical specifications and detailed program specifications to promote a solid core application that can be re-used across projects.
• Analyze existing systems and architectures for improvement recommendations.
• Assist in troubleshooting production issues and new build deployments.
• Ensures code quality, performs code reviews, and mentors development team members.
• Ensure users expectations are met, gain understanding when desired outcomes are not feasible and provide alternative solutions to meet objective(s).
• Design and develop software for new functionality, improvements and system longevity.
• Ensure all documentation of technical architecture and systems are complete.
• Provide training and guidance to team members and users as required.
• Must be available to meet schedules of global operation by being available for off hour meetings.
• Must be able to travel if necessary.
• Maintain regular and predictable attendance.
• Perform other duties as assigned.

Requirements

• Bachelor s degree in Computer Science or equivalent education and experience
• 3+ years experience in programming/systems analysis.
• 2+ years of hands-on experience with big data and data-at-scale platform services
• 2+ years experience utilizing Python for big data processing
• 1+ years experience developing cloud-native applications and deploying to a cloud environment (i.e. AWS, Google Cloud Platform, Azure, etc.)
• Experience with JavaScript/Node.js
• Experience with Scala/Groovy/Java
• API Development (proper microservice separation, HTTP verb usage)
• Experience in infrastructure automation on AWS with Terraform or similar approaches
• Experience working with bash scripting and the AWS CLI in Linux-based systems
• Experience working with data in various compression formats (including Parquet, ORC) and serialization (AVRO).
• Solid knowledge of the following technologies: HTTP, SSL/TLS, REST, SQL, and JSON.

• Hands-on expertise with multiple database technologies (Postgres, Mongo, Elastic Search, etc.) as well as SQL and related query languages
• Knowledge of Continuous Integration environment such as Jenkins, CruiseControl, Continuum, Travis, etc
• Knowledge of Test Driven Development processes and tooling such as JUnit, Mocha, Jasmine, or Protractor
• Experience with or understanding of container services including Docker and Kubernetes.
• Experience in working with the Agile environments (i.e., Scrum and Kanban).
• Experience deploying platform monitoring and performance tracking services
• Advanced ability to translate business needs and problems into systems design and technical solutions.
• Complex analytical and problem-solving skills
• Advanced oral and written communication skills

Preferred:

• Master s degree and/or LOMA certification
• 6+ years experience in programming/systems analysis