Our client is currently seeking a Senior Data Engineer. If you believe your skill set aligns with the description below, please send updated resume to email@example.com
*THIS IS A FULL-TIME/PERMANENT OPPORTUNITY*
About the Job
Data is an essential part of our client's business. We use it to build financial profiles for our users, to inform product and business strategy, and to understand and optimize every aspect of our service.
Data engineers work with data scientists, product managers, fellow engineers and other internal stakeholders. We design internal data services and tools, develop pipelines and APIs for our various data feeds, build libraries and frameworks to enable new types of analysis for our data science team, and instrument, measure and monitor key metrics within the company.
As a senior data engineer you'll be a member of a small team with a big impact. You will create systems that shape the future of how our client uses data. Your work will reach millions of people worldwide, and help bring modern financial services to the developing world.
* Build data pipelines and frameworks to automate ETL for batch and streaming data
* Define and lead the build out of architectures and best practices for big data processing
* Decompose complex data handling challenges to design robust, fault-tolerant systems
* Recommend and implement ways to improve data reliability, efficiency, transparency and quality
* Manage, scale and operate data system infrastructure
* Work with Product Owners, Developers, Data Scientists and Customers in a collaborative and agile environment
* Prior experience working on data intensive systems building robust ETL pipelines
* Comfortable working in a UNIX environment
* Experience writing high quality code in Python, Java, Go, Clojure or a similar general purpose language
* Background working with distributed data processing frameworks (Spark, Hadoop, etc) and understand when to use them (and when not to)
* Solid understanding of SQL and relational databases
* Experience defining large data systems including provisions for security, availability and recovery
* Experience with deployment tools and infrastructure such as Terraform, Ansible, Docker, Teamcity or equivalents
* Experience with AWS infrastructure provisioning and configuration