Overview
Remote
Up to $73
Full Time
Skills
Druid
Amazon Web Services
Data Engineering
SQL
Python
Systems Design
Application Development
Apache Hadoop
Apache Kafka
Big Data
Apache Spark
Scala
Visualization
Storage
Job Details
Data Engineer, Distributed Systems Druid Engineer
Remote or Austin, Texas
KEY RESPONSIBILITIES
Design and Development:
o Design and implement data ingestion pipelines to load data into Apache Druid.
o Develop and optimize Druid schemas and data models for efficient querying and performance.
o Design and implement data aggregations against event-level data in distributed database (preferably in Druid)
o Migration from existing data stores to Druid
Optimization:
o Implement best practices for data ingestion, storage, and querying to improve system performance.
o Optimize query performance and resource utilization.
KEY QUALIFICATIONS
5+ years of experience in data engineering or analytics in distributed data systems, i.e., Druid, Snowflake, Redshift, CockroachDb, or Pinot (Druid experience preferred).
Strong understanding of distributed data store architecture, data ingestion methods, scaling mechanisms and querying capabilities.
Proficiency in SQL and experience with data modeling and ETL processes.
Hands-on application coding experience in Java, Python or others.
A fast learner of new technologies.
Experience with performance tuning and optimization of data systems.
Excellent problem-solving skills and the ability to work independently as well as in a team.
Familiarity with other big data technologies (e.g., Hadoop, Spark, Kafka) is a plus.
Highly beneficial:
Familiarity with cloud platforms (e.g., AWS, Google Cloud Platform, Azure) and containerization (e.g., Docker, Kubernetes).
Knowledge of programming languages such as Scala.
Experience generating reports against distributed data systems with SQL and visualization tools.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.