Senior Data Engineer

Overview

On Site
Full Time

Skills

Apache Hadoop
Management
Data Architecture
Scalability
Data Quality
Accessibility
Big Data
Leadership
Data Engineering
Apache Kafka
Computer Cluster Management
Data Storage
Amazon S3
HDFS
Databricks
Data Processing
Apache Spark
Apache Flink
Java
SQL
Data Science
Docker
Kubernetes
Grafana
Statistics
Unix
Scripting
Bash
Python

Job Details

Primary Duties

  • Design and implement scalable infrastructure for large-scale data systems (e.g., Kafka, Hadoop, Dremio)
  • Develop, deploy, and oversee data pipelines using technologies such as Java, Python, Spark, and Flink
  • Partner with engineering teams to support data architecture, ingestion strategies, and system scalability
  • Ensure data quality, consistency, and accessibility for internal stakeholders
  • Serve as a subject matter expert in Big Data, offering guidance and support to both technical and non-technical teams


Qualifications and Skills

  • Minimum of 5 years in a robust data engineering environment
  • At least 3 years of hands-on experience with Kafka, including stream processing and cluster management
  • 2+ years working with large-scale data storage solutions (e.g., S3, HDFS, Databricks, Iceberg)
  • Proficiency with distributed data processing tools like Apache Spark or Flink
  • Strong programming background in Java, Python, and SQL
  • Familiarity with Python-based data science libraries and toolkits
  • Experience deploying applications in containerized environments using Docker and Kubernetes
  • Knowledge of monitoring and alerting tools such as Prometheus, Grafana, Alert Manager, Alerta, and OpsGenie
  • Solid foundation in statistical analysis and troubleshooting methodologies
  • Comfortable with Unix-based scripting (e.g., Bash, Python)
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.