Overview
On Site
Depends on Experience
Full Time
Skills
Advertising
Analytics
Apache Flink
Apache Hadoop
Apache Kafka
Apache Spark
Artificial Intelligence
Big Data
Collaboration
Data Engineering
Data Processing
Extract
Transform
Load
Scala
Python
Conflict Resolution
Data Security
Data Storage
Database
Docker
Java
Kubernetes
Machine Learning (ML)
NoSQL
Orchestration
Problem Solving
Real-time
Regulatory Compliance
SQL
Streaming
Job Details
We are seeking Sr. Data Engineer to join our engineering team. This role is ideal for a skilled developer with a strong background in big data processing, data pipeline development, and distributed systems. As a Data Engineer, you will play a crucial role in designing, building, and maintaining the data infrastructure that powers AI-driven advertising platform. You will collaborate closely with data scientists, software engineers, and product managers to ensure efficient and scalable data solutions.
Key Responsibilities:
- Design, develop, and maintain scalable data pipelines and ETL processes for handling large-scale data from multiple sources.
- Optimize data storage, processing, and retrieval for high-performance analytics and machine learning applications.
- Work closely with data scientists to prepare, clean, and structure data for machine learning model development and deployment.
- Develop and maintain real-time and batch data processing systems using technologies like Spark, Kafka, and Flink.
- Implement monitoring, logging, and alerting solutions to ensure the reliability of data pipelines.
- Ensure data security, governance, and compliance with industry regulations.
- Collaborate with engineering teams to integrate data solutions into broader ad-tech infrastructure.
Qualifications & Skills:
- 3+ years of experience in data engineering, big data processing, or related roles (6+ years for Senior Data Engineer role)
- Strong programming skills in Python, Java, or Scala.
- Experience with distributed data processing frameworks such as Apache Spark, Flink, or Hadoop.
- Strong understanding of SQL and NoSQL databases for data storage and retrieval.
- Experience with real-time data streaming technologies like Apache Kafka or Pulsar.
- Knowledge of containerization and orchestration tools like Docker and Kubernetes is a plus.
- Strong problem-solving skills and ability to work in a fast-paced environment.
- Bachelor's or Master s degree in Computer Science, Engineering, or a related field.
Thanks & Regards,
Parijat Banerjee
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.