Senior Data Engineer

Job Title: Senior Data Engineer 

Job Summary

We are seeking an experienced Senior Data Engineer with 12+ years of expertise in designing, building, and optimizing scalable data pipelines and architectures. The ideal candidate will have deep knowledge of big data technologies, cloud platforms, and data modeling, along with strong leadership and mentoring skills.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines (batch & real-time).
• Architect and optimize data lakes, data warehouses, and ETL/ELT processes.
• Work closely with data scientists, analysts, and business teams to deliver high-quality data solutions.
• Ensure data quality, governance, security, and compliance standards.
• Lead the design and implementation of data models (dimensional & normalized).
• Optimize performance of large-scale data processing systems.
• Build and manage streaming solutions (Kafka, Spark Streaming, etc.).
• Drive adoption of best practices in data engineering, CI/CD, and DevOps.
• Mentor junior engineers and lead technical discussions/decisions.
• Collaborate with cross-functional teams in Agile/Scrum environments.

Required Skills & Qualifications

Experience

• 12+ years in data engineering, data warehousing, or related fields.
• Proven experience in handling large-scale distributed systems.

Technical Skills

• Strong programming skills in Python / Scala / Java.
• Expertise in SQL and NoSQL databases.
• Hands-on experience with:
  • Apache Spark / Hadoop ecosystem
  • Kafka or other streaming platforms
  • Airflow or similar orchestration tools
• Experience with cloud platforms: AWS / Azure / Google Cloud Platform
• Data warehousing solutions like Snowflake, Redshift, BigQuery, etc.
• Strong understanding of ETL/ELT frameworks.

Data & Architecture

• Advanced knowledge of:
  • Data modeling (Star/Snowflake schema)
  • Data governance and lineage
  • Data partitioning and indexing strategies

Preferred Qualifications

• Experience with real-time data processing systems
• Exposure to machine learning pipelines
• Knowledge of DataOps / MLOps practices
• Certifications in cloud platforms (AWS, Azure, Google Cloud Platform)
• Experience with containerization (Docker, Kubernetes)

Soft Skills

• Strong leadership and mentoring abilities
• Excellent problem-solving and analytical skills
• Effective communication with technical and non-technical stakeholders
• Ability to drive projects independently

Education

• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field