Big Data Developer/Engineer with Python-Hybrid-Full time

We are seeking a Python Data Engineer with strong expertise in Apache Spark (PySpark) to build scalable and efficient data processing systems. The ideal candidate will work on large datasets, develop robust pipelines, and support analytics and business intelligence initiatives.

Key Responsibilities

• Design and build scalable data pipelines using Python and Apache Spark.
• Develop and maintain ETL/ELT workflows for processing large volumes of data.
• Perform data cleansing, transformation, and aggregation for analytics and reporting.
• Collaborate with data scientists, analysts, and business teams to improve data workflows.
• Ensure data quality, consistency, and integrity across multiple data sources.
• Monitor and troubleshoot data processing jobs to ensure performance and reliability.
• Optimize Spark jobs and distributed data processing systems.
• Integrate data pipelines with enterprise systems and data platforms.
• Follow best practices in coding, testing, and documentation.

Roles & Responsibilities

• Develop efficient and reusable Python/PySpark code for data engineering tasks.
• Work with large structured and unstructured datasets.
• Analyze data to extract insights and support business decisions.
• Collaborate with stakeholders to translate business requirements into data solutions.
• Mentor junior team members and contribute to knowledge sharing.
• Manage deployment and monitoring of data pipelines in production environments.