Senior Data Engineer (AWS, PySpark, Redshift, Datalake) ** Direct end client **

We are seeking a Senior Data Engineer with a strong background in software development, AWS cloud services, PySpark, S3 based datalake, EMR, Hive, SparkSQL. The successful candidate will collaborate with our team of AI scientists and data analysts to design, develop, and maintain data pipelines and systems that support clinical and operational data use-cases.

Key Responsibilities:

• Design, develop, and maintain scalable data pipelines and systems to support clinical and operational data use-cases.
• Utilize Airflow/Kafka and event-driven architecture to build efficient real-time data pipelines.
• Collaborate with AI scientists and data analysts to align data solutions with business needs.
• Develop and maintain data warehouse and ETL processes, ensuring data quality and integrity.
• Implement data security, privacy, and compliance measures.
• Perform data analysis, troubleshoot data issues, and provide technical support to end-users.
• Integrate predictive models into the analytics platform.
• Monitor and optimize data pipelines for performance and scalability.
• Stay updated with the latest technologies in data engineering, AWS cloud services, and healthcare/pharmaceutical industry.

Skills Required:

• Bachelor’s degree in Computer Science or related field.
• 5+ years of experience in software or data engineering.
• Experience in designing and developing data pipelines in a cloud environment (preferably AWS).
• Proficiency in Python programming language.
• Expertise in Airflow/Kafka and event-driven architecture.
• Familiarity with SQL, NoSQL databases, and data warehousing concepts.
• Experience with data security, privacy, and compliance measures.
• Strong analytical skills and problem-solving abilities.
• Excellent communication skills.