Data Engineer with Pyspark and Cloud Era Experience

Data Engineer with Pyspark and Cloud Era Experience

Rancho Cucamonga, CA - Hybrid(3 days Onsite/ week)

Fulltime

Job Description:

We are seeking an experienced and motivated Data Engineer Data and PySpark to join our team. As a key member of our data engineering team, you will be responsible for designing, building, and maintaining scalable data pipelines to support data processing and analytics within the healthcare domain.

Key Responsibilities:

• Design, develop, and optimize data pipelines using PySpark for large-scale data processing and transformation tasks.
• Work closely with data scientists, analysts, and healthcare stakeholders to understand data requirements and deliver efficient solutions.
• Extract, transform, and load (ETL) data from a variety of healthcare data sources (e.g., EHR systems, insurance claims, patient records, etc.) into data storage solutions like data lakes, data warehouses, or cloud-based platforms.
• Perform data quality checks, identify discrepancies, and clean up data issues to ensure the integrity of healthcare-related datasets.
• Write complex SQL queries for data extraction, manipulation, and reporting purposes.
• Develop and manage SSIS (SQL Server Integration Services) packages to automate data workflows and integrate with multiple healthcare systems.
• Maintain and optimize databases, ensuring high availability, reliability, and performance.
• Collaborate with cross-functional teams to deliver data solutions that meet business requirements and enhance decision-making capabilities.
• Troubleshoot and resolve data processing issues, ensuring that data pipelines run efficiently.
• Stay up to date with industry trends and best practices in data engineering and healthcare data management.

Requirements:

• Bachelor's degree in Computer Science, Information Technology, or a related field (or equivalent work experience).
• 8+ years of hands-on experience in Data Engineering with strong expertise in SQL, SSIS, and PySpark.
• Solid experience working with healthcare data, including Electronic Health Records (EHR), insurance claims, patient data, and other healthcare-related datasets.
• Proficiency in building and optimizing data pipelines using PySpark for large-scale data processing.
• Strong SQL skills for writing complex queries, data manipulation, and reporting.
• Experience with SSIS for ETL processes and integration of data from diverse sources.
• Solid understanding of database architecture, data warehousing concepts, and cloud technologies (e.g., AWS, Azure).
• Excellent problem-solving skills with the ability to troubleshoot data issues and optimize performance.
• Familiarity with healthcare data standards such as HL7, HIPAA compliance, ICD-10, and other healthcare regulations is a plus.
• Strong communication skills with the ability to collaborate effectively with both technical and non-technical stakeholders.

Thanks,

Vinod.