Data Engineer with Pyspark

Rocky Hill, CT, US • Posted 5 days ago • Updated 1 day ago

Full Time

On-site

Depends on Experience

Hexplora

Fitment

Dice Job Match Score™

👤 Reviewing your profile...

Job Details

Skills

ETL
Python and Pyspark

Summary

Title: Data Engineer (PySpark)
Location: Rocky Hill, CT (Onsite)

Job Summary

We are seeking a highly skilled Data Engineer with strong experience in PySpark, Databricks, and Apache Spark to design, build, and optimize scalable data pipelines. The ideal candidate will have a solid background in big data processing, cloud platforms, and distributed systems, with a focus on delivering high-quality, reliable data solutions.

Key Responsibilities

Design, develop, and maintain scalable data pipelines using PySpark and Apache Spark
Build and optimize ETL/ELT workflows on Databricks
Collaborate with data scientists, analysts, and stakeholders to understand data requirements
Ensure data quality, integrity, and governance across data platforms
Optimize performance of Spark jobs and large-scale data processing systems
Work with structured and unstructured data from multiple sources
Implement data storage solutions using cloud platforms (AWS, Azure, or GCP)
Monitor, troubleshoot, and resolve data pipeline issues
Maintain documentation for data architecture and workflows

Required Qualifications

Bachelor’s degree in Computer Science, Engineering, or related field
Strong hands-on experience with PySpark and Apache Spark
Proven experience working with Databricks
Proficiency in Python and SQL
Experience with distributed data processing and big data technologies
Familiarity with cloud platforms (AWS, Azure, or GCP)
Experience with data warehousing solutions (e.g., Snowflake, Redshift, BigQuery)
Strong problem-solving and analytical skills

Infowave Systems is an equal opportunity employer that is committed to diversity and inclusion in the workplace.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 91015321
Position Id: 26-00040
Posted 5 days ago

Company Info

About Hexplora

Our Purpose:

In today s rapidly evolving healthcare space, all healthcare organizations large and small are looking to leverage data to obtain actionable information that can be used to improve the health and wellness of patients in the most cost effective manner. Hexplora offers a comprehensive, end-to-end data warehousing and business intelligence solution for Health Plans, Accountable Care Organizations (ACOs), Independent Physicians Associations (IPAs), self-insured employers, and institutional providers to address all of their reporting and analytics requirements, allowing them to easily decipher mass information through a single source and apply it to cost savings and care management analysis.

Our Team:

Hexplora is led by a highly dynamic and motivated team of technology experts with deep healthcare expertise and experience in implementing industry leading solutions that enable ACOs, IPAs, and Payers to leverage Informatics as a strategic asset that can deliver market differentiating and cost saving capabilities.

Overview:

Through establishing Infowave Systems, an IT Services and Solutions company, our leadership has been instrumental in assisting Healthcare companies with designing, developing and deploying Enterprise Data Warehouses (EDW), Business Intelligence (BI), Analytics, Web Portals and Reporting Modules in various technologies for over fifteen years. With the experience and expertise gained over the years, Infowave Systems designed and developed a Business Analytics Solution referred to as Healthcare Analytical and Reporting Platform (HARP). In 2013, HARP solution was spun off as HEXPLORA to offer the solution under a SaaS model (Software as a Service).

Go to company profile

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Data Engineer I (Snowflake)

Hartford, Connecticut

•

5d ago

Who Are We? Taking care of our customers, our communities and each other. That's the Travelers Promise. By honoring this commitment, we have maintained our reputation as one of the best property casualty insurers in the industry for over 170 years. Join us to discover a culture that is rooted in innovation and thrives on collaboration. Imagine loving what you do and where you do it. Job Category Technology Compensation Overview The annual base salary range provided for this position is a nationw

Full-time

USD 109,300.00 - 180,200.00 per year

Data Engineer I (Databricks, AWS, Python)

Hartford, Connecticut

•

Yesterday

{"description": " Who Are We? Taking care of our customers, our communities and each other. That's the Travelers Promise. By honoring this commitment, we have maintained our reputation as one of the best property casualty insurers in the industry for over 170 years. Join us to discover a culture that is rooted in innovation and thrives on collaboration. Imagine loving what you do and where you do it. Job Category Data Analytics, Data Science, Technology Compensation Overview The annual base sala

Full-time

USD 109,300.00 - 180,200.00 per year

Data Engineer II, Strategic Ingestion Product

Hartford, Connecticut

•

Today

{"description": " Who Are We? Taking care of our customers, our communities and each other. That's the Travelers Promise. By honoring this commitment, we have maintained our reputation as one of the best property casualty insurers in the industry for over 170 years. Join us to discover a culture that is rooted in innovation and thrives on collaboration. Imagine loving what you do and where you do it. Job Category Technology Compensation Overview The annual base salary range provided for this pos

Full-time

USD 126,500.00 - 208,700.00 per year

Lead Data Engineer, Imagery Models

Hartford, Connecticut

•

Yesterday

{"description": " Who Are We? Taking care of our customers, our communities and each other. That's the Travelers Promise. By honoring this commitment, we have maintained our reputation as one of the best property casualty insurers in the industry for over 170 years. Join us to discover a culture that is rooted in innovation and thrives on collaboration. Imagine loving what you do and where you do it. Job Category Technology Compensation Overview The annual base salary range provided for this pos

Full-time

USD 161,400.00 - 266,300.00 per year

Search all similar jobs