Location: Lincoln, NE
Salary: $60.00 USD Hourly - $65.00 USD Hourly
Description: ONLY W2
Job Title: Senior Data Engineer - Encounter Processing & Data Location: Lincoln, NE (Remote) Duration: 12+ Months About the Role We are seeking a highly experienced Senior Data Engineer to serve as the primary technical owner of a large-scale enterprise data processing platform responsible for ingesting, transforming, validating, and processing healthcare and operational data. This mission-critical environment supports complex batch processing workflows, business rule execution, and downstream reporting and analytics systems.
The successful candidate will be responsible for the ongoing maintenance, enhancement, operational support, and optimization of a distributed data processing framework built on Scala, Apache Spark, Hive, and Drools, operating within a Cloudera Data Platform (CDP) environment.
This role requires deep technical expertise, strong production support capabilities, and the ability to independently manage the full lifecycle of enterprise data processing solutions in a highly regulated environment.
What You'll Do Technical Ownership & Platform Management - Own and maintain the enterprise Scala/Spark data processing framework.
- Manage and enhance business rule implementations using the Drools Rules Engine.
- Oversee enterprise data ingestion processes for reference, master, and source-system data.
- Support and optimize Spark and Hive batch processing workflows.
- Lead production issue investigation, root cause analysis, and defect resolution.
- Coordinate application releases, deployments, and version management activities.
- Drive application performance tuning, scalability improvements, and operational efficiency.
- Maintain technical documentation, operational procedures, and knowledge-sharing resources.
- Provide platform monitoring and basic administrative support within the Cloudera ecosystem while collaborating with infrastructure and cloud operations teams.
Production Support & Continuous Improvement - Troubleshoot and resolve issues across distributed data processing applications.
- Monitor system health, job execution, and data pipeline performance.
- Implement business, regulatory, and operational enhancements within existing data processing frameworks.
- Ensure platform reliability, maintainability, and operational continuity.
- Collaborate with cross-functional teams to deliver high-quality, scalable data solutions.
Minimum Qualifications - Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field, or equivalent practical experience.
- 7+ years of experience designing and developing enterprise-scale distributed data processing applications.
- Strong hands-on programming experience with Scala.
- Experience building and maintaining Apache Spark (2.x and/or 3.x) applications.
- Experience working with Cloudera Data Platform (CDP) or comparable enterprise Hadoop ecosystems.
- Strong expertise with Apache Hive for data processing, data modeling, and table management.
- Experience implementing and maintaining business rules using the Drools Rules Engine.
- Advanced SQL development, query tuning, and performance optimization skills.
- Experience supporting Linux-based production environments.
- Proven experience troubleshooting and optimizing distributed Spark workloads.
- Experience with Git and modern software development and version control practices.
Preferred Qualifications - Experience in healthcare, payer, or regulated industry environments.
- Familiarity with large-scale encounter, claims, or transactional data processing systems.
- Experience with Cloudera Manager, HDFS, and YARN administration.
- Experience integrating with enterprise ETL platforms such as IBM DataStage.
- Familiarity with cloud environments supporting Hadoop and Cloudera workloads, including AWS.
Experience working within Agile development methodologies and tools such as Jira and Confluence
By providing your phone number, you consent to: (1) receive automated text messages and calls from the Judge Group, Inc. and its affiliates (collectively "Judge") to such phone number regarding job opportunities, your job application, and for other related purposes. Message & data rates apply and message frequency may vary. Consistent with Judge's Privacy Policy, information obtained from your consent will not be shared with third parties for marketing/promotional purposes. Reply STOP to opt out of receiving telephone calls and text messages from Judge and HELP for help.
Contact: This job and many more are available through The Judge Group. Please apply with us today! 
Never miss an opportunity! Create an alert based on the job you applied for.