Overview
0.0
Contract - W2
Skills
Data Lake
Warehouse
Data Migration
FOCUS
Collaboration
Extract
Transform
Load
Big Data
PySpark
Linux
Red Hat Enterprise Linux
Python
Coaching
Mentorship
SQL
Streaming
Java
Machine Learning (ML)
API
Performance Tuning
Optimization
Apache Kafka
Amazon Web Services
Electronic Health Record (EHR)
Data Processing
Kubernetes
Management
Apache Spark
Cloud Computing
HL7
Job Details
Role:Senior Spark Architect/Engineer
Location: Remote (core business hours are 9am to 3pm cst)
Duration: 6 month contract to start
Key Responsibilities
Architect, develop, test, and deploy data lake/warehouse solutions using modern development tools in AWS.
Design and implement robust data ingestion pipelines for high-volume datasets.
Lead data migration and transformation initiatives with precision and efficiency.
Develop and integrate software applications using industry-standard methodologies and architectural patterns, with a strong focus on performance and security.
Collaborate with Business Analysts, Architects, and Senior Developers to define and establish the AWS application framework.
Automate end-to-end ETL processes for diverse datasets within the big data ecosystem.
Required Skills & Experience
Proficiency in SQL, Spark/PySpark, Spark Structured Streaming, Kafka, Linux/RHEL 8 , Python, Java, and REST APIs.
Strong interpersonal skills with the ability to communicate complex technical concepts to both technical and non-technical stakeholders.
Proven experience in leading, coaching, and mentoring technical teams.
Hands-on experience with:
Spark SQL, Spark Structured Streaming, and Java Spark Framework for large-scale data processing.
Machine learning capabilities via API
Performance tuning and optimization of Spark jobs.
Designing high-throughput, low-latency pipelines using Kafka.
AWS EMR for distributed data processing.
Experience with Kubernetes or like framework to deploy and manage Spark clusters and applications.
Experience with Hybrid Cloud Architectures
Delta Table Experience
Preferred Qualifications
Familiarity with HL7 FHIR specifications is a plus.
Experience with Serverless Architectures.
Location: Remote (core business hours are 9am to 3pm cst)
Duration: 6 month contract to start
Key Responsibilities
Architect, develop, test, and deploy data lake/warehouse solutions using modern development tools in AWS.
Design and implement robust data ingestion pipelines for high-volume datasets.
Lead data migration and transformation initiatives with precision and efficiency.
Develop and integrate software applications using industry-standard methodologies and architectural patterns, with a strong focus on performance and security.
Collaborate with Business Analysts, Architects, and Senior Developers to define and establish the AWS application framework.
Automate end-to-end ETL processes for diverse datasets within the big data ecosystem.
Required Skills & Experience
Proficiency in SQL, Spark/PySpark, Spark Structured Streaming, Kafka, Linux/RHEL 8 , Python, Java, and REST APIs.
Strong interpersonal skills with the ability to communicate complex technical concepts to both technical and non-technical stakeholders.
Proven experience in leading, coaching, and mentoring technical teams.
Hands-on experience with:
Spark SQL, Spark Structured Streaming, and Java Spark Framework for large-scale data processing.
Machine learning capabilities via API
Performance tuning and optimization of Spark jobs.
Designing high-throughput, low-latency pipelines using Kafka.
AWS EMR for distributed data processing.
Experience with Kubernetes or like framework to deploy and manage Spark clusters and applications.
Experience with Hybrid Cloud Architectures
Delta Table Experience
Preferred Qualifications
Familiarity with HL7 FHIR specifications is a plus.
Experience with Serverless Architectures.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.