Apache Spark jobs

Berkeley Heights, New Jersey

•

3d ago

Job Description 10+ years solid experience with deploying, configuring, monitoring, and troubleshooting Apache Spark clusters and associated big data frameworks (such as Hadoop, Kafka, or cloud-based data platforms) to ensure high availability and performance.Focuses on cluster health, performance tuning, and production data pipelines.Technical Experience: Deep hands-on experience with Apache Spark architecture (Driver, Workers, Executors, RDDs/DataFrames) and cluster managers (YARN, Kubernetes,

Easy Apply

Full-time

$100,000 - $120,000

Application Programmer (Python/Spark) - Onsite

Pennington, New Jersey

•

Today

Genesis10 is currently seeking an Application Programmer (Python/Spark) for an Onsite position with a Global Financial Institution located in Charlotte, NC, Pennington, NJ or Plano, TX. This is a 12 month contract opportunity. This role is part of a growing Developer Experience (DevEx) department within the Global Technology division. The team is responsible for improving the developer experience, advancing technology platforms through innovation, and reducing risk and improving quality across

Easy Apply

Full-time

USD 55.06 - 63.06 per hour

Kafka/Spark Developer

Pittsburgh, Pennsylvania

•

28d ago

Job Title: Kafka/Spark Developer Location: Pittsburgh, Pennsylvania Type: Permanent Full-Time Work Model: Onsite onsite Overview System One is looking for mid level Kafka and Spark Software Developers to join our Applications Development and Maintenance team, supporting our client which is a large US Bank, working in an advanced technology environment. This role will require someone at our client site 5 days a week in Pittsburgh, PA. Responsibilities Develop and maintain scalable big data s

Easy Apply

Full-time

$156700

Reliability Engineer (Spark/Foundry Workflows)

No location provided

•

Today

M9 Solutions is dedicated to providing IT services and solutions to the Federal Government by mobilizing the right people, skills, clearance levels, and technologies to help organizations that desire improved performance and modern, sustainable change. M9 has provided quality IT services and support to more than 30 Federal Agencies and multiple commercial customers nationwide. Our capabilities include IT Talent Solutions, Data Delivery & Analytics, Cyber Security, Cloud Migration, Applications a

Full-time

USD 130,000.00 - 145,000.00 per year

Kafka/Spark Developer

Millvale, Pennsylvania

•

26d ago

Job Title: Kafka/Spark Developer Location: Pittsburgh, PA | Onsite Duration: Full-time Job Description: Our client is looking for mid level Kafka and Spark Software Developers to join Applications Development and Maintenance team, supporting our client which is a large Client, working in an advanced technology environment. Duties and Responsibilities: As Kafka Spark Software developers, you will be responsible for developing and maintaining scalable big data solutions using Hadoop, S

Easy Apply

Full-time, Third Party

$110000 - $120000

Senior AI Engineer, Databricks, ML, Spark ML

Remote

•

Today

We are looking for a Senior AI Engineer to build, test, and deploy ML and GenAI components for asset onboarding, industrial diagnostics, and device analytics on Databricks. This specialist will create predictive models, anomaly-detection algorithms, RAG pipelines, document extraction, and AI agent capabilities. Responsibilities Construct, test, and deploy ML and GenAI components on Databricks Create predictive models for asset onboarding and device analytics Architect and implement anomaly-detec

Full-time

Lead Data Software Engineer with Databricks with Apache Kafka, Apache Spark, Kubernetes

Remote

•

Today

We are looking for a Lead Data Software Engineer to drive the migration of an existing data analytics platform, originally built on Azure resources such as Data Factory, Databricks, Event Hub, and Cassandra, to a cloud-agnostic platform deployable on-premises. Responsibilities Implement streaming and batch Spark pipelines running on Kubernetes Lead the implementation of objects within the data generator framework Oversee deployment and testing across local and development environments Drive impr

Full-time

Kafka/Spark Developer

Pittsburgh, Pennsylvania

•

27d ago

Easy Apply

Full-time

Depends on Experience

Kafka/Spark Developer

Pittsburgh, Pennsylvania

•

28d ago

Easy Apply

Full-time

$156700

Kafka/Spark Developer

Hybrid in Pittsburgh, Pennsylvania

•

22d ago

Job Title: Kafka/Spark Developer Location: Pittsburgh, Pennsylvania Type: Permanent Full Time Work Model: Hybrid onsite and remote Responsibilities Develop and maintain scalable big data solutions using Hadoop, Spark, Kafka, and Impala to support enterprise data processing and analytics initiatives.Design, build, and optimize batch and real-time data pipelines for ingesting, processing, transforming, and delivering large volumes of structured and unstructured data.Develop Spark applications usi

Easy Apply

Full-time

Depends on Experience

Need Sr. Hadoop Developer with AB Initio, Spark, Python & Health Insurance experience:: Jacksonville, FL (Hybrid)

Hybrid in Jacksonville, Florida

•

Today

Job Title: Sr. Hadoop Developer Location: Jacksonville, FL (Hybrid/Onsite) Experience: 10+ years preferred Job Description: We are seeking an experienced Senior Hadoop Developer with strong expertise in AB Initio, Hadoop, Spark, Python, SQL, and Stored Procedures. The ideal candidate will be responsible for designing, developing, and optimizing large-scale data processing solutions, building ETL/ELT pipelines, and managing complex data integration workflows. Experience with Snowflake is a plus.

Easy Apply

Contract

Depends on Experience

Apache Spark/Core Java Developer

Bayonne, New Jersey

•

Today

Location: Bayonne, NJ Salary: $140,000.00 USD Annually - $145,000.00 USD Annually Description: Apache Spark/core Java Developer New York, NY 10110 Fulltime Final round F2F in NYC Core java, Apache Spark, Python, DB SQL, Linux, Application/ server-side development. Hands-on experience designing, building, and integrating analytical systems. Experience - 5-10 years Strong programming experience in Core Java Hands-on expertise in Apache Spark Proficiency in Python Strong knowledge of SQL and Data

Full-time

USD 140,000.00 - 145,000.00 per year

Scala / Apache Spark Developer

Hybrid in Jersey City, New Jersey

•

Today

Job Role: Scala / Apache Spark Developer (Onsite or Hybrid or Remote): Hybrid 3 days (Relocation allowed or Not allowed): Need locals either from NY/NJ Contract Duration: 11+ months Interview Mode: In-person at client location Position Overview The ideal candidate will have solid programming skills, experience developing scalable data processing solutions, and the ability to collaborate with technical teams in a fast-paced environment. Key Responsibilities Design, develop, and maintain applicat

Easy Apply

Contract, Third Party

Depends on Experience

Lead AI Engineer with Spark, AWS Services

Remote

•

Today

We're looking for a Lead AI Engineer skilled in Spark and AWS Services to become part of the RBQM Production Pod within the program. In this position, you'll construct and sustain data pipelines that drive AI/GenAI applications supporting Risk-Based Quality Management for clinical trials. The primary focus of this role includes RAG document ingestion, vector indexing, and developing data APIs for AI applications. Responsibilities Architect and construct RAG document ingestion pipelines (chunking

Full-time

Senior Data Software Engineer/ Databricks, Apache Spark, PySpark

Remote

•

Today

We are seeking a skilled Senior Data Software Engineer with strong expertise in PySpark , SQL , and unit testing to join our data engineering team. The ideal candidate will have hands-on experience with Apache Spark , preferably within the Databricks environment, and will be responsible for building scalable data pipelines, optimizing data workflows, and ensuring code quality through rigorous testing practices. Responsibilities Design, develop, and maintain scalable data pipelines using Apache S

Full-time

Senior Data Software Engineer/ AI Agents, Azure, Spark

Remote

•

Today

We are looking for an experienced Senior Data Software Engineer with strong hands-on expertise in Azure and the Spark ecosystem , primarily focused on building and maintaining data transformation pipelines. The candidate should be comfortable with any Spark-adjacent technology (PySpark, Spark SQL, Scala Spark, Databricks, Synapse, etc.) rather than being locked into one specific flavor. Experience with Microsoft Fabric is a strong plus but not a requirement. The ideal candidate combines great te

Full-time

Senior AI Engineer with Spark, AWS Services

Remote

•

Today

We are seeking a Senior AI Engineer with Spark and AWS Services expertise to join the RBQM Production Pod within the program. You will build and maintain data pipelines that power AI/GenAI applications for Risk-Based Quality Management in clinical trials. This role focuses on RAG document ingestion, vector indexing, and building data APIs for AI applications. Responsibilities Design and build RAG document ingestion pipelines (chunking, embedding, vector indexing) for clinical trial quality data

Full-time

Python AWS Spark Engineer, Plano, TX (Hybrid)

Hybrid in Plano, Texas

•

16d ago

Python AWS Spark Engineer Location: Plano, TX Duration: Long-TermJob SummarySeeking an experienced Python AWS Spark Engineer to join the team in Plano, TX. The ideal candidate will have strong hands-on experience with Python, Apache Spark/PySpark, AWS cloud services, and data engineering. This role will focus on developing scalable data processing solutions, building cloud-based data pipelines, and supporting enterprise data platforms. Key ResponsibilitiesDesign, develop, and maintain scalable d

Easy Apply

Contract

Depends on Experience

Senior AWS Python Developer

Hybrid in Lafayette, Louisiana

•

Today

Job Title: Senior AWS Python Developer Location: Lafayette, LA Knoxville, TN Columbia, SC Birmingham, AL Type: Direct Hire Work Model: Hybrid Overview We are looking for a Senior Python + PySpark Data Engineer with strong AWS big data experience ideally someone who has worked in cybersecurity or security data environments. This is primarily a hands-on data engineering role, not a traditional Python application developer role. Responsibilities Developing scalable Python applications and dat

Easy Apply

Full-time

Depends on Experience

Remote Data Engineer (Snowflake, Spark/Scala)

Remote or Jacksonville, Florida

•

Today

Software Guidance & Assistance, Inc., (SGA), is searching for a Remote Data Engineer (Snowflake and Spark/Scala) for a contract assignment with one of our premier Healthcare Services clients in Jacksonville, FL . This role will be remote. Top Skills Needed: Snowflake Spark/Scala Responsibilities : Development, programming, coding of Information Technology solutions. Documenting detailed system specifications, participation in unit testing and maintenance of planned and unplanned internally dev

Contract

USD 50.00 - 55.00 per hour