Apache spark jobs in new york

Berkeley Heights, New Jersey

•

3d ago

Job Description 10+ years solid experience with deploying, configuring, monitoring, and troubleshooting Apache Spark clusters and associated big data frameworks (such as Hadoop, Kafka, or cloud-based data platforms) to ensure high availability and performance.Focuses on cluster health, performance tuning, and production data pipelines.Technical Experience: Deep hands-on experience with Apache Spark architecture (Driver, Workers, Executors, RDDs/DataFrames) and cluster managers (YARN, Kubernetes,

Easy Apply

Full-time

$100,000 - $120,000

Databricks Azure Consultant with Apache Spark- Onsite NJ- and - no corps

Remote or Jersey City, New Jersey

•

Today

Please contact Renu Goel Location: NJ - No corps Must Haves: -Azure -Databricks -Apache Spark -CI/CD Databricks / Azure Consultant Job Summary Seeking two experienced Databricks / Azure Consultants to support the design, implementation, and ongoing development of the Databricks data platform. The consultants will work with technology and data teams to build and enhance scalable data solutions using Databricks and Microsoft Azure, with a strong focus on data engineering, integration, securit

Easy Apply

Full-time

Competitive

Scala / Apache Spark Developer

Hybrid in Jersey City, New Jersey

•

Today

Job Role: Scala / Apache Spark Developer (Onsite or Hybrid or Remote): Hybrid 3 days (Relocation allowed or Not allowed): Need locals either from NY/NJ Contract Duration: 11+ months Interview Mode: In-person at client location Position Overview The ideal candidate will have solid programming skills, experience developing scalable data processing solutions, and the ability to collaborate with technical teams in a fast-paced environment. Key Responsibilities Design, develop, and maintain applicat

Easy Apply

Contract, Third Party

Depends on Experience

Apache Spark/Core Java Developer

Bayonne, New Jersey

•

Today

Location: Bayonne, NJ Salary: $140,000.00 USD Annually - $145,000.00 USD Annually Description: Apache Spark/core Java Developer New York, NY 10110 Fulltime Final round F2F in NYC Core java, Apache Spark, Python, DB SQL, Linux, Application/ server-side development. Hands-on experience designing, building, and integrating analytical systems. Experience - 5-10 years Strong programming experience in Core Java Hands-on expertise in Apache Spark Proficiency in Python Strong knowledge of SQL and Data

Full-time

USD 140,000.00 - 145,000.00 per year

Senior AI Engineer, Databricks, ML, Spark ML

Remote

•

Today

We are looking for a Senior AI Engineer to build, test, and deploy ML and GenAI components for asset onboarding, industrial diagnostics, and device analytics on Databricks. This specialist will create predictive models, anomaly-detection algorithms, RAG pipelines, document extraction, and AI agent capabilities. Responsibilities Construct, test, and deploy ML and GenAI components on Databricks Create predictive models for asset onboarding and device analytics Architect and implement anomaly-detec

Full-time

Lead Data Software Engineer with Databricks with Apache Kafka, Apache Spark, Kubernetes

Remote

•

Today

We are looking for a Lead Data Software Engineer to drive the migration of an existing data analytics platform, originally built on Azure resources such as Data Factory, Databricks, Event Hub, and Cassandra, to a cloud-agnostic platform deployable on-premises. Responsibilities Implement streaming and batch Spark pipelines running on Kubernetes Lead the implementation of objects within the data generator framework Oversee deployment and testing across local and development environments Drive impr

Full-time

Lead AI Engineer with Spark, AWS Services

Remote

•

Today

We're looking for a Lead AI Engineer skilled in Spark and AWS Services to become part of the RBQM Production Pod within the program. In this position, you'll construct and sustain data pipelines that drive AI/GenAI applications supporting Risk-Based Quality Management for clinical trials. The primary focus of this role includes RAG document ingestion, vector indexing, and developing data APIs for AI applications. Responsibilities Architect and construct RAG document ingestion pipelines (chunking

Full-time

Senior Data Software Engineer/ Databricks, Apache Spark, PySpark

Remote

•

Today

We are seeking a skilled Senior Data Software Engineer with strong expertise in PySpark , SQL , and unit testing to join our data engineering team. The ideal candidate will have hands-on experience with Apache Spark , preferably within the Databricks environment, and will be responsible for building scalable data pipelines, optimizing data workflows, and ensuring code quality through rigorous testing practices. Responsibilities Design, develop, and maintain scalable data pipelines using Apache S

Full-time

Senior Data Software Engineer/ AI Agents, Azure, Spark

Remote

•

Today

We are looking for an experienced Senior Data Software Engineer with strong hands-on expertise in Azure and the Spark ecosystem , primarily focused on building and maintaining data transformation pipelines. The candidate should be comfortable with any Spark-adjacent technology (PySpark, Spark SQL, Scala Spark, Databricks, Synapse, etc.) rather than being locked into one specific flavor. Experience with Microsoft Fabric is a strong plus but not a requirement. The ideal candidate combines great te

Full-time

Senior AI Engineer with Spark, AWS Services

Remote

•

Today

We are seeking a Senior AI Engineer with Spark and AWS Services expertise to join the RBQM Production Pod within the program. You will build and maintain data pipelines that power AI/GenAI applications for Risk-Based Quality Management in clinical trials. This role focuses on RAG document ingestion, vector indexing, and building data APIs for AI applications. Responsibilities Design and build RAG document ingestion pipelines (chunking, embedding, vector indexing) for clinical trial quality data

Full-time

Database Engineer

New York, New York

•

6d ago

A client of Innova Solutions is immediately hiring for a Database Engineer Position type: Full-time Contract (W2) Role: Database Engineer Duration: 4+ months Contract Location : New York, NY Remote role Must have Skills Python, Spark / PySpark, RESTful APIs, API versioning, input validation, pagination, and error standards in Python using FastAPI / Django / Flask, Airflow, Dagster, or similar Primary Skill Required for the Role: 5+ years of experience in Data Engineering or equivalent comb

Easy Apply

Contract

Depends on Experience

Remote Data Engineer (Snowflake, Spark/Scala)

Remote or Jacksonville, Florida

•

Today

Software Guidance & Assistance, Inc., (SGA), is searching for a Remote Data Engineer (Snowflake and Spark/Scala) for a contract assignment with one of our premier Healthcare Services clients in Jacksonville, FL . This role will be remote. Top Skills Needed: Snowflake Spark/Scala Responsibilities : Development, programming, coding of Information Technology solutions. Documenting detailed system specifications, participation in unit testing and maintenance of planned and unplanned internally dev

Contract

USD 50.00 - 55.00 per hour

Lead Software Engineer - Python and Spark

New York, New York

•

Yesterday

Job Description We have an opportunity to impact your career and provide an adventure where you can push the limits of what's possible. As a Lead Software Engineer at JPMorgan Chase within the Consumer and Community Banking's Technology team, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way. As a core technical contributor, you are responsible for conducting critical technology sol

Full-time

Lead Software Engineer - Python and Spark

New York, New York

•

Today

Full-time

Senior Software Development Engineer, Consumer Domains

Newark, New Jersey

•

Today

At Audible, we believe stories have the power to transform lives. It's why we work with some of the world's leading creators to produce and share audio storytelling with our millions of global listeners. We are dreamers and inventors who come from a wide range of backgrounds and experiences to empower and inspire each other. Imagine your future with us. ABOUT THIS ROLE This opportunity is for a Senior Software Development Engineer for Audible's Consumer Domains group. We are the team that create

Full-time

Remote Sr. IT Developer with Hadoop & Spark

Remote or Jacksonville, Florida

•

Today

Software Guidance & Assistance, Inc., (SGA), is searching for a Sr. IT Developer with Hadoop & Spark for a contract assignment with one of our premier Healthcare Services clients in Jacksonville, FL . This role will be remote. Top Experience Needed: -HBASE/Scala/Spark development required (all three) -Experience using Hadoop Distributed File System (HDFS) required -Development experience with Kafka required -Data warehousing experience required Responsibilities : Responsible for the design, d

Contract

USD 62.86 - 67.86 per hour

Data Engineer / Lead Engineer / Architect

Hybrid in New York, New York

•

Today

Job SummaryWe are seeking a hands-on Lead Data Architect to lead the US architecture function for a multi-year data platform modernization program for a leading global financial services client. The initiative focuses on migrating on-premises Hadoop workloads to Snowflake. The ideal candidate will own the target-state architecture, lead migration strategy and solution design, collaborate with senior client stakeholders, and guide an India-based delivery team through an AI-assisted migration appr

Easy Apply

Full-time, Third Party

$160,000 - $200,000

iOS Engineer - Apple Sports

New York, New York

•

Today

Apple is where individual imaginations come together to build innovative products, services, and experiences that enrich lives worldwide. By fostering a culture of collaboration and diversity, we strengthen each other's ideas, unlocking innovation that runs through everything we do. Join us, and you'll do more than be part of something - you'll add something. Description Apple Sports is looking for an iOS Engineer to help build a world-class sports experience for fans everywhere. Like any grea

Full-time

Data Platform Architect

Hybrid in Jersey City, New Jersey

•

28d ago

Role: Data Platform Architect (DataBricks) Location: Jersey City, New Jersey or New York City (3 days Onsite Mandatory) Rate: $75/hr on W2 Duration: 12+ months CTH MUST HAVE: Large-scale data migration and modernization experience - greenfield large data platform implementation preferred Expertise in Medallion Architectures Big data movement and transport Data warehousing fundamentals - schema design (star/snowflake), fact and dimensions, SCDs Spark Kafka AWS, Azure, or Google Cloud Platform J

Easy Apply

Contract, Third Party

Depends on Experience

Data Engineer

Remote or New York, New York

•

Yesterday

RESPONSIBILITIES: Kforce's client in NYC is seeking a highly skilled Data Engineer with strong hands-on experience in Apache Spark/PySpark, AWS EMR, Databricks, and Snowflake. The Data Engineer will be responsible for designing and building scalable, high-volume data pipelines, optimizing distributed data processing workloads, implementing data quality and observability frameworks, and resolving complex performance and production challenges across cloud-based data platforms. REQUIREMENTS: * Str

Contract

$74 - $74 hourly

Apache spark jobs in New York, NY, USA