Hadoop PySpark Data Pipeline Build Engineer jobs

Hadoop/Data Platform Operations Engineer

Charlotte, North Carolina

•

Today

Job Title: Hadoop/Data Platform Operations Engineer Location: Charlotte, NC Duration: Contract - 11 months Pay Range: $53.55/hr (W2) Job ID: 409100 About BCforward BCforward is a leading global IT consulting and workforce solutions firm providing services and support to Fortune 500 and government clients. Founded in 1998, BCforward has grown with our customers needs into a full-service business solutions provider. With delivery centers and offices across North America and India, we take pri

Easy Apply

Contract

$53.00 - $53.55 per hour

Hadoop Data Engineer

Hybrid in Dallas, Texas

•

27d ago

Sr. Feature Engineer Dallas,TX/ Pittsburgh, PA / Clevland OH Tech: Data Engineering/Pipeline MLOps Engineering/Pipeline OpenShift Git Linux Programming language: Python SQL Spark Hive Title Job Description Skillsets Typical Experience Feature Engineer (Sr) Design and implement scalable, reusable feature pipelines (batch and real-time)Develop complex feature transformations and advanced data modeling logicOptimize feature performance, latency, and cost efficiencyEnsure feature quality, validati

Easy Apply

Full-time

$140,000 - $160,000

Data Engineer - AWS & PySpark

Dallas, Texas

•

Today

Data Engineer AWS & PySpark Location: Dallas, TX or Richmond, VA Client: Confidential Job Summary We are looking for an experienced Data Engineer with strong hands-on expertise in AWS and PySpark. The ideal candidate must have prior professional experience working with Banking domain and be capable of developing scalable data pipelines and data processing solutions in a cloud environment. Key Responsibilities Design, develop, and maintain scalable data pipelines using PySpark and AWS. Develo

Easy Apply

Contract

Senior Data Engineer - PySpark & Python

Charlotte, North Carolina

•

21d ago

i Associate, Greetings for the day, We have a position as a Senior Data Engineer - PySpark & Python role in Charlotte, NC. Please find the job description below for this position. Role: Senior Data Engineer - PySpark & Python Location: Charlotte, NC Duration: Fulltime Key Responsibilities: Design, develop, and optimise large-scale data pipelines using PySpark and Python.Implement and adhere to best practices in object-oriented programming to build reusable, maintainable code.Write advanced SQL q

Easy Apply

Contract

Depends on Experience

W2 ONLY -Data Engineer ( Databricks, Scala , PYSPARK)

Dallas, Texas

•

11d ago

Experienced Senior Data Engineer to work on a focused migration project converting existing Scala-based Spark pipelines to PySpark and transitioning job workloads from classic Databricks clusters to Databricks Serverless compute. The ideal candidate has hands-on experience with both Scala and PySpark, deep knowledge of Azure Databricks, and a strong understanding of Serverless compute constraints and best practices. Key ResponsibilitiesReview and analyze existing Scala Spark jobs identify depe

Easy Apply

Contract

Depends on Experience

Pyspark Google Cloud Platform data engineer (Onsite in NY)

Sleepy Hollow, New York

•

7d ago

Bachelor s or master s degree in computer science, Information Systems, or a related field.8+ years of data engineering experience (at least 2 years in financial services or regulated environments).Expert knowledge of BigQuery, Cloud Storage, Dataflow, and Pub/Sub on Google Cloud Platform.Strong experience with SQL, Spark, Python, PySparkand data orchestration tools (Airflow, Cloud Composer, Dagster).Demonstrated success designing modular, reusable data frameworks and libraries.Deep understandin

Easy Apply

Third Party, Contract

Depends on Experience

Sr. Associate Data Engineer (PySpark, Python & ETL)

Overland Park, Kansas

•

Today

McKesson is an impact-driven, Fortune 10 company that touches virtually every aspect of healthcare. We are known for delivering insights, products, and services that make quality care more accessible and affordable. Here, we focus on the health, happiness, and well-being of you and those we serve - we care. What you do at McKesson matters. We foster a culture where you can grow, make an impact, and are empowered to bring new ideas. Together, we thrive as we shape the future of health for patien

Full-time

USD 80,300.00 - 133,900.00 per year

Senior Consultant, Data Engineer (Python/PySpark/AWS)

Ashburn, Virginia

•

Today

*Candidates must be local to the Washington D.C. metro area. About Infinitive: Infinitive is a data and AI consultancy that enables its clients to modernize, monetize and operationalize their data to create lasting and substantial value. . We possess deep industry and technology expertise to drive and sustain adoption of new capabilities. We match our people and personalities to our clients' culture while bringing the right mix of talent and skills to enable high return on investment. Infiniti

Full-time

USD 90,000.00 per year

Platform Data Engineer - (DataBricks, PySpark, AWS)

West Chester, Pennsylvania

•

7d ago

Make your mark at Comcast -- a Fortune 30 global media and technology company. From the connectivity and platforms we provide, to the content and experiences we create, we reach hundreds of millions of customers, viewers, and guests worldwide. Become part of our award-winning technology team that turns big ideas into cutting-edge products, platforms, and solutions that our customers love. We create space to innovate, and we recognize, reward, and invest in your ideas, while ensuring you can prou

Full-time

Lead Data Engineer Pipeline Engineering Focus

Toronto, Ontario

•

Today

Lead Data Engineer - Pipeline Engineering Focus Client: Mastercard 121 Bloor st E, Toronto, Canada (onsite 5days) Position Overview We are looking for a Lead Data Engineer to build, operate, and scale the data pipelines and processing frameworks that power downstream analytics and data products. This role is execution-focused: the primary mandate is engineering excellence - reliable ETL/ELT, strong data quality practices, and operational rigor - rather than direct business stakeholder manage

Easy Apply

Third Party, Contract

Data Infrastructure Engineer (Hadoop)

New York, New York

•

Today

Data Infrastructure Engineer (Hadoop) New York City, NY Hybrid Schedule (M/F remote, T/W/TH in-office) At Magnite, we cultivate an environment of continuous growth and collaboration. Our work impacts what millions of people read, watch, and buy, and we're looking for people to help us tackle that responsibility with creativity and focus. Magnite (NASDAQ: MGNI) is the world's largest independent sell-side advertising platform. Publishers use our technology to monetize their content across all

Full-time

USD 170,000.00 - 180,000.00 per year

Senior Data Software Engineer with AWS, LLM, PySpark

Remote

•

Today

We are seeking a highly motivated Senior Data Software Engineer to design, build, and maintain modern cloud-based data platforms while enabling the next generation of AI and Copilot solutions. The ideal candidate will have strong expertise in AWS data ecosystems, PySpark, and experience developing AI-powered solutions using large language models and Generative AI applications. Responsibilities Design and implement scalable ETL/ELT pipelines Build and optimize data ingestion frameworks from APIs,

Full-time

Lead Software Engineer- Financial Services Data Engineering: Pyspark / Java / BigData / Datalake / AI

Jersey City, New Jersey

•

Today

Job Description We have an opportunity to impact your career and provide an adventure where you can push the limits of what's possible. As a Lead Software Engineer- Financial Service Data Engineering: Pyspark / Java / BigData / Datalake / AI, at JPMorganChase within the Asset and Wealth Management- Global Prime Brokerage Team, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way. As

Full-time

USD 156,750.00 - 215,000.00 per year

Senior AI Data Infrastructure/Pipeline Engineer

Santa Clara, California

•

Today

XPENG is a leading smart technology company at the forefront of innovation, integrating advanced AI and autonomous driving technologies into its vehicles, including electric vehicles (EVs), electric vertical take-off and landing (eVTOL) aircraft, and robotics. With a strong focus on intelligent mobility, XPENG is dedicated to reshaping the future of transportation through cutting-edge R&D in AI, machine learning, and smart connectivity. As a core member of our AI Infrastructure team, you will b

Full-time

USD 145,600.00 - 246,400.00 per year

Senior Staff AI Data Infrastructure/Pipeline Engineer

Santa Clara, California

•

Today

Full-time

USD 203,450.00 - 344,300.00 per year

Principal Scientist - Data Pipeline Engineer

San Jose, California

•

Today

ABOUT THE ROLE We're looking for a Principal ML Engineer to architect and scale the multimodal data processing pipelines and infrastructure behind Adobe Firefly's multimodal foundation models (image, video, audio). In this role, you'll sit at the intersection of data engineering and applied ML building distributed, GPU-accelerated systems that turn billions of raw assets into training-ready data at scale. Your work will directly determine how fast and how well Adobe models can learn directly imp

Full-time

USD 206,300.00 per year

Robotics Data Pipeline Engineer - Multimodal Data

Pensacola, Florida

•

Today

Job Title: Robotics Data Pipeline Engineer - Multimodal Data Department: Software Reports To: Teleoperations Lead Employment Type: Full-Time Location: Houston, TX or Pensacola Fl Who We Are Persona AI is building humanoid robots for the most demanding environments in heavy industry - shipyards, steel mills, fabrication facilities, and offshore platforms - performing welding, grinding, maintenance, inspection, and material-handling work that is dangerous, physically demanding, and increasin

Full-time

Oracle AI Data Pipeline Engineer

Reston, Virginia

•

Today

Job Number: R0246376 Oracle AI Data Pipeline Engineer The Opportunity: Everyone is trying to "harness the cloud," but not everyone knows how. As an Oracle AI Data Pipeline Engineer, you know how to create a cloud-based technical architecture that meets client needs and takes advantage of cloud AI capabilities. What if you could use your cloud architecture skills to improve mission-critical applications and products? We need you to help us develop cloud-based solutions for some of the nation's to

Full-time

USD 99,000.00 - 225,000.00 per year

Big Data Engineer – Python/Scala, Spark, Hadoop

Arlington, Virginia

•

11d ago

Role: Big Data Engineer Python/Scala, Spark, Hadoop Location: Arlington, VA (3 Days onsite/week) Exp Level : 6 to 12 Year Position : 2 Job Summary: Ability to easily move between business, data management, and technical teams; ability to quickly intuit the business use case and identify technical solutions to enable it Experience building robust and efficient data pipelines end-to-end with a strong focus on data quality High proficiency in using Python or Scala, Spark, Hadoop platforms & tools

Easy Apply

Full-time, Third Party

$120000 - $135000

Java J2EE, PySpark/Spark Engineer - Intermediate

Wilmington, Delaware

•

Today

Job Title: Software Engineering - Java J2EE - Intermediate Location: Wilmington, DE (Primary); Columbus, OH; Houston, TX; Plano, TX Duration: Contract - 6 months Pay Range: $64/hr -$68/hr (W2) Interview Process: 1st round Zoom, 2nd round in-person About BCforward BCforward is a leading global IT consulting and workforce solutions firm providing services and support to Fortune 500 and government clients. Founded in 1998, BCforward has grown with our customers needs into a full-service busine

Easy Apply

Contract

$66.00 - $69 per hour