Apache Spark jobs in Pennsylvania

Remote

•

Today

Python Engineer Remote 6 month contract to hire Overview: At a high level, they have migrated from Hadoop to Google Cloud Platform for data processing. Have a Google Cloud Platform data environment, predominantly for big data applications on the cloud. Seeking 3-5 Senior Level Data Engineers with strong Python skills to support ongoing data migration and ingestion efforts. Source systems: get data from multiple external channels - provider data, healthcare groups, hospitals, etc. send data and t

Easy Apply

Contract

Depends on Experience

Senior AI Engineer, Databricks, ML, Spark ML

Remote

•

Today

We are looking for a Senior AI Engineer to build, test, and deploy ML and GenAI components for asset onboarding, industrial diagnostics, and device analytics on Databricks. This specialist will create predictive models, anomaly-detection algorithms, RAG pipelines, document extraction, and AI agent capabilities. Responsibilities Construct, test, and deploy ML and GenAI components on Databricks Create predictive models for asset onboarding and device analytics Architect and implement anomaly-detec

Full-time

Remote Data Engineer (Snowflake, Spark/Scala)

Remote or Jacksonville, Florida

•

Today

Software Guidance & Assistance, Inc., (SGA), is searching for a Remote Data Engineer (Snowflake and Spark/Scala) for a contract assignment with one of our premier Healthcare Services clients in Jacksonville, FL . This role will be remote. Top Skills Needed: Snowflake Spark/Scala Responsibilities : Development, programming, coding of Information Technology solutions. Documenting detailed system specifications, participation in unit testing and maintenance of planned and unplanned internally dev

Contract

USD 50.00 - 55.00 per hour

W2 Senior Data Engineer - Telecom Background, Python, BigQuery, Looker, Tableau, Power BI, Spark (PySpark)

Remote

•

Today

Senior Data Engineer - lead the technical delivery of a data and analytics program for a large enterprise telecom customer. You'll own the work from ingestion through to dashboards. You'll be the main technical contact with the customer's data and engineering teams, and you'll work with a distributed team that uses AI-assisted development every day. This is a 6 months contract with a potential to extend. Please note that candidates must be based in the United States and available to travel to c

Easy Apply

Contract

55 - 67

Remote Sr. IT Developer with Hadoop & Spark

Remote or Jacksonville, Florida

•

Today

Software Guidance & Assistance, Inc., (SGA), is searching for a Sr. IT Developer with Hadoop & Spark for a contract assignment with one of our premier Healthcare Services clients in Jacksonville, FL . This role will be remote. Top Experience Needed: -HBASE/Scala/Spark development required (all three) -Experience using Hadoop Distributed File System (HDFS) required -Development experience with Kafka required -Data warehousing experience required Responsibilities : Responsible for the design, d

Contract

USD 62.86 - 67.86 per hour

Business Analyst - Detroit, MI, Madison, WI, Columbus, OH, Chicago, IL, Minneapolis, MN.

Remote or Detroit, Michigan

•

Today

Job Title: Business Analyst Location: Detroit, MI, Madison, WI, Columbus, OH, Chicago, IL, Minneapolis, MN. Contact: 12+ Months Looking for W2 candidates. No C2C Job Summary: We are seeking an experienced Business Analyst with strong expertise in Distributed Technologies, Web Services, Big Data, SQL, Python, and Financial Risk. The ideal candidate will collaborate with business stakeholders and technology teams in an Agile environment to analyze requirements, improve application processes, enha

Easy Apply

Contract

Depends on Experience

Databricks Engineer

Remote

•

7d ago

Employment Type: W2 Only Desired Skills Experience with the Databricks Lakehouse Platform Strong knowledge of Apache Spark, PySpark, Python, SQL, or Scala Experience developing and optimizing data pipelines Knowledge of Delta Lake, Delta Live Tables, Unity Catalog, and Databricks Workflows Experience with data warehousing, data lakes, ETL, and ELT processes Understanding of data governance, security, access controls, and platform administration Experience integrating Databricks with AWS, M

Easy Apply

Contract, Third Party

$70 - $80

Data Architect

Remote

•

18d ago

About the Company Blackstraw.ai is an end-to-end technology services company specializing in Artificial Intelligence (AI) and Engineering solutions across Data Science, Data Engineering, LLM/GenAI and LLMOps. Founded in 2018, we help global enterprises across North America, Europe and Asia to build and operationalize AI systems that create measurable business impact. Our mission is to make AI adoption simpler, faster and scalable through a blend of deep domain expertise, reusable accelerators an

Easy Apply

Full-time

Depends on Experience

Databricks Engineer with Health Insurance

Remote

•

Today

Over 8 Years of experience in IT. Design and implement end-to-end lifecycle of Databricks platform on AWS.Good experience in working with Healthcare insurance clients.Design and implement different environments like Dev/SIT/UAT/PROD for Databricks.Expertise in AWSImplement data access controls, Lineage and auditing using Unity catalog.Implement CI/CD pipelines between different environments.Hands-on architecture experience within AWS, Azure, or Google Cloud Platform environments.Deep expertise i

Easy Apply

Contract

Depends on Experience

Databricks Engineer

Remote

•

Today

Easy Apply

Contract

Depends on Experience

Forward Deployed Engineer (Databricks)

Remote

•

11d ago

Senior Forward Deployed Engineer, Data & AI We''re looking for a hands-on engineer who thrives working side-by-side with clients to solve real-world data problems. You''ll be involved with technical delivery end-to-end building the data pipelines that move information from raw to ready, and the models that turn it into insight. What you''ll do: Partner directly with clients and product teams, translating business problems into technical requirements (and technical solutions back into plain-lang

Easy Apply

Contract

65 - 80

Senior Databricks Engineer

Remote

•

Today

Position: Senior Databricks Data Engineer Location: Remote / Hybrid Duration: Long Term Responsibilities Develop scalable data pipelines using Databricks, PySpark and SQL.Build and maintain Delta Lake tables using the Medallion Architecture.Develop batch and real-time data processing pipelines.Implement data ingestion using Auto Loader, APIs, Kafka or cloud storage.Optimize Spark jobs, SQL queries and Delta tables.Implement data governance and security using Unity Catalog.Develop Databricks Work

Easy Apply

Contract, Third Party

Depends on Experience

Principal Data Engineer – Data Platform - Atlanta, GA, Charlotte, NC, Raleigh, NC, Durham, NC, Columbia, SC

Remote or Atlanta, Georgia

•

Today

Job Title: Principal Data Engineer Data Platform Location: Atlanta, GA, Charlotte, NC, Raleigh, NC, Durham, NC, Columbia, SC. Looking for W2 candidates. No C2C Job Summary We are looking for a Principal Data Engineer to architect enterprise-scale data platforms supporting AI, Machine Learning, analytics, and real-time data processing. The ideal candidate should possess deep expertise in distributed systems, modern data architecture, and cloud-native engineering. Key Responsibilities Architect e

Easy Apply

Contract

Depends on Experience

Lead Sr Software Engineer (Cloudera Hadoop stack)

Remote

•

4d ago

Lead Sr Software Engineer (Cloudera Hadoop stack) REMOTE Visa Eligibility:Please confirm if the candidate is legally authorized to work in the United States Experience Level: 10+ years Contract 12 months Qualifications: Bachelors degree in computer science, Information Technology, Information Systems, or related field. 10+ years of experience with enterprise and application design patterns that are applied on Hadoop and Big data platforms in aws. The ideal candidate will possess deep expertise

Easy Apply

Third Party, Contract

60 - 70

Iceberg DBA REMOTE

Remote

•

4d ago

Interview Mode :MS teams Must have :Iceberg DBA experience in Big Data / Data Operations / DBA roles Minimum 1+ year of experience with Apache Iceberg or similar table formats (Hive/Delta/Hudi) 4+ years of experience with Cloudera ecosystem (CDP)Hands-on experience with: Iceberg table operations and maintenance Spark SQL, Hive, or Impala Experience in: Production support and incident handling Monitoring, troubleshooting, and operational support Apply established data modeling and Lakehouse stand

Contract

$40 - $50

AI & Data Architect (Databricks | Lakehouse | AI)

Remote

•

6d ago

Role: AI & Data Architect (Databricks | Lakehouse | AI) Location:Indianapolis, IN (Remote but 1-2 days a month travel to client's office in Indianapolis) About the role Define scalable enterprise data architectures that support Analytics, AI, and GenAI initiatives. Translate business and data challenges into scalable architectures and new engineering initiatives. Responsibilities Design enterprise data architectures on Databricks and the LakehouseBuild ETL/ELT, data integration, and scalable pla

Easy Apply

Contract

Depends on Experience

Databricks Data Engineer / Business Analyst - Open Contracting Data Standard (OCDS)

Remote

•

3d ago

Position Summary Scicom Infrastructure Services is seeking aData Engineer / Business Analystto support a large-scale Databricks implementation. This hybrid role will bridge business, contracting, data-governance, and engineering teams to define business requirements, construct data contracts, map source data, and develop reliable Databricks data pipelines. The successful candidate must have hands-on experience with theOpen Contracting Data Standard (OCDS)and understand how contracting and procur

Easy Apply

Contract

Depends on Experience

Databricks Architect / Resident Solution Architect

Remote or Hybrid in San Francisco, California

•

5d ago

Job Title: Databricks Architect / Resident Solution Architect Openings: 2 Location: San Francisco, CA Local preferred; Remote considered Work Hours: Must be able to work PST hours Duration: 812 Months Visa: Any Rate: flexible Industry: Banking / Financial Services Experience: 1215+ Years Job DescriptionWe are looking for a highly experienced Databricks Architect / Resident Solution Architect with strong hands-on Data Engineering and Databricks architecture experience. The ideal candidate should

Easy Apply

Full-time

Depends on Experience

Cloud Computing / Data Engineer - Entry Level

Remote

•

26d ago

We are seeking a motivated and detail-oriented Entry-Level Cloud Computing / Data Engineer to join our technology team. The ideal candidate will assist with developing data pipelines, managing cloud-based data solutions, processing datasets, and supporting cloud infrastructure and data engineering projects. This is an entry-level position and does not require senior-level experience. Candidates with 02 years of experience, entry level candidates, internships, academic projects, bootcamp training

Easy Apply

Contract

20 - 25

Databricks Solutions Architect

Remote

•

3d ago

Databricks Solutions Architect 100% Remote 12 months plus Core Responsibilities Some of the major responsibilities for an RSA are: Engage with customers on short- to mid-term professional services projects using the Databricks platform (data engineering, data science, cloud). Design and build reference architectures, help with production-level use cases and deployments of big data / AI applications. Collaborate with engagement/project managers, customer teams, and internal engineering/support te

Easy Apply

Contract

Depends on Experience

Apache Spark jobs in Pennsylvania, USA