lead data engineer python/scala, spark, aws Jobs in san francisco, ca

Refine Results
1 - 20 of 63 Jobs

Lead Data Engineer

Exl Service

•

Fremont, California, USA

Full-time

Hi, We have urgent requirements for our direct client, please go through the below Job Description. If you are interested please send me your updated word format resume to and reach me @ . Role: Lead Data Engineer Location: Fremont, CA (3 days onsite, 2 days work from home) Duration: Full Time Experience range: 12-15 years Job Description: Candidate should Provide technical expertise in needs identification, data modelling, data movement, translating business needs into technical solutions wit
Easy Apply
Posted 7 days ago

Staff Software Engineer, Batch Processing Platform

Pinterest, Inc.

•

Remote or San Francisco, California, USA

Full-time

About Pinterest: Millions of people around the world come to our platform to find creative ideas, dream about new possibilities and plan for memories that will last a lifetime. At Pinterest, we're on a mission to bring everyone the inspiration to create a life they love, and that starts with the people behind the product. Discover a career where you ignite innovation for millions, transform passion into growth opportunities, celebrate each other's unique experiences and embrace the flexibility
Posted 60+ days ago | Updated 5 hours ago

Software Engineer II - GTM Data

Uber Corporate

•

San Francisco, California, USA

Full-time

About the Role Your expertise in software engineering, data systems, and data governance will play a critical role in building scalable and efficient data platforms and tools while ensuring data integrity and compliance. This is a unique opportunity to make a significant impact on our organization, contribute to the success of our team What the Candidate Will Need / Bonus Points \\-\\-\\-\\- What the Candidate Will Do 01. Solution Development: Collaborate with stakeholders to understand bus
Posted 18 days ago | Updated 5 hours ago

Databricks Architect

Buaut

•

San Francisco, California, USA

Contract

Key ResponsibilitiesDesign and implement end-to-end data architecture on Databricks (Spark, Delta Lake, MLflow). Develop and optimize large-scale ETL/ELT pipelines using PySpark/SQL. Architect data lakes and lakehouses integrating cloud storage systems (e.g., ADLS, S3, GCS). Define and enforce best practices around security, data governance, and cost optimization. Lead technical workshops and collaborate with data engineers, data scientists, and DevOps. Implement data orchestration workflows (e.
Easy Apply
Posted 39 days ago | Updated 1 day ago

Senior Machine Learning Engineer

Strava, Inc.

•

San Francisco, California, USA

Full-time

About This Role Strava is the app for active people. With over 150 million athletes in more than 185 countries, Strava is where connection, motivation, and personal bests thrive. No matter your activity, gear, or goals, we help you find your crew, crush your milestones, and keep moving forward. Start your journey with Strava today. Our mission is simple: to motivate people to live their best active lives. We believe in the power of movement to connect and drive people forward. We are looking
Posted 60+ days ago | Updated 5 hours ago

Software Engineer, Backend, AI Tooling

Tesla Motors

•

Palo Alto, California, USA

Full-time

As a member of the Autopilot AI Tooling team you will design and implement a diverse set of tools that power our machine learning workflows. You will work closely with some of the best, world-class AI researchers and machine learning experts towards Tesla's goal of a self-driving future. The AI Tooling team is a central part of Autopilot AI. The systems you build will have a large impact on the entire lifecycle of model development. This includes data processing, data discovery, annotation, and
Posted 21 days ago | Updated 5 hours ago

Staff Machine Learning Engineer

Strava, Inc.

•

San Francisco, California, USA

Full-time

About This Role Strava is the app for active people. With over 150 million athletes in more than 185 countries, it's more than tracking workouts-it's where connection, motivation, and personal bests thrive. No matter your activity, gear, or goals, Strava's got you covered. Find your crew, crush your milestones, and keep moving forward. Start your journey with Strava today. Our mission is simple: to motivate people to live their best active lives. We believe in the power of movement to connect
Posted 60+ days ago | Updated 5 hours ago

Data Engineer - Scala

Exl Service

•

Remote

Full-time

We will only be able to move forward with candidates who have ship or hold active s for this role What you'll Do: Create and maintain optimal data pipeline architectureBuild data pipelines that transform raw, unstructured data into formats that data analyst can use to for analysisAssemble large, complex data sets that meet functional / non-functional business requirements Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-des
Easy Apply
Posted 25 days ago | Updated 25 days ago

Big Data Engineer - W2

Pyramid Technology Solutions, Inc.

•

San Francisco, California, USA

Contract

Title: Senior Big Data Engineer Location: San Francisco, CA - Hybrid Duration: 12 Months Contract Type: Only W2 Mandatory Skills: Either Java or Scala, SQL and NoSQL, Apache Spark, Cloud Platforms Data Engineer Responsibilities: System development and maintenance activities of the team to meet service level agreements and create solutions with innovation, cost effectiveness, high quality and faster time to market.Support code versioning, and code deployments for Data Pipelines.Ensure test covera
Easy Apply
Posted 60+ days ago | Updated 7 days ago

Lead Data Engineer

Visa Inc.

•

Foster City, California, USA

Full-time

Company Description Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable, and secure payments network, enabling individuals, businesses, and economies to thrive while driven by a common purpose - to uplift everyone, everywhe
Posted 2 days ago | Updated 5 hours ago

Staff, Data Scientist - Customer Data & Identity

Walmart Inc.

•

San Bruno, California, USA

Full-time

Position Summary We are seeking a Staff Data Scientist with a strong foundation in machine learning, information retrieval, and LLMs. The selected candidate will be responsible for architecting and developing new Machine Learning models to enhance Customer Segmentation landscape within Walmart. This is an exciting opportunity for a creative and innovative professional who thrives in a fast-paced, results-driven environment. What you'll do Develop generative AI LLM-based applications, , multi-ag
Posted 60+ days ago | Updated 5 hours ago

Staff Software Engineer, Airbnb

Airbnb

•

San Francisco, California, USA

Full-time

Airbnb was born in 2007 when two hosts welcomed three guests to their San Francisco home, and has since grown to over 5 million hosts who have welcomed over 2 billion guest arrivals in almost every country across the globe. Every day, hosts offer unique stays and experiences that make it possible for guests to connect with communities in a more authentic way. The Community You Will Join: Join our innovative team as a Distributed Systems Engineer, where you'll play a pivotal role in designing an
Posted 57 days ago | Updated 4 hours ago

Druid Engineer

Biztec Global LLC

•

Remote

Full-time

Data Engineer, Distributed Systems Druid Engineer Remote or Austin, Texas KEY RESPONSIBILITIES Design and Development:o Design and implement data ingestion pipelines to load data into Apache Druid.o Develop and optimize Druid schemas and data models for efficient querying and performance.o Design and implement data aggregations against event-level data in distributed database (preferably in Druid)o Migration from existing data stores to Druid Optimization:o Implement best practices for data in
Easy Apply
Posted 13 days ago

Gen AI Software Engineer

Judge Group, Inc.

•

Concord, California, USA

Contract

Location: Concord, CA Salary: $79.00 USD Hourly - $85.00 USD Hourly Description: Job Title: Software Engineer - AI/ML & Distributed Systems (Contingent Assignment) Location: Concord, CA (Hybrid/On-site) Job Type: Contingent Resource Assignment About the Role We are seeking a highly skilled Software Engineer to support complex, large-scale software engineering initiatives with a focus on AI/ML, distributed systems, and cloud infrastructure. In this role, you will consult on strategic project
Posted 22 days ago | Updated 1 hour ago

Lead II Software Engineer - ThousandEyes

Cisco Systems, Inc.

•

San Francisco, California, USA

Full-time

About The Role The Application Window is expected to close on 7/9/25. However, the job posting may be removed earlier if the position is filled or if a sufficient number of applications are received. As a Principal Engineer at ThousandEyes, you will play a pivotal role in steering the architectural and design direction for a wide array of projects, encompassing AI/ML workloads, on-premises solutions, and cloud-based agents and applications. With the accelerating growth in application connectivi
Posted 60+ days ago | Updated 5 hours ago

Gen AI Software Engineer

Judge Group, Inc.

•

Concord, California, USA

Contract

Location: Concord, CA Salary: $79.00 USD Hourly - $85.00 USD Hourly Description: Job Title: Software Engineer - AI/ML (Contingent Resource) Location: Concord, CA Employment Type: Contract / Contingent About the Role: We are seeking a highly skilled Software Engineer with deep expertise in AI/ML and distributed systems to join our team on a contingent basis. In this role, you will consult on complex, high-impact software engineering initiatives, contributing to the design, development, and d
Posted 22 days ago | Updated 1 hour ago

Staff AI/ML Software Engineer

ServiceNow, Inc.

•

Remote or Santa Clara, California, USA

Full-time

Company Description It all started in sunny San Diego, California in 2004 when a visionary engineer, Fred Luddy, saw the potential to transform how we work. Fast forward to today - ServiceNow stands as a global market leader, bringing innovative AI-enhanced technology to over 8,100 customers, including 85% of the Fortune 500 . Our intelligent cloud-based platform seamlessly connects people, systems, and processes to empower organizations to find smarter, faster, and better ways to work. But thi
Posted 2 days ago | Updated 5 hours ago

Google Cloud Platform Data Engineer

Intellect Quest LLC

•

Remote

Contract, Third Party

Role : Google Cloud Platform Data EngineerLocation: Remote Description: We are seeking a highly motivated and talented Data Engineer to join our dynamic team. As a Data Engineer, you will play a critical role in designing, developing, and implementing data pipelines and data integration solutions using Spark, Scala, Python, and Google Cloud Platform (Google Cloud Platform). You will be responsible for building scalable and efficient data processing systems, optimizing data workflows, and ensur
Easy Apply
Posted 20 days ago

Software Engineering Architect - Marketing Automation

Salesforce

•

San Francisco, California, USA

Full-time

To get the best candidate experience, please consider applying for a maximum of 3 roles within 12 months to ensure you are not duplicating efforts. Job Category Software Engineering Job Details About Salesforce We're Salesforce, the Customer Company, inspiring the future of business with AI+ Data +CRM. Leading with our core values, we help companies across every industry blaze new trails and connect with customers in a whole new way. And, we empower you to be a Trailblazer, too - driving you
Posted 60+ days ago | Updated 5 hours ago

Lead Software Engineer - AI Applications

Salesforce

•

Palo Alto, California, USA

Full-time

To get the best candidate experience, please consider applying for a maximum of 3 roles within 12 months to ensure you are not duplicating efforts. Job Category Software Engineering Job Details About Salesforce We're Salesforce, the Customer Company, inspiring the future of business with AI+ Data +CRM. Leading with our core values, we help companies across every industry blaze new trails and connect with customers in a whole new way. And, we empower you to be a Trailblazer, too - driving you
Posted 24 days ago | Updated 5 hours ago