Kafka/Spark Admin Jobs in San Francisco, CA

Refine Results
1 - 20 of 79 Jobs

Senior Data Engineer - Scala, Spark, Kafka, Azure

BayOne Solutions

•

Remote

Full-time

Job Title: Senior Data Engineer Job Type: Full Time Remote Eligible: Yes Skills: Scala programming, Spark, Kafka, Azure Responsibilities: Streamline the intake of the raw data into our Azure Data lake.Perform production support and deployment activities.Proactively drive the execution of our core data engineering, business intelligence, and data warehouse frameworksBuild data pipelines from systems such as CRM, Ecommerce etc. with the emphasis on scalability and reliabilityLeverage central data
Easy Apply
Posted 60+ days ago | Updated 6 days ago

Manager II, Engineering - Big Data Query Platform

Pinterest, Inc.

•

San Francisco, California, USA

Full-time

About Pinterest: Millions of people across the world come to Pinterest to find new ideas every day. It's where they get inspiration, dream about new possibilities and plan for what matters most. Our mission is to help those people find their inspiration and create a life they love. In your role, you'll be challenged to take on work that upholds this mission and pushes Pinterest forward. You'll grow as a person and leader in your field, all the while helping Pinners make their lives better in the
Posted 11 days ago | Updated 7 hours ago

Data Infrastructure Engineer

OpenAI

•

San Francisco, California, USA

Full-time

About the Team You'll join the team that's behind OpenAI's data infrastructure that powers critical engineering, product, alignment teams that are core to the work we do at OpenAI. The systems we support include our data warehouse, batch compute infrastructure, streaming infrastructure, data orchestration system, data lake, vector databases, critical integrations, and more. About the Role The Applied Data Platform team designs, builds, and operates the foundational data infrastructure that ena
Posted 53 days ago | Updated 7 hours ago

Data Engineer II

Strava, Inc.

•

San Francisco, California, USA

Full-time

About this role Strava is the leading digital community for active people with more than 120 million athletes, in more than 190 countries. The platform offers a holistic view of your active lifestyle, no matter where you live, which sport you love and/or what device you use. Everyone belongs on Strava when they are pursuing an active life. We are seeking data engineers to join our Data Platform team. Our vision is that key decisions and product at Strava can be greatly enriched with data to ben
Posted 9 days ago | Updated 7 hours ago

Software Engineer, Analytics Platform

OpenAI

•

San Francisco, California, USA

Full-time

About the Team The Research Platform Analytics team designs, builds, and operates the critical foundational data and analytics infrastructure that enables research at OpenAI. Our goal is one, and one only: accelerate the progress of research towards AGI. We do this by owning critical components of the research training stack, starting from key data processing pipelines and complex libraries that directly feed into our distributed training, followed by a variety of observability and analytics s
Posted 11 days ago | Updated 7 hours ago

Data Platform Engineer

Motion Recruitment Partners, LLC

•

San Francisco, California, USA

Full-time

Our large crypto company is looking for a contract Data Platform Engineer. This is a remote contract position. Contract Duration: 12-Months Required Skills & Experience 4+ years of relevant work experience. Fluent SQL skills for ETL/ELT, data quality, and reporting. Strong Python or Spark backend development skills. General experience working with Snowflake, Databricks or Trino. Experience with core AWS services and concepts (S3, IAM, ASG, RDS). Leverage your Python, Airflow, and Looker/Sup
Easy Apply
Posted 48 days ago | Updated 6 hours ago

Engineering Manager II, Uber Ads

Uber Corporate

•

San Francisco, California, USA

Full-time

About the Role As an Engineering Manager, you will be helping drive our fast-growing Uber Ads Engineering org! This Engineering Manager will lead our Data Platform and Reporting team. As an EM, You will lead a dedicated group of engineers in building a best-in-class data platform that ingests, stores, and aggregates data on billions of ad events across Uber Rides and Uber Eats monthly. You will play a pivotal role in crafting the future of our advertising analytics product and tech, which is ess
Posted 60+ days ago | Updated 7 hours ago

Senior Staff Software Engineer, Data and ML Platform

Discord Inc.

•

San Francisco, California, USA

Full-time

Discord is about giving people the power to create space to find belonging in their lives. We want to make it easier for you to talk regularly with the people you care about. We want you to build genuine relationships with your friends and communities close to home or around the world. Original, reliable, playful, and relatable. These are the values that connect our users and our employees at Discord. We are seeking an accomplished and experienced Senior Staff software Engineer to join our dynam
Posted 32 days ago | Updated 7 hours ago

Engineering Manager, Efficiency

Stripe

•

San Francisco, California, USA

Full-time

Who we are About Stripe Stripe is a financial infrastructure platform for businesses. Millions of companies-from the world's largest enterprises to the most ambitious startups-use Stripe to accept payments, grow their revenue, and accelerate new business opportunities. Our mission is to increase the GDP of the internet, and we have a staggering amount of work ahead. That means you have an unprecedented opportunity to put the global economy within everyone's reach while doing the most important
Posted 60+ days ago | Updated 7 hours ago

Certified Databricks Developer

Dolphin Solutions Inc

•

Remote

Full-time, Third Party, Contract

Job Title: Sr Databricks Consultant Location: Remote Duration: 6 months with possible extension Industry: Retail Rate: $60/HR All Inclusive on C2C Requirements: Must have at least 10 years experience in IT industry Expertise in Databricks Experience in Azure Data Lake, Spark, ADF, SQL and Kafka Experience / knowledge in Azure Cosmos DB, Synapse Analytics, Snowflake is plus Python/Java/JavaScript/Shell scripting Databricks certification is required
Easy Apply
Posted 11 days ago | Updated 3 days ago

Data Engineer-Full time perm

Pioneer Consulting Services LLC

•

Remote

Full-time

Data Engineer-Full time Strong hands-on experience withSparkScalaKafkaExperience in cloud-based architecture, development, and deployment (AWS) Familiar with:PythonKafkaAirflowData bricksUnderstanding of Machine Learning roshanATpioneeritconsultingDOTcom
Easy Apply
Posted 16 days ago | Updated 6 days ago

Staff Software Engineer - Backend

Databricks Inc

•

San Francisco, California, USA

Full-time

P-150 At Databricks, we are obsessed with enabling data teams to solve the world's toughest problems, from security threat detection to cancer drug development. We do this by building and running the world's best data and AI infrastructure platform, so our customers can focus on the high value challenges that are central to their own missions. Founded in 2013 by the original creators of Apache Spark , Databricks has grown from a tiny corner office in Berkeley, California to a global organizati
Posted 60+ days ago | Updated 7 hours ago

Data Architect - DBX + ML

UNICOM TECHNOLOGIES INC

•

Remote

Full-time, Part-time, Contract, Third Party

Role: Data Architect - DBX + ML Location: Remote (US) Exp: 10+ Years Visa / USC Required Past Experience: Databricks Certified Machine Learning Professional Preferred to have current credentials or ability to obtain within first 60 days of employment 5+ years of experience with a proven experience in ETL testing Handson experience in SQL with advanced level skills Experience in scripting language (shell/python) Knowledge and understanding in Data Warehouse and data modeling Experience in healthc
Easy Apply
Posted 2 days ago | Updated 1 hour ago

Engineering Manager, Data Marketplace

Enfoweb

•

Remote

Full-time

This is an exciting opportunity as the full-time Engineering Manager as we are in the early stages of building an Enterprise Data Marketplace with modern engineering practices and hybrid cloud implementation. As a Engineering Manager, you will : Own the engineering practices (code quality, unit testing, observability)Accountable for solutions delivery.Own product delivery and operations, including production support.Technical Execution of the featureEngage with Product Owners to engage with cus
Easy Apply
Posted 11 days ago | Updated 11 days ago

Data Architect - DBX + ML

Multivision Inc-IL

•

Remote

Third Party, Contract

Data Architect - DBX + ML Location: Remote (US) Exp: 10+ Years Required Past Experience: Databricks Certified Machine Learning Professional Preferred to have current credentials or ability to obtain within first 60 days of employment 5+ years of experience with a proven experience in ETL testing Hands on experience in SQL with advanced level skills Experience in scripting language (shell/python) Knowledge and understanding in Data Warehouse and data modeling Experience in healthcare insura
Easy Apply
Posted 2 days ago

Competency - Data Analytics

Techwave Consulting Inc

•

Remote

Full-time

Why Techwave With passion and purpose, we work in collaboration with our clients, colleagues, and communities, integrating innovative approaches and deep expertise to deliver results. Our vision is to become the world s most trusted digital transformation partner. We strongly believe in the power of collaboration and competency.At Techwave, we are always in an exercise to foster a culture of growth, and inclusivity. We ensure whoever is associated with the brand is being challenged at every step
Easy Apply
Posted 2 days ago

NLP Engineer

Genzeon

•

Remote

Third Party, Contract

About Genzeon Delivering excellence in IT services and solutions for the retail and healthcare sectors. Genzeon is a leading provider of intelligent automation, security, compliance, cloud, and managed services. Our healthcare capabilities integrate data systems, facilitate information flow, and leverage interoperability to improve clinical and operational outcomes. Genzeon empowers retail enterprises with multichannel IT solutions that deliver a personalized experience for consumers. Genzeon ha
Easy Apply
Posted 41 days ago | Updated 19 days ago

Software Developer - Machine Learning

BayOne Solutions

•

Remote

Contract

Title: Software Developer - Machine Learning Contract: 12+ Months Location: Remote Pay Rate $75/Hr on W2 Mid-level / Exp Needed 3-5 years of experience. (No C2C or Third-Party Vendors) Description: Ability to write robust code in one or more of Python, Go, and Java Proficient in core technologies like Spark, Hadoop and Hive. Experience in building real-time applications, preferably in Spark and streaming platforms like Kafka and Kinesis. Good understanding of machine learning pipelines and ma
Easy Apply
Posted 60+ days ago | Updated 26 days ago

Spark Developer

Nityo Infotech Corporation

•

San Francisco, California, USA

Contract, Third Party

JD: Advanced level of Java/Scala/Python development experience.Intermediate level of SQL and NoSQL experience (handling structured and unstructured data).Extensive experience with Spark Processing engine.Experience with Big data tools/technologies/ Streaming (Hive, Impala, OOZIE, Airflow, NIFI, Kafka)Experience analyzing data to discover opportunities and address gaps.Experience working with cloud or on-prem Big Data platforms (i.e. Netezza, Teradata, AWS Redshift, Google BigQuery, Azure Data W
Easy Apply
Posted 4 days ago

Senior Data Engineer

Sparibis, LLC

•

Remote

Full-time

Location: 100% Remote Years Experience: 10+ years Education: Bachelor s in IT related field Work Authorization: Must show that applicant is legally permitted to work in the United States. Clearance: Applicants must be able to meet the requirements to obtain an Public Trust security clearance. NOTE: United States Citizenship is required to be eligible to obtain this security clearance. Key Skills: 10+ years of IT experience focusing on enterprise data architecture and management Experience with
Easy Apply
Posted 13 days ago