Apply Now

Member of Technical Staff (Data Scientist/Engineer, Online Metrics)

New York, NY, US • Posted 1 hour ago • Updated 1 hour ago

Full Time

On-site

USD $200,000.00 - 320,000.00 per year

Fitment

Dice Job Match Score™

🎯 Assessing qualifications...

Job Details

Skills

Search Engines
Proxies
A/B Testing
Dashboard
Training
Clarity
Collaboration
Analytics
Python
SQL
Statistics
Business Intelligence
Visualization
Reporting
Workflow
Artificial Intelligence
Apache Spark
Databricks
Customer Facing

Summary

Perplexity serves tens of millions of users daily with reliable, high-quality answers grounded in an LLM-first search engine and specialized data sources. The Answer Quality team ensures that our prompts, tools, search, and specialized datasets, combined with both frontier and in-house models, create the best possible experience for our users. As a Data Scientist/Engineer on this team, you will derive online signals from user interactions to bridge the gap between changes in answer quality and observed user behavior.

Responsibilities

Discover and validate online signals from user interactions that serve as reliable proxies for true answer quality
Design and implement novel online metrics to be tracked both in A/B testing and on product health dashboards, ensuring alignment with ground-truth evaluations
Analyze experimental results to validate these metrics, ensuring they accurately predict user satisfaction and drive product decisions
Build and maintain the data pipelines that calculate these metrics at scale, delivering actionable quality signals to Search, Product, and model training teams
Communicate findings and bring clarity through close collaboration with Product and Search teams
Operate in a small, high-impact team where your work directly shapes how Perplexity measures and improves Answer Quality

Qualifications

MS in a technical field or equivalent experience
4+ years of experience working as a Data Scientist, Analytics Engineer, or related role
Experience working on search, recommendation, or LLM-based products, with an emphasis on designing online metrics and analyzing A/B experiments
Strong proficiency in Python and SQL (expected to write production-grade code)
Deep knowledge of statistical analysis
Experience with Business Intelligence (BI) tools for visualization and reporting
Comfortable with agentic coding workflows and using AI-assisted development tools to iterate faster

Preferred Qualifications

Proficiency with Apache Spark and Databricks
Experience with the development or validation of LLM-as-a-judge systems
Prior work supporting customer-facing products at scale

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 91140125
Position Id: ad36c36cbbb1e1127740ea6c34d253c3
Posted 1 hour ago

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

Staff, Data Scientist

Hoboken, New Jersey

•

Today

Position Summary What you'll do We are seeking a Staff Data Scientist with a strong foundation in machine learning, information retrieval, and LLMs. The selected candidate will be responsible for architecting and developing new Machine Learning models to enhance Customer Intelligence and Segmentation landscape within Walmart. This is an exciting opportunity for a creative and innovative professional who thrives in a fast-paced, results-driven environment. Experience building machine learning,

Full-time

USD 132,000.00 - 264,000.00 per year

Data Scientist II, Tech

New York, New York

•

Today

Employer: Uber Technologies, Inc. Job Title: Data Scientist II, Tech Job Location: New York, New York Job Type: Full Time Rate of Pay: $142,000 to $174,000 per year You will be eligible to participate in Uber's bonus program, and may be offered other types of comp. You will also be eligible for various benefits. More details can be found at the following link [](;br> Duties: Work closely with the web product team to design and analyze experiments. Monitor and understand business trends, pro

Full-time

USD 142,000.00 - 174,000.00 per year

Senior Manager, Data Science Services

New York, New York

•

Today

LiveRamp is the data collaboration platform of choice for the world's most innovative companies. A groundbreaking leader in consumer privacy, data ethics, and foundational identity, LiveRamp is setting the new standard for building a connected customer view with unmatched clarity and context while protecting precious brand and consumer trust. LiveRamp offers complete flexibility to collaborate wherever data lives to support the widest range of data collaboration use cases-within organizations, b

Full-time

USD 181,000.00 - 263,000.00 per year

VP Data Scientist Lead - Product, Experience and Technology (PXT) Analytics Team

New York, New York

•

Today

Job Description Join us to shape how we measure and improve the software delivery experience across Chase. You will lead a team that turns product and engineering data into clear, trusted insights that influence strategy, investment decisions, and roadmaps. This role offers the opportunity to define measurement frameworks in an area where standards are still evolving, and to build a culture of analytical rigor and practical impact. If you enjoy ambiguous problems, influencing senior stakeholder

Full-time

USD 142,500.00 - 210,000.00 per year

Search all similar jobs