Sr. Data Engineer

Data, Data Engineer, Data Pipeline, Apache Kafka, Apache Spark, AWS, AWS Redshift, AWS RDS, Amazon EC2, Amazon Kinesis, Amazon Web Services, EMR, NoSQL, Streaming, Amazon S3, Amazon Redshift, Amazon RDS
Full Time
$155,000 - $190,000
Work from home available Travel not required

Job Description

Please note I have direct access to the Hiring VP on this position.

100% Remote until Covid is contained. Then in San Francisco, CA or Irvine, CA or Dallas, TX or Denver, CO

Compensation: Base Salary + Performance Bonus + Options + Sign on Bonus

Relocation assistance provided if applicable.

Telecommute: 2-3 days/week

Be a major contributor to the success of a company whose business model is based on the accumulation, analysis and selling of Data. This pre-IPO company is already a major player in its industry, is profitable and successful. The company is global, has $2.5 Billion in Revenue, has been realizing significant yearly growth for several consecutive years, has 600+ global employees and is an active philanthropic institution.

The Sr. Data Engineer will be solely responsible for:

  • Reviewing the current Data Architecture (Pipeline, Data Stores, Databases, etc.) and recognition of issues, gaps and opportunities for performance improvements of the Data Pipeline.
  • Designing and building out a brand-new mission critical high-throughput Data Pipeline from the ground up with technologies and tools up based on your professional recommendations and preferences.
  • As the amount Data continues to increase and additional revenue streams are identified, the Sr. Data Engineer will begin to build and lead a team of Data Engineers.
  • Legacy Data Pipeline aggerates data from 10+ million sources, has 10+ PB s of total data, ingests 5+ TB s data/day and is currently built utilizing Python, PostgreSQL and AWS Kinesis Data Streams, Kinesis Firehose, S3, 30+ RDS databases, Kinesis clusters (250+ shards) and 3,000+ EC2 instances

The Sr. Data Engineer reports to the VP of Operations

Requirements:

  • Must have recent experience architecting and building out a high throughput Data Pipeline at scale from the ground up utilizing the Apache Kafka framework (or modern streaming framework)
  • Experience with any of the following is only a plus; NOT mandatory: Apache Kafka Confluent, Apache Spark Core, Apache Spark Streams, Apache Spark Analytics Engine, Apache Spark SQL or AWS Redshift, RDS, S3, Kinesis Data Streams, Kinesis Firehose and EMR or NoSQL databases (PostgreSQL, CouchDB, Neo4j, Cassandra, Redis, etc.). Again, not mandatory.

Benefits

Company offers a comprehensive benefits package including: 4% matching 401K, Stock Options, 2 Weeks PTO, HMO/PPO medical, dental and vision coverage; paid company holidays, flexible spending plans, tuition reimbursement, parental paid leave, meals and a casual work environment.

Dice Id : equest
Position Id : 6910091
Originally Posted : 1 month ago
Have a Job? Post it

Similar Positions

SR Data Engineer
  • Surya Systems
  • San Francisco, CA, USA
Senior Data Engineer (Kafka)
  • ITECS
  • San Francisco, CA, USA
Senior Data Engineer
  • Jobot
  • San Francisco, CA, USA
Senior Data Engineer
  • Jobot
  • San Francisco, CA, USA
Senior Data Engineer - Java/Scala
  • Motion Recruitment
  • San Francisco, CA, USA
Sr. Data engineer
  • Jobot
  • San Francisco, CA, USA
Senior Software Engineer - Big Data
  • Uber Corporate
  • San Francisco, CA, USA
Senior Android Engineer, Mobile Data Platform
  • Uber Corporate
  • San Francisco, CA, USA
Sr Software Engineer, Data
  • Uber Corporate
  • San Francisco, CA, USA