Senior Data Engineer

  • Remote or San Francisco, CA
  • 11 hours ago
API, Amazon Kinesis, Amazon Web Services, Apache Ant, Apache Kafka, Apache Maven, Apache Storm, Apache Spark, DBA, Continuous integration, Cloud, Data mapping, Database, CAN, Data warehouse, Data engineering, Django, ERwin, Employment authorization, Implementation, NoSQL, Leadership, Product design, Python, RESTful, SAN, Snow flake schema, Secure Shell, Modeling, Migration, ETL, Financial services, Data modeling, Data processing, Unix, QA, Jenkins, Gradle, Java, Integration, Performance tuning, Streaming
Full Time
Depends on Experience
Travel required to 25%.

Job Description


PETADATA is looking for a Senior Data Engineer(AWS, ETL, Kafka & Python)  to work with one of our clients.

Job Location: San Francisco, CA (On-site)

Work Authorization: W2, Full-time

Visa: U.S. Citizens and those authorized to work in the U.S. are encouraged to apply. We are unable to sponsor at this time.

Note: In a week 1-3 business days will work remotely.

PETADATA is looking for candidates to provide design & development projects in large or complex ETL  Frameworks & Data API python packages projects. Involving in application development, migrations, integrations, enhancements to existing applications.

Roles & Responsibilities:

  • Candidates can do the design and development of the Data Warehouse from End to End lifecycle.
  • To build distributed backend application in the AWS cloud.
  • Understand repeatable automated processes for building the application, testing it, documenting it, and deploying it at scale.
  • A desire to work as part of a growing, fast-paced, and highly-flexible team. Ability to quickly learn new technologies and adapt to a fast-paced development environment.
  • You can coordinate with Data Engineering developers and Database Administrators resulting in effective data-driven solutions.
  • You can work with, and incorporate feedback from, product designers and other stakeholders in the company.
  • To establish quality processes to deliver a stable and reliable solution.
  • Understand the project proposal and assist the team in analyzing how the new system or functionality can be integrated with the current environment.
  • Ability to identify and resolve any performance and/or data-related issues
  • Provide documentation (Data Mapping, Technical Specifications, Production Support, data dictionaries, test cases, etc.) for all projects

Required Skills:

  • Candidates should have 8+ years of experience as ETL Developers.
  • 5+ years of hands-on coding experience in Java, Python, pandas, Django, and Stream processing services like Kafka, AWS Kinesis, Apache Storm, Spark Streaming, etc., 
  • 4+ years of experience in CI/CD tools other dependency management tools & build tools like Jenkins, Gradle, Maven, Ant, and Ivy.
  • 4+ years of experience in Development Methodologies, Databases Platforms, and Data Modeling tools (ERwin/Model Manager).
  • 3+ years of experience in Cloud implementation & Restful API & Server-side API integration tools.
  • 3+ years of experience in building data warehouse solutions and Data Modeling.
  • Expert-level skill in modeling, managing, scaling, and performance tuning high volume transactional database
  • Should Be able to establish scalable, efficient, automated processes for dataset analysis, model development, and validation.
  • Expertise in relational and NoSQL databases with batch data processing with SFTP/SSH, Unix/Linux. 
  • Financial services industry experience Will be helpful.
  • Must have Project Management skills to lead and guide the project teams.

Preferred Qualifications

  • Bachelor's/ Master’s degree in Computer Science, Engineering, or a related field.
  • Candidates must have skills in AWS, Python, ETL, Apache Kafka, Restful API & snowflake.

If you are interested and meet the above job requirements, please submit your resume to

After carefully reviewing your experience and skills, one of our Hiring team members will contact you on the next steps.


Dice Id : 90751012
Position Id : 7188274
Originally Posted : 3 months ago
Have a Job? Post it

Similar Positions

Senior QA Analyst
  • Randstad Technologies
  • San Francisco, CA, USA
Senior Data Engineer / Spark
  • Motion Recruitment
  • San Francisco, CA, USA
Data Engineer
  • Prism IT Corp
  • Menlo Park, CA, USA
Senior Data Engineer / REMOTE ONLY
  • Motion Recruitment
  • San Francisco, CA, USA
Sr. Data engineer
  • Jobot
  • San Jose, CA, USA
Senior Data Engineer
  • Motion Recruitment
  • San Francisco, CA, USA
Senior Data Engineer / Snowflake
  • Motion Recruitment
  • San Francisco, CA, USA
Junior Developer - Infrastructure Tools
  • FriendFinder Networks, Inc.
  • San Jose, CA, USA
Data Science Practice Director
  • Unify Consulting
  • San Francisco, CA, USA
Data Engineering Consultants
  • Unify Consulting
  • San Francisco, CA, USA