Senior Data Engineer

Amazon Redshift, Amazon Web Services, Apache Spark, Big data, Business intelligence, C, Cloud, Computer science, Dashboard, Data analysis, Data engineering, Data modeling, Data processing, Data science, Data visualization, Data warehouse, Django, Docker, ETL, Engineering, Java, Linux, MPP, Machine learning, Mathematics, Microsoft Windows Azure, Modeling, Physics, Problem solving, Python, Reporting, SAN, SQL, Scala, Snow flake schema, Software, Statistics, TBS, Accessibility
Full Time
$100,000 - $200,000
Travel not required

Job Description

Our client, a mid-size global investment firm with over $40B in capital, is looking to hire a talented Senior Data Engineer (ideally 3-7 years of experience, but flexible) to finish building out their team!

The firm has presence in both NYC and CT. With that in mind, the firm is open to the ideal candidate being in NYC 1-2 days/week, CT 1-2 days/week, and remote 1-3 days/week, depending on your accessibility to the offices.

JOB FUNCTIONS:

The Senior Data Engineer will be part of the strategic cloud data engineering team, responsible for conducting the firm's data engineering efforts. The role is focused on the development of performant data pipelines and architectures on the alternative data team.

The Senior Data Engineer will work closely with data engineers, data analysts, data scientists, and other developers to help build and extend Viking’s significant data platform.

Responsibilities:

• Build and maintain robust data pipelines that ingest TBs of data.

• Build new Airflow DAGs that manage different operators or individual tasks.

• Create logical and physical data models for big data to be stored in cloud data warehouses.

• Optimize Spark and SQL queries.

• Create new SQL tables for reports and dashboards.

• Work with data warehouses: Snowflake, AWS Redshift, and other cloud data warehouses.

• Work with data analysts on implementing dashboards and performing analysis.

• Use Databricks Spark and data visualization tools to analyze business problems.

• Generate reports and dashboards for business insights.

• Write analytical queries to extract insights for large datasets (billions of rows, multi-TB in size).

QUALIFICATIONS:

• A minimum of 8 years of relevant work experience.

• A degree or advanced degree in Computer Science, Engineering, Physics, Mathematics, Statistics,

or Machine Learning, with a record of academic success.

• Excellent development experience in Python, Apache Spark, and SQL.

• Extensive experience with large-scale data processing solutions.

• Extensive working knowledge of Apache Spark, Databricks and pandas.

• Extensive working experience with AWS ecosystem.

• Excellent computer science fundamentals and problem-solving skills.

• Strong ability to work in Linux environment.

• Experience in the fields of data warehousing, business intelligence and big data related technologies.

• Experience in creating logical and physical data models.

• Experience with one or more cloud MPP data warehouses including: Snowflake, AWS Redshift, Azure Data Warehouse (Synapse), or Google BigQuery.

The ideal candidate will also have:

• Experience with Java/Scala or C++

• Familiarity with machine learning libraries and applications

• Experience with object relational modeling frameworks (SQLAlchemy, DJango)

• Strong understanding of Docker and containerization

Dice Id : 10513353
Position Id : 6937888
Originally Posted : 3 months ago
Have a Job? Post it