Sr. Data Engineer

data engineer, data pipelines, machine learning, etl, jupyter notebooks, spark, python, sql
Full Time
Depends on Experience
Work from home not available Travel not required

Job Description

A Top Fortune 500 Firm in Manhattan is looking for a Full-Time, permanent, employee for a Sr. Lead Data Engineer:

The Senior Data Engineer role within the Data Strategy group at the Firm provides an opportunity to own and oversee the build-out and implementation of data solutions to power one of the world s largest and most respected risk management and reinsurance firms. The Data Strategy group has a start-up style mandate (within a $1.3 billion company) to quickly and efficiently enhance the acquisition, storage, analysis, fidelity, and monetization of massive amounts of client, internal, and third-party data across the Firm s organization.

As a member of the Data Strategy group, the Senior Data Engineer will work with fellow engineers, data scientists, product managers, business analysts, and stakeholders from other internal groups to design and improve data-centric projects with the dual mandate of (1) increasing the efficiency of the data collection and analysis process across the Firm and (2) driving the monetization of data via newly designed and existing products for the Firm s reinsurance clients. The senior data engineer will be the head facilitator on a number of innovative initiatives and will have ownership over the design, development, and delivery of projects which will require direct reporting to senior-level management in both business and technical groups.

----- Leadership Responsibilities -----

- Lead data engineers in 1-1s, regular professional development and performance reviews in a center of excellence context
- Partner with head of data strategy establishing a best-in-class data engineering competency and tooling within the organization
- Contribute to the overall data strategy vision and execution via quarterly planning and executive committee reporting
- Participate and improve in the data engineering recruiting process from resume reviews and phone screens to on-site interviews

----- Data Engineer Responsibilities -----

- Develop, implement, and deploy custom data pipelines powering machine learning algorithms, insights generation, client benchmarking tools, business intelligence dashboards, reporting and new data products.
- Innovate new ways to leverage enormous amounts of various datasets to drive revenues via the development of new products with the Data Strategy team, as well as the enhanced delivery of existing products
- Consume data from a variety of sources (relational DBs, APIs, NetApp and other cloud storage, FTPs) & formats (excel, CSV, XML, parquet, unstructured))
- Construct and maintain data pipelines between the Firm s databases, and other sources, with the data lake utilizing modern ETL frameworks
- Own the role of data steward for a variety of high value datasets and implement innovative quality assurance practices
- Establish and implement data security and privacy standards to ensure internal and regulatory compliance
- Establish and implement metadata management standards and capabilities, including lineage mapping
- Establish and maintain strong relationships with internal clients as an engineering representative for data strategy
- Enforce strong development standards across the team through code reviews, unit testing, and monitoring
- Perform basic data analysis within Jupyter Notebooks to validate the fulfillment of requirements for data pipelines
- Evangelize data strategy techniques and best practices throughout global strategic advisory
- Keep up-to-date on the latest trends and innovation in data technology and how these trends apply to the Firms business and data strategy

----- Required Qualifications -----

- 5-8+ years of relevant experience as a data engineer or in a similar role
- Master s Degree or PhD in data science, computer science or related quantitative field such as applied mathematics, statistics, engineering or operations research
- Extensive experience with Spark, Python, JSON, and SQL
- Extensive experience integrating data from semi-structured and unstructured sources
- Knowledge of various industry-leading SQL and NoSQL database systems
- Experience working in an Agile environment to facilitate the quick and effective fulfillment of group goals
- Good interpersonal skills for establishing and maintaining good internal relationships, working well as part of a team and for presentations and discussions
- Strong analytical skills and intellectual curiosity as demonstrated through academic experience or work assignments
- Ability to communicate technical concepts to non-technical audience
- Excellent English verbal and writing skills for complex communications with the Firm's colleagues at all levels of the organization
- Good ability to prioritize workload according to volume, urgency, etc. and to deliver on required projects in a timely fashion
- Desire to take ownership of proposed and assigned tasks and to seek assistance when needed

----- Preferred Qualifications -----

- Strong understanding of entity resolution, streaming technologies, and ELT/ETL frameworks
- Ability to articulate the advantages of various cloud and on-premises deployment options
- Experience with Master Data Management
- Experience with web scraping and crowd sourcing technologies
- Familiarity with modern data productivity frameworks and their alternatives such as Databricks, DataRobot, and Alteryx

Posted By

Marla Liebel

6 East 39th St., Suite 1102 New York, NY, 10016

Dice Id : emtekny
Position Id : 6288862
Originally Posted : 2 weeks ago
Have a Job? Post it