Job DescriptionGuardian is seeking a Senior Data Engineer to join the team responsible for establishing and maintaining an optimized, scalable data pipelines to provide access to high quality, consistent data in an easy and convenient manner to all authorized end users and applications of Guardian and its subsidiaries.
As a Senior Data Engineer, you will play a vital role in our data infrastructure and analytics ecosystem. You will work closely with data architects and data scientists to build and maintain data pipelines, transform raw data into actionable insights, and contribute to the development of cutting-edge data solutions. This is an excellent hands-on opportunity to experience the latest technologies and make a significant impact on our data-driven initiatives.
You will- Collaborate with cross-functional teams to understand data requirements and translate them into effective data pipelines.
- Provide technical leadership and mentorship to a team of Data Engineers. Oversee the design, development, and optimization of scalable and robust big data solutions and pipelines.
- Utilize extensive expertise in Databricks technologies with Python, PySpark, and SQL, and distributed data processing frameworks to design, develop, and implement highly performant and fault-tolerant ETL/ELT pipelines for large-scale data ingestion, processing, and transformation.
- Partner with multiple management teams to ensure appropriate integration of functions to meet goals. Identify and define necessary system enhancements to deploy new products and process improvements. Conduct in-depth analysis of complex business processes and system processes to recommend innovative solutions.
- Shape opportunities and create execution approaches throughout the lifecycle of client engagements
- Ensure all deliverables are of high quality by setting development standards, adhering to the standards, and participating in code reviews
- Assist in data modeling, schema design, and database optimization.
- Monitor and troubleshoot data pipelines to ensure data accuracy, reliability, and performance.
Participate in code reviews, testing, and documentation efforts to maintain high-quality code and data documentation. - Stay updated on industry trends and best practices in data engineering.
- Establish and maintain the quality, supportability, and performance of all processes.
- Continue to look for innovative and next generation solutions for solving data challenges.
- Establish real time data pipelines using AWS services like SQS / SNS and OpenSearch.
You are- A self-starter, self-motivated team player and have excellent communication skills, enabling you to engage with all levels of stakeholders who is fueled by collaboration, able to transform conceptual designs into reliable, scalable processes that meet or exceed customer needs.
- An individual who works closely with IT and business leaders across the enterprise to drive and embed the Enterprise Data Strategy into transformational roadmaps ensuring key principles of Data Governance, Data Management, Data Analytics and Architecture are implemented.
You have- 15+ Years of IT experience, with 8+ years of experience as a Data Architect, data engineer or other data professional
- Experience with Databricks and ETL tools
- Knowledge of Mainframe and VSAM data structures
- Knowledge and hands on with Jenkins pipelines and CI/CD
- Expert Relational database concepts and experience across multiple database technologies (e.g., SQL, NoSQL, Oracle, Hadoop, Postgres) environments.
- Hands on experience in data engineering desired with extensive knowledge of SQL and PySpark, on Databricks Platform.
- Hands on experience designing, building, and optimizing data pipelines, architecture for large scale data warehousing projects.
- Experience with cloud data services such as Redshift, Relational database services (RDS), Snowflake and Databricks.
- Knowledge of AWS Data services is a big plus
- Demonstrated expertise with data sourcing, profiling, data mapping and data analysis.
- Strong SQL, PL/SQL experience and skilled in creating and optimizing queries and data structures.
- Knowledge of ETL concepts, preferably Informatica or Syncsort
- Experience with one or more SQL-on-Hadoop technology (HIVE, Presto, Spark SQL) and Hadoop ecosystem.
- Ability to design cloud and on Prem data solutions and provide guidance across storage, big data platform services, Hadoop ecosystem, vendor SaaS integrations, RDBMS and NoSQL databases is a plus.
- Excellent written and verbal communication skills and creative problem-solving skills.
- Ability to adapt to changing business priorities and experience with effectively communicating change across different levels in the organization.
- Communication Skills: Articulate the vision and goals, both to the team members and other stakeholders.
- Emotional Intelligence & Empathy: Motivate, inspire, and resolve conflicts with others while building a positive rapport with team members and partners.
- Collaboration: Collaborate effectively with the Product Manager and your leadership and keep appraised with key updates and planning. Encourage partnership between team members to create a more dynamic and effective work environment.
- Strong listeners and able to provide constructive feedback.
- Understanding of Insurance domain
- Significant experience interfacing with both customers and management across business and IT
Location- Three days a week at a Guardian office in Bethlehem, PA. New York NY, or Holmdel NJ
- Up to 10% travel within the NY/NJ/PA locations
Position Level: 7
Salary Range:$99,150.00 - $162,885.00
The salary range reflected above is a good faith estimate of base pay for the primary location of the position. The salary for this position ultimately will be determined based on the education, experience, knowledge, and abilities of the successful candidate. In addition to salary, this role may also be eligible for annual, sales, or other incentive compensation.
Our PromiseAt Guardian, you'll have the support and flexibility to achieve your professional and personal goals. Through skill-building, leadership development and philanthropic opportunities, we provide opportunities to build communities and grow your career, surrounded by diverse colleagues with high ethical standards.
Inspire Well-BeingAs part of Guardian's Purpose - to inspire well-being - we are committed to offering contemporary, supportive, flexible, and inclusive benefits and resources to our colleagues. Explore our company benefits at Benefits apply to full-time eligible employees. Interns are not eligible for most Company benefits.
Equal Employment OpportunityGuardian is an equal opportunity employer. All qualified applicants will be considered for employment without regard to age, race, color, creed, religion, sex, affectional or sexual orientation, national origin, ancestry, marital status, disability, military or veteran status, or any other classification protected by applicable law.
AccommodationsGuardian is committed to providing access, equal opportunity and reasonable accommodation for individuals with disabilities in employment, its services, programs, and activities. Guardian also provides reasonable accommodations to qualified job applicants (and employees) to accommodate the individual's known limitations related to pregnancy, childbirth, or related medical conditions, unless doing so would create an undue hardship. If reasonable accommodation is needed to participate in the job application or interview process, to perform essential job functions, and/or to receive other benefits and privileges of employment, please contact Please note: this resource is for accommodation requests only. For all other inquires related to your application and careers at Guardian, refer to the Guardian Careers site.
Visa SponsorshipGuardian is not currently or in the foreseeable future sponsoring employment visas. In order to be a successful applicant. you must be legally authorized to work in the United States, without the need for employer sponsorship.
Current Guardian Colleagues: Please apply through the internal Jobs Hub in Workday. 
Never miss an opportunity! Create an alert based on the job you applied for.
