Senior Data Engineer - PySpark

  • Tampa, FL
  • Posted 22 days ago | Updated 22 days ago

Overview

Hybrid
$120,000 - $140,000
Full Time

Skills

PySpark
Python
spark
cloudera
hadoop
aws
aws glue
aws emr
mapreduce
Glue
Data Engineer

Job Details

Hi,

DATAECONOMY is one of the fastest-growing Data & Analytics company with global presence. We are well-differentiated and are known for our Thought leadership, out-of-the-box products, cutting-edge solutions, accelerators, innovative use cases, and cost-effective service offerings.

We offer products and solutions in Cloud, Data Engineering, Data Governance, AI/ML, DevOps and Blockchain to large corporates across the globe. Strategic Partners with AWS, Collibra, cloudera, neo4j, DataRobot, Global IDs, tableau, MuleSoft and Talend.

 

Senior Data Engineer - PySpark

Tampa, FL 33602

Full-time

Job Summary:

We are seeking a highly skilled and experienced Senior Data Engineer to lead the end-to-end development of complex models for compliance and supervision. The ideal candidate will have deep expertise in cloud-based infrastructure, ETL pipeline development, and financial domains, with a strong focus on creating robust, scalable, and efficient solutions.

Key Responsibilities:

      •     Model Development: Lead the development of advanced models using AWS services such as EMR, Glue, and Glue Notebooks.

      •     Cloud Infrastructure: Design, build, and optimize scalable cloud infrastructure solutions with a minimum of 5 years of experience.

      •     ETL Pipeline Development: Create, manage, and optimize ETL pipelines using PySpark for large-scale data processing.

      •     CI/CD Implementation: Build and maintain CI/CD pipelines for deploying and maintaining cloud-based applications.

      •     Data Analysis: Perform detailed data analysis and deliver actionable insights to stakeholders.

      •     Collaboration: Work closely with cross-functional teams to understand requirements, present solutions, and ensure alignment with business goals.

      •     Agile Methodology: Operate effectively in agile or hybrid agile environments, delivering high-quality results within tight deadlines.

      •     Framework Development: Enhance and expand existing frameworks and capabilities to support evolving business needs.

      •     Documentation and Communication: Create clear documentation and present technical solutions to both technical and non-technical audiences.

Required Qualifications:

      •     10+ years of experience with Python programming.

      •     5+ years of experience in cloud infrastructure, particularly AWS.

      •     3+ years of experience with PySpark, including usage with EMR or Glue Notebooks.

      •     3+ years of experience with Apache Airflow for workflow orchestration.

      •     Solid experience with data analysis in fast-paced environments.

Domain Expertise:

      •     Strong understanding of capital markets, financial systems, or prior experience in the financial domain is a must.

Technical Skills:

      •     Proficiency with cloud-native technologies and frameworks.

      •     Familiarity with CI/CD practices and tools like Jenkins, GitLab CI/CD, or AWS CodePipeline.

      •     Experience with notebooks (e.g., Jupyter, Glue Notebooks) for interactive development.

Soft Skills:

      •     Excellent problem-solving skills and ability to handle complex technical challenges.

      •     Strong communication and interpersonal skills for collaboration across teams and presenting solutions to diverse audiences.

      •     Ability to thrive in a fast-paced, dynamic environment.

Preferred Qualifications:

      •     Experience with data governance and compliance frameworks.

      •     Familiarity with machine learning frameworks and techniques for financial modeling.

      •     Certifications in AWS or other cloud technologies.

If you are interested in the opportunity listed above, please share your resume at

If you aren t interested, perhaps you know an excellent referral?

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

About DATAECONOMY