Senior Data Engineer

New York, NY, US • Posted 25 days ago • Updated 7 hours ago
Contract W2
On-site
Company Branding Image
Fitment

Dice Job Match Score™

📊 Calculating match score...

Job Details

Skills

  • Financial Services
  • Migration
  • Testing
  • Agile
  • Problem Analysis
  • Data Analysis
  • Reporting
  • Communication
  • Workflow
  • Data Processing
  • Data Quality
  • Collaboration
  • Scalability
  • Cloud Computing
  • Data Lake
  • High Availability
  • Debugging
  • Performance Tuning
  • ELT
  • Apache Spark
  • PySpark
  • Extract
  • Transform
  • Load
  • Databricks
  • Amazon EC2
  • Electronic Health Record (EHR)
  • Storage
  • Amazon S3
  • EBS
  • Database
  • Remote Desktop Services
  • Amazon RDS
  • Amazon DynamoDB
  • Data Integration
  • Orchestration
  • Docker
  • Kubernetes
  • Identity Management
  • Computer Networking
  • Amazon Web Services
  • Continuous Integration
  • Continuous Delivery
  • GitLab
  • IaaS
  • Provisioning
  • Terraform
  • Programming Languages
  • Python
  • Scala
  • SQL
  • MEAN Stack
  • Customer Service
  • Training And Development
  • SAP BASIS
  • Dialog Manager
  • Dimensional Modeling
  • SAS Display Manager

Summary

Software Guidance & Assistance, Inc., (SGA), is searching for a Senior Data Engineer for a contract assignment with one of our premier Financial Services clients in New York, NY.

Responsibilities :
  • Work on migrating applications from an on-premises location to the cloud service providers.
  • Develop products and services on the latest technologies through contributions in development, enhancements, testing and implementation.
  • Develop, modify, extend code for building cloud infrastructure, and automate using CI/CD pipeline.
  • Partners with business and peers in the pursuit of solutions that achieve business goals through an agile software development methodology.
  • Perform problem analysis, data analysis, reporting, and communication.
  • Work with peers across the system to define and implement best practices and standards.
  • Assess applications and help determine the appropriate application infrastructure patterns.
  • Use the best practices and knowledge of internal or external drivers to improve products or services.
  • Design, develop, and optimize scalable ETL/ELT pipelines using ETL/ELT pipelines using Databricks and Apache Spark (PySpark and Scala).
  • Build data ingestion workflows from various sources (structured, semi-structured, and unstructured).
  • Develop reusable components and frameworks for efficient data processing.
  • Implement best practices for data quality, validation, and governance.
  • Collaborate with data architects, analysts, and business stakeholders to understand data requirements.
  • Tune Spark jobs for performance and scalability in a cloud-based environment.
  • Maintain robust data lake or Lakehouse architecture.
  • Ensure high availability, security, and integrity of data pipelines and platforms.
  • Support troubleshooting, debugging, and performance optimization in production workloads.
Required Skills:
  • Develop, and optimize scalable ETL/ELT pipelines using Databricks and Apache Spark (PySpark and Scala)
  • Experience in developing data pipeline solutions to ingest and exploit new and existing data sources.
  • Expertise in leveraging SQL, programming language like Python and ETL tools like Databricks
  • Perform code reviews to ensure requirements, optimal execution patterns and adherence to established standards.
  • Expertise in AWS Compute (EC2, EMR), AWS Storage (S3, EBS), AWS Databases (RDS, DynamoDB), AWS Data Integration (Glue).
  • Advanced understanding of Container Orchestration services including Docker and Kubernetes, and a variety of AWS tools and services.
  • Good understanding of AWS Identify and Access management, AWS Networking and AWS Monitoring tools.
  • Proficiency in CI/CD and deployment automation using GITLAB pipeline.
  • Proficiency in Cloud infrastructure provisioning tools e.g., Terraform.
  • Proficiency in one or more programming languages e.g., Python, Scala.
  • Experience in Starburst, Trino and building SQL queries in federated architecture.
  • Good knowledge of Lake house architecture
SGA is a technology and resource solutions provider driven to stand out. We are a women-owned business. Our mission: to solve big IT problems with a more personal, boutique approach. Each year, we match consultants like you to more than 1,000 engagements. When we say let's work better together, we mean it. You'll join a diverse team built on these core values: customer service, employee development, and quality and integrity in everything we do. Be yourself, love what you do and find your passion at work. Please find us at .

SGA is an Equal Opportunity Employer and does not discriminate on the basis of Race, Color, Sex, Sexual Orientation, Gender Identity, Religion, National Origin, Disability, Veteran Status, Age, Marital Status, Pregnancy, Genetic Information, or Other Legally Protected Status. We are committed to providing access, equal opportunity, and reasonable accommodation for individuals with disabilities in employment, and our services, programs, and activities. Please visit our company to request an accommodation or assistance regarding our policy.

#LI-DM1
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: sgainc
  • Position Id: 26-00351
  • Posted 25 days ago

Company Info

About Software Guidance & Assistance

Founded in 1981, SGA is a technology and resource solutions provider with a national footprint and headquartered in the shadow of Wall Street. We’re a certified women-owned business. We provide contingent staffing, direct placement, and professional and managed services to transform businesses and evolve careers. We’re small enough to tailor our services to each client and big enough to deliver for some of the world’s largest employers. Our professionals are experts in areas such as IT, finance, accounting, risk, and clinical.

SGA provides contingent staffing, direct placement, and professional and managed services nationwide for Fortune 500 companies, mid-size businesses and select startups.

Our core skillsets include all areas of technology – business & data analysis, cyber & network security, database administration, development & architecture, infrastructure, program & project management, quality assurance & testing. We also deliver talent across professional business functions such as finance, accounting, risk, and clinical.

Our Professional & Managed Services team delivers IT projects through onshore, offshore and hybrid delivery models. We develop software products, modernize applications, add features, and integrate and maintain systems. Our scope covers, among others, complex application suites, data management and visualizations, machine learning and mobile applications.

About_Company_OneAbout_Company_Two
Create job alert
Set job alertNever miss an opportunity! Create an alert based on the job you applied for.

Similar Jobs

San Francisco, California

Today

Contract

USD 88.00 - 90.00 per hour

Lehi, Utah

Today

Contract

USD 55.00 - 62.00 per hour

New York, New York

Today

Contract

USD 80.00 - 80.00 per hour

Reston, Virginia

Today

Full-time

USD 160,000.00 - 185,000.00 per year

Search all similar jobs