Big Data Engineer

job summary:

Data Engineer Job Requirements

Data Pipeline Expertise - Capable of implementing a diverse range of data solutions with significant autonomy, demonstrating the ability to independently build and maintain complex ETL (Extract, Transform, Load) pipelines

Possesses advanced skills in pipeline complexity management and orchestration, with a proven track record of developing high-performance data products accessible via APIs for critical tier 1 trading applications

Demonstrates expertise in archiving data products to data lakes for comprehensive analytics purposes

Data Quality and Advanced Analytics - Robust proficiency in Data Quality, Data Validation, and Anomaly Detection techniques - Experience with AI/ML and Large Language Model (LLM) approaches to data analysis and validation - Ability to implement sophisticated data integrity and verification processes

location: Malvern, Pennsylvania

job type: Contract

salary: $49.46 - 54.46 per hour

work hours: 8am to 5pm

education: Bachelors



responsibilities:

Data Engineer Job Requirements

• Data Pipeline Expertise - Capable of implementing a diverse range of data solutions with significant autonomy, demonstrating the ability to independently build and maintain complex ETL (Extract, Transform, Load) pipelines
• Possesses advanced skills in pipeline complexity management and orchestration, with a proven track record of developing high-performance data products accessible via APIs for critical tier 1 trading applications
• Demonstrates expertise in archiving data products to data lakes for comprehensive analytics purposes
• Data Quality and Advanced Analytics - Robust proficiency in Data Quality, Data Validation, and Anomaly Detection techniques - Experience with AI/ML and Large Language Model (LLM) approaches to data analysis and validation - Ability to implement sophisticated data integrity and verification processes

qualifications:

Technical Skill Requirements:

Programming and Cloud Technologies - Expert-level skills in: - Python - SQL - AWS Cloud Services: - Cloud Storage - Lambda - ECS - Glue Data Quality - HDMI - Redshift Machine Learning - OpenSearch - RDS

API and Resilience Technologies - Advanced skills in building resilient API architectures, including: - Microservices design patterns - API gateway configuration - Circuit breaker implementations - Fault tolerance and error handling mechanisms - Expertise in API development and management tools: - Swagger/OpenAPI - Postman - GraphQL - RESTful API design principles

Proficiency in resilience engineering techniques: - Distributed system design - Load balancing - Failover strategies - Horizontal and vertical scaling approaches - Containerization and orchestration: - Docker - Kubernetes - Helm charts

Monitoring and observability: - Prometheus - Grafana - ELK Stack (Elasticsearch, Logstash, Kibana) - Distributed tracing with Jaeger or Zipkin Additional Technical Competencies

Proficient in Apache Flink - Advanced Java programming skills - Proven ability to create and optimize complex data products - Demonstrated Subject Matter Expertise (SME) in data engineering principles ## Professional Attributes

Capability to work independently through complex data challenges - Robust preference for candidates with prior experience in investment data domain and related technologies

Key Differentiators - Deep understanding of data engineering best practices - Ability to design scalable, efficient, and resilient data solutions - Proven track record of solving intricate data infrastructure problems with robust API architectures

Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.

At Randstad Digital, we welcome people of all abilities and want to ensure that our hiring and interview process meets the needs of all applicants. If you require a reasonable accommodation to make your application or interview experience a great one, please contact

Pay offered to a successful candidate will be based on several factors including the candidate's education, work experience, work location, specific job duties, certifications, etc. In addition, Randstad Digital offers a comprehensive benefits package, including: medical, prescription, dental, vision, AD&D, and life insurance offerings, short-term disability, and a 401K plan (all benefits are based on eligibility).

This posting is open for thirty (30) days.


Any consideration of a background check would be an individualized assessment based on the applicant or employee's specific record and the duties and requirements of the specific job.