Overview
On Site
Depends on Experience
Contract - W2
Contract - Independent
Contract - 12 Month(s)
Skills
Agile
Amazon Web Services
Apache Avro
Apache Cassandra
Apache Flink
Apache Hadoop
Apache Parquet
Apache Spark
Apache Storm
Artificial Intelligence
Behavior-driven Development
Big Data
Cloud Computing
Conflict Resolution
Continuous Delivery
Continuous Integration
Critical Thinking
Data Engineering
Data Lake
Data Modeling
Data Processing
Data Structure
Databricks
Documentation
Generative Artificial Intelligence (AI)
Git
JSON
Java
Jenkins
MongoDB
NoSQL
Oracle
Orchestration
PostgreSQL
Problem Solving
Programming Languages
Python
Roadmaps
Scala
Scheduling
Step-Functions
Storage
Terraform
Test-driven Development
Job Details
Role: Sr. Data Engineer
Location: New York, NY
Job Type: Contract/W2/Full Time
Job Description:
Must have skill: Python, Spark, AWS, Airflow
Detailed JD:
1.Proficiency in data engineering programming languages (preferably Python, alternatively Scala or Java
2. Proficiency in atleast one cluster computing frameworks (preferably Spark, alternatively Flink or Storm)
3. Proficiency in atleast one cloud data lakehouse platforms (preferably AWS data lake services or Databricks, alternatively Hadoop), atleast one relational data stores (Postgres, Oracle or
similar) and atleast one NOSQL data stores (Cassandra, Dynamo, MongoDB or similar)
Proficiency in atleast one scheduling/orchestration tools (preferably Airflow, alternatively AWS Step Functions or similar)
4. Proficiency with data structures, data serialization formats (JSON, AVRO, Protobuf, or similar), big-data storage formats (Parquet, Iceberg, or similar), data processing methodologies
(batch, micro-batching, and stream), one or more data modelling techniques (Dimensional, Data Vault, Kimball, Inmon, etc.), Agile methodology (develop PI plans and roadmaps), TDD (or
BDD) and CI/CD tools (Jenkins, Git,)
Strong organizational, problem-solving and critical thinking skills; Strong documentation skills
Preferred skills:
Experience using AWS Bedrock APIs
Knowledge of Generative AI concepts (such as RAG, Vector embeddings, Model fine tuning, Agentic AI)
Experience in IaC (preferably Terraform, alternatively AWS cloud formation)
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.