Lead Data Engineer | New York City, NY | Long Term - Only Local Profiles

Job Title: Lead Data Engineer

Location: New York City, NY Only Local Candidate

Job Type: Contract

Client interview Face to Face

Job Description:

Required Skills:

• Proficiency in data engineering programming languages (preferably Python, alternatively Scala or Java)
• Proficiency in atleast one cluster computing frameworks (preferably Spark, alternatively Flink or Storm)
• Proficiency in atleast one cloud data lakehouse platforms (preferably AWS data lake services or Databricks, alternatively Hadoop), atleast one relational data stores (Postgres, Oracle or similar) and atleast one NOSQL data stores (Cassandra, Dynamo, MongoDB or similar)
• Proficiency in atleast one scheduling/orchestration tools (preferably Airflow, alternatively AWS Step Functions or similar)
• Proficiency with data structures, data serialization formats (JSON, AVRO, Protobuf, or similar), big-data storage formats (Parquet, Iceberg, or similar), data
• processing methodologies (batch, micro-batching, and stream), one or more data modelling techniques (Dimensional, Data Vault, Kimball, Inmon, etc.), Agile methodology (develop PI plans and roadmaps), TDD (or BDD) and CI/CD tools (Jenkins, Git,)
• Strong organizational, problem-solving and critical thinking skills; Strong documentation skills

Preferred skills:

• Experience using AWS Bedrock APIs
• Knowledge of Generative AI concepts (such as RAG, Vector embeddings, Model fine tuning, Agentic AI)
• Experience in IaC (preferably Terraform, alternatively AWS cloud formation)
  
  Applicant Consent:
  By submitting your application, you agree to ApTask's ()  and  , and provide your consent to receive SMS and voice call communications regarding employment opportunities that match your resume and qualifications. You understand that your personal information will be used solely for recruitment purposes and that you can withdraw your consent at any time by contacting us at or . Message frequency may vary. Msg & data rates may apply.