Overview
Hybrid
Depends on Experience
Contract - W2
Contract - 12 Month(s)
Skills
Bigdata
Spark
Python
Java
scala
Kafka
AWS
Job Details
Position: Senior Big Data Engineer
Location: Costa Mesa, CA
Duration: 12+ Months
Our team is responsible of dealing with huge amount of data and also have an integration from various systems, create a required pipeline, and schedule, also team is dealing with steaming data for Realtime applications. What you will do here:
- Design & build all batch/Realtime components/data pipelines lines/integrations for our MLOps platform on AWS.
- Collaborate with geographically distributed cross-functional teams.
- Participate in on-call rotation with the rest of the team to handle production incidents.
About the Role:
- Big Data Experience 4+ years experience.
- Strong in Python/SCALA/Java programming
- Strong understanding/experience in SPARK.
- Good exposure on Kafka/streaming.
- AWS Experience is a must EMR, ECS, Elastic Search, S3, Cloud Watch, Lambda, Step Functions,
- Good communication skills to work with various stakeholders on operational support work
- DEVOPS Experience is a must CI/CD, Jenkins Build, Docker, ECR Repo
Nice to have skills:
- Experience with AWS
- Experience Terraform is good to have.
- Experience with various Python packaging options such as Wheel, PEX or Conda.
- Experience with various automated testing framework is plus.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.