Cynosure Technologies LLC
Cynosure Technologies LLC

data Engineer

New York, NY, US • Posted 8 hours ago • Updated 8 hours ago
Contract W2
6 Months
No Travel Required
On-site
Depends on Experience
Fitment

Dice Job Match Score™

🔢 Crunching numbers...

Job Details

Skills

  • Amazon Web Services
  • Analytics
  • Apache Hadoop
  • Apache Hive
  • Apache Pig
  • Apache Spark
  • Batch Processing
  • Cloud Computing
  • Communication
  • Computer Science
  • Conflict Resolution
  • Data Lake
  • Data Quality
  • Data Science
  • Data Security
  • Data Warehouse
  • DevOps
  • Docker
  • FOCUS
  • Good Clinical Practice
  • Google Cloud
  • Google Cloud Platform
  • Java
  • Machine Learning (ML)
  • Microsoft
  • Privacy
  • Problem Solving
  • Programming Languages
  • Python
  • Real-time
  • Software Engineering
  • Streaming
  • Workflow Management

Summary

title : Data Engineer

location :  NY/Chicago/Dallas (Onsite)

Responsibilities:
  • Design, develop, and maintain batch processing pipelines for large-scale data transformation and analysis using technologies like Apache Hadoop, Pig, Hive, Spark, or similar frameworks.
  • Experience with Airflow and/or Cloud Composer for workflow management and orchestration
  • Work with data scientists and product managers to develop and deploy data pipelines for data warehousing and data lake use cases
  • Collaborate with other engineers to integrate batch processing systems with our data streaming and real-time analytics infrastructure
  • Ensure data quality, data security, and data privacy across all batch processing systems
  • Develop and implement batch processing jobs using programming languages like Java or Python. (Experience with both is a plus)
  • Work with public cloud providers such as Google Cloud Platform (Google Cloud Platform), Amazon Web Services (AWS), or Microsoft Azure
  • Experience with Dataproc is a strong preference
  • Optimize batch pipelines for performance and resource efficiency.
  • Monitor and troubleshoot batch jobs to ensure successful execution and data quality.
  • Implement DevOps principles and practices to ensure efficient and reliable deployment of batch processing systems
  • Knowledge of containerization technologies such as Docker and Kubernetes
Requirements:
  • Bachelor's degree in Computer Science or a related field
  • 5+ years of experience in software engineering, with a focus on batch processing and analytics
  • Strong programming skills in Java or Python
  • Experience with a public cloud provider, with a focus on Google Cloud Platform
  • Strong experience in batch processing systems such as Hadoop, Pig, Hive, and Spark
  • Experience with Dataproc is a plus
  • Knowledge of DevOps principles and practices
  • Understanding of containerization technologies such as Docker and Kubernetes
  • Excellent problem-solving skills, with the ability to work independently and collaboratively
  • Excellent communication skills, with the ability to explain complex technical concepts to non-technical stakeholders
  • Experience with machine learning and data science technologies is a plus
  • Certification in a public cloud provider or a relevant technology is a plus
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 10123013
  • Position Id: 9139959
  • Posted 8 hours ago
Contact the job poster
Viswanath Yerubandi

Viswanath Yerubandi

Cynosure Technologies LLC Recruiter @ Cynosure Technologies LLC

View Profile
Create job alert
Never miss an opportunity! Create an alert based on the job you applied for.
More jobs at Cynosure Technologies LLC in New York, NY