Data Engineer

Introduction:

As a Data Engineer, you will be responsible for designing, developing, and maintaining data pipelines and infrastructure to support our data analytics and machine learning initiatives. You will work closely with data scientists, analysts, and other stakeholders to ensure that our data systems are efficient, scalable, and reliable.

Responsibilities:

• Design, build, and maintain scalable data pipelines to ingest, transform, and store data from various sources.
• Optimize data infrastructure for performance, reliability, and cost efficiency.
• Collaborate with data scientists and analysts to understand their data requirements and provide them with the necessary tools and infrastructure.
• Implement data quality and monitoring processes to ensure the integrity of our data.
• Work with cross-functional teams to identify opportunities for data-driven decision-making and develop solutions to support those initiatives.

Requirements:

Required Skills:

• Proficiency in SQL and experience working with relational databases.
• Strong programming skills in languages such as Python, Java, or Scala.
• Experience with data modeling, ETL processes, and data warehousing.
• Knowledge of distributed computing frameworks such as Hadoop, Spark, or Flink.
• Understanding of cloud computing platforms such as AWS, Google Cloud Platform, or Azure.

Preferred Skills:

• Experience with real-time data processing and stream processing technologies.
• Knowledge of data visualization tools such as Tableau or Power BI.
• Familiarity with machine learning concepts and algorithms.
• Excellent communication and collaboration skills.
• Ability to work independently and in a team environment.