Overview
On Site
$DOE
Full Time
Contract - Independent
Contract - W2
Contract - Long Term
Skills
Big Data
Kafka
Hive
Job Details
Job Title : Data Engineer
Location : San Francisco, California (Hyrbid - 3 days a week in office)
Duration : Long Term
Location : San Francisco, California (Hyrbid - 3 days a week in office)
Duration : Long Term
Responsibilities
Job Description:
Design, develop, test, deploy, and maintain scalable Big Data solutions.
Manage project priorities, deadlines, and deliverables across multiple initiatives.
Build and optimize data pipelines using Spark and other distributed processing frameworks.
Develop software using modern programming languages such as Java or Scala.
Work with both relational (e.g., MySQL, Postgres) and NoSQL databases.
Integrate and manage data workflows using tools like Hive, Impala, OOZIE, Airflow, NIFI, and Kafka.
Write shell scripts and manage deployments in Linux/Unix environments.
Collaborate with cross-functional teams in agile ceremonies including daily stand-ups, story pointing, and retrospectives.
Mentor junior engineers and foster a culture of continuous improvement and technical excellence.
Extensive experience as a Big Data engineer, with expertise in cloud or on-premise data platforms.
Proficient in at least one modern programming language (Java, Scala, or Python).
Strong foundation in computer science principles including object-oriented design, data structures, algorithms, and complexity analysis.
Deep experience with Spark and distributed data processing.
Hands-on experience with both SQL and NoSQL databases.
Familiarity with data orchestration and streaming tools (Kafka, Airflow, NIFI, etc.).
Comfortable working in Linux/Unix environments with basic shell scripting skills.
Proven ability to design and develop software systems at scale.
Strong understanding of Agile methodologies, with the ability to drive iterative delivery and cross-team collaboration.
Strong communicator with the ability to explain complex concepts to both technical and non-technical audiences, and to influence stakeholders across product, engineering, and acquisition teams.
Bachelor's degree in Computer Science, Data Analytics, Mathematics, Software Engineering, or a related field or equivalent practical experience
Job Description:
Design, develop, test, deploy, and maintain scalable Big Data solutions.
Manage project priorities, deadlines, and deliverables across multiple initiatives.
Build and optimize data pipelines using Spark and other distributed processing frameworks.
Develop software using modern programming languages such as Java or Scala.
Work with both relational (e.g., MySQL, Postgres) and NoSQL databases.
Integrate and manage data workflows using tools like Hive, Impala, OOZIE, Airflow, NIFI, and Kafka.
Write shell scripts and manage deployments in Linux/Unix environments.
Collaborate with cross-functional teams in agile ceremonies including daily stand-ups, story pointing, and retrospectives.
Mentor junior engineers and foster a culture of continuous improvement and technical excellence.
Extensive experience as a Big Data engineer, with expertise in cloud or on-premise data platforms.
Proficient in at least one modern programming language (Java, Scala, or Python).
Strong foundation in computer science principles including object-oriented design, data structures, algorithms, and complexity analysis.
Deep experience with Spark and distributed data processing.
Hands-on experience with both SQL and NoSQL databases.
Familiarity with data orchestration and streaming tools (Kafka, Airflow, NIFI, etc.).
Comfortable working in Linux/Unix environments with basic shell scripting skills.
Proven ability to design and develop software systems at scale.
Strong understanding of Agile methodologies, with the ability to drive iterative delivery and cross-team collaboration.
Strong communicator with the ability to explain complex concepts to both technical and non-technical audiences, and to influence stakeholders across product, engineering, and acquisition teams.
Bachelor's degree in Computer Science, Data Analytics, Mathematics, Software Engineering, or a related field or equivalent practical experience
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.