Role Overview
The Big Data Engineer/ Developer will support data engineering initiatives for the GIS team, focusing on building, optimizing, and maintaining scalable Big Data solutions. The role involves close collaboration with business, analytics, and technology teams to deliver near real-time and batch data pipelines.
Required Skills & Experience
Core Technical Skills
Strong SQL skills one or more of MySQL, HIVE, Impala, SPARK SQL
Hands-on experience with Hadoop / Big Data and distributed systems
Experience working with Spark, Sqoop, Kafka, MapReduce
Experience with NoSQL databases such as HBase, SOLR
Exposure to CDP or HDP, Cloudera or Hortonworks
Experience with Elastic Search, Kibana, etc.
Data Engineering & Processing
Experience in data ingestion from message queues, file shares, REST APIs, and relational databases
Experience working with data formats such as JSON, CSV, XML
Hands-on experience with Spark Structured Streaming
Experience in end-to-end design and build of Near-Real Time and Batch Data Pipelines
Performance tuning experience with Spark, MapReduce, or SQL jobs
Programming & Systems
Hands-on programming experience in at least one of Scala, Python, PHP, or Shell Scripting
Strong proficiency with Linux operating systems
Experience using source code and version control systems such as SVN, Git, Bitbucket
Experience working with Jenkins and JAR management
Development Methodologies & Soft Skills
Experience working in Agile development environments
Strong understanding of the Software Development Life Cycle (SDLC)
Self-starter who can work with minimal supervision
Ability to work effectively in a team of diverse skill sets
Strong analytical and problem-solving skills
Ability to comprehend customer requirements and provide the appropriate solution
Willingness to troubleshoot, resolve issues, and deep-dive into complex problems
Ability and willingness to continuously learn and adapt to new technologies
Never miss an opportunity! Create an alert based on the job you applied for.
