Title: Big Data Engineer
Location: Sunnyvale, CA
Duration: 6 Months
- Build distributed, scalable, and reliable data pipelines that ingest and process data at scale and in real-time.
- Collaborate with other teams to design and develop and deploy data tools that support both operations and product use cases.
- Perform offline analysis of large data sets using components from the Hadoop ecosystem.
- Evaluate and advise on technical aspects of open work requests in the product backlog with the project lead.
- Own product features from the development phase through to production deployment.
- Evaluate big data technologies and prototype solutions to improve our data processing architecture.
- BS in Computer Science or related area
- Around 6 years software development experience
- Minimum 2 Year Experience on Big Data Platform
- Must have active current experience with Scala, Java, Python, Oracle, HBase, Hive
- Flair for data, schema, data model, how to bring efficiency in big data related life cycle
- Understanding of automated QA needs related to Big data
- Understanding of various Visualization platform
- Experience in Cloud providers like AWS preferable
- Proficiency with agile or lean development practices
- Strong object-oriented design and analysis skills
- Excellent written and verbal communication skills
Top skill sets / technologies in the ideal candidate: * Programming language -- Java (must), Python, Scala
* Database/Search – Oracle, complex SQL queries, stored procedures, performance tuning concepts, SOLR, AWS RDS, AWS Aurora
* Batch processing -- Hadoop MapReduce, Cascading/Scalding, Apache Spark, AWS EMR * Stream processing -- Spark streaming, Kafka, Apache Storm, Flink * NoSQL -- HBase, Cassandra, MongoDB * ETL/ELT Frameworks including monitoring, alerting, restart * Code/Build/Deployment -- git, svn, maven, sbt, Jenkins, bamboo: PERSONAL QUALIFICATIONS * Excellent communication and decision making skills are essential. * Strong analytical, problem solving and decision-making skills.
* Zeal to lean new technologies, frameworks and appetite for growth * Identify project risks and recommend mitigation efforts. * Identify project issues, communicate them and assist in their resolution. * Assist in continuous improvement efforts in enhancing project team methodology and performance. * Cooperative team focused attitude