We're looking for a senior Big Data Engineer is keen on diving deep into big data using Big Data Ecosystem to perform data aggregations, data curation, and data ETL to be consumed by data analytics, data science, and BI Analytics solutions.
This person will work with a highly integrated, cross-discipline agile team and will be responsible for building and supporting modern Big Data Engineering solutions leveraging enterprise data lake across various business units
• Experience with Big Data Ecosystem using Snowflake Data Warehouse, Snowpipe, Streams, Tasks, Query Optimization, Tuning, and Best Practices
• Core competency in SQL optimization efficient Joins – MySQL & Hive
• Automation of ETL Jobs and Orchestration using scripts, pipelines, and workflows.
• Good exposure to AWS Services, including EMR, AWS Data Pipeline, AWS Aurora, S3, Lambda, Amazon Kinesis, Amazon Simple Queue Service (Amazon SQS), and Amazon Simple Notification Service (Amazon SNS).
• AWS/Big Data/Realtime data processing Tech Stack development environment with hands-on experience designing and implementing solutions.
• Hands-on experience working on Production Solutions with an understanding of Scalability, Reliability, Uptime, and Cost Optimization of a solution.
• Some exposure to GitLab, Jenkins, and Jira tools for Agile development & CI/CD
• Participate in design review sessions and ensure all solutions are aligned with pre-defined architectural specifications
• Work with distributed teams to design and develop frameworks, solution accelerators, proofs of concept, and external customer-facing products.
• Evaluate and incorporate new technologies into new and existing frameworks and solutions
• Collaborate with and mentor members of the team and other coworkers
• Analysis and debugging to diagnose and fix client issues, including root cause analysis and future issue avoidance
• Ability to understand business requirements and translate them into technical requirements
Nice to have:
Experience with Java Enterprise technical stack, AWS Infra basic understanding, Data Governance & Security