Data Engineer

data warehouse, Scala, Python, Java
Full Time
Depends on Experience
Travel not required

Job Description

Job Duties 

  • Strong problem-solving skills with acute attention to detail and ability to meet tight deadlines and project plans.
  • Design and develop complex, large scale data structures and pipelines to organize, collect and standardize data to generate insights and address analytical/reporting needs
  • Design standards and best practices into new data engineering solutions.
  • Proactively identify performance and data quality problems and drive to remediate them.
  • Manage code versions and deployment process in source control and coordinate changes across team.
  • Partner with Product Management and Operation teams to proactively identify monitoring gaps and improving upstream data collection and advocating for data driven thinking. 
  • Assist in running code reviews, guide team to follow standards, supporting team members in brainstorming and challenging designs.
  • Solve and help other team members complex data issues around data integration of structured and unstructured sets other data processing incidents.

Required Skills

  • Bachelor’s degree in Computer Science, Computer Engineering, Information Systems, or related technology.
  • Minimum of 4 years of relevant data processing experience; 2+ years utilizing cloud-based services.
  • Experience with large scale data lake and data warehouse implementations on cloud. 
  • Significant experience with programming languages Scala, Python, Java, etc.
  • Technical proficiency including RDBMS, DBMS, AWS, cloud computing, NoSQL, Apache Accumulo, Hadoop technologies (Hive, Pig).
  • Expertise building data lakes and data warehouses in AWS using a variety of cloud services EMR, EC2, S3, Kinesis, Lambda, DynamoDB, SNS, SQS, Glue, Athena, Redshift.
  • Experience with web services, API Integration and Data streaming.
  • Experience in data analysis and writing and developing complex queries using SQL.
  • Experience in Agile methodologies (Kanban and SCRUM).
  • Ability to work autonomously and with small cross-functional teams.
  • Strong communication skills including technical documentation, development and delivery of demonstrations and presentations and the ability to listen and communicate effectively with functional leaders.

Desired Skills

  • Master’s in computer science, Engineering, or Information Systems
  • Experience with Metadata management and Data Governance.
  • Experience with Tableau, Power BI or similar data visualization tools
  • knowledge with Elastic, Hosted Graphite, Grafana or similar data streaming and monitoring technologies
  • Knowledge of DevOps and CloudOps.
  • Experience in airline / aviation, internet, or telecommunications

 

Dice Id : 10382761
Position Id : 7115240
Originally Posted : 3 months ago
Have a Job? Post it