Job title: Data engineer
Location: Richmond, VA(Remote)
Job Description:
· BS or MS degree in computer science, computer engineering, or other technical discipline, or equivalent work experience.
· 5+ years of hands-on software development experience with Big Data & Analytics solutions
· Google Cloud Platform Cloud - Big Query, Airflow, DataProc, PubSub, Hadoop Hive, Spark, Python, shell scripting,
· Strong experience in designing, developing, and optimizing data pipelines for large-scale data processing, transformation, and analysis using Big Data and Google Cloud Platform technologies.
· Proficiency in SQL and database systems, with experience in designing and optimizing data models for performance and scalability.
· Experience of building event processing pipelines with Kafka or Google Cloud Platform PubSub.
· Design and development experience with Airflow, PubSub Kafka, Git, Jenkins is desirable.
· Knowledge of distributed (multi-tiered) systems, algorithms & relational databases.
· Strong Object-Oriented Programming skills and design patterns.
· Experience with CICD pipelines, Automated test frameworks, and source code management tools (XLR, Jenkins, Git).
· Good knowledge and experience with configuration management tools like GitHub
· Ability to analyze complex data engineering problems, propose effective solutions, and implement them effectively.
· Ability and willingness to learn, adopt and build solutions using the Enterprise frameworks established for Big Data
· Looks proactively beyond the obvious for continuous improvement opportunities.
· Communicates effectively with product and cross functional team.
· Willingness to learn new technologies and leverage them to their optimal potential.
· Understanding of various SDLC methodologies, familiarity with Agile & scrum ceremonies.
Never miss an opportunity! Create an alert based on the job you applied for.
