Join us as we build a next-generation Enterprise Analytics Platform
to reengineer a real estate enterprise. As a Senior Big Data Platform Engineer, you will be joining our Enterprise Analytics Platform Team to build EAP platform from ground up to serve agents, brokers, home buyers and sellers. What we're looking for:
You're a talented, creative, and motivated Data Platform Engineer who loves to build data platform tools and ready to work with a team of individuals who share your passion. With a related degree under your belt--and relevant experience--you're ready to take your programming and data knowledge to the next level. You enjoy building tools such as Data Ingestion Platform to ingest data from diverse sources to Data Lake, building data lake tools to ingest data, monitor pipelines, and grant access to data. With your unwavering commitment to quality, excellent data/cloud skills, and collaborative work ethic, you'll do great things here at Realogy AI Lab.What you'll do:
- You'll be responsible for designing and building high performance, scalable data solutions that meet the needs of millions of agents, brokers, home buyers, and sellers.
EDUCATION AND EXPERIENCE/SPECIAL SKILLS/TECHNOLOGIES/TOOLS REQUIREMENTS
- You'll work with other Data Engineers for build out of Next Generation Data Ingestion Platform.
- You'll design and develop data ingestion pipelines for batch and real-time streaming of data from in-house OLTP systems and third-party data.
- You'll work with team to design and develop Data Lake to store and process 10s of terabyte of data.
- You'll work with team to design Data Lake CLI to manage Data Lake Storage and Access.
- You will design and develop ETL pipelines to process data in data lake for descriptive and prescriptive reporting.
- You'll develop ETL data pipelines to build Enterprise Data Models for Property, Agent, Broker, office and other master entities.
- You will design and develop CI/CD process for continuous delivery in AWS Cloud.
- You'll design, develop, and test robust, scalable data platform components.
- You'll work with a variety of teams and individuals, including product engineers to understand their data pipeline needs and come up with innovative solutions.
- Bachelor's in Computer Science, Engineering, or related technical discipline or equivalent combination of training and experience
- 10+ years programming experience: building business logic layers and back-end systems for high-volume pipelines
- 2 years' experience with Golang
- 2 years' experience with Spark, Spark SQL, and Scala
- 2 years' experience using AWS Data Services: any combination of DMS, EMR, Glue, Athena, S3, CloudWatch, Lambda or IAM
- 2 years' experience with high-speed messaging frameworks and streaming (Kafka)
- 1-year exp. with data architecture, ETL and processing of structured and unstructured data
- 2 years' experience with DevOps tools (any combination of GitHub, Travis CI or Jira) and methodologies (Lean, Agile, Scrum, Test Driven Development)
Excellent written and verbal communication skills in English