- Java OR Golang
- Good experience with Data Bricks
- Experience with Java OR Golang
What the ideal candidate looks like:
- 5+ years of experience designing, building, deploying, testing, maintaining, monitoring and owning scalable, resilient and distributed data pipelines.
- High Proficiency in at least two of Scala, Python, Spark or Flink applied to large scale data sets.
- Strong understanding of workflow management platforms (Airflow or similar).
- Familiarity with advanced SQL.
- Expertise with big data technologies (Spark, Flink, Data Lake, Presto, Hive, Apache Beam, NoSQL, ...).
- Knowledge of batch and streaming data processing techniques.
- Obsession for service observability, instrumentation, monitoring and alerting.
- Understanding of the Data Lifecycle Management process to collect, access, use, store, transfer, delete data.
- Strong knowledge of AWS or similar cloud platforms.
- Expertise with CI/CD tools (CircleCI, Jenkins or similar) to automate building, testing and deployment of data pipelines and to manage the infrastructure (Pulumi, Terraform or CloudFormation).
- Understanding of relational databases (e.g., MySQL, PostgreSQL), NoSQL databases (e.g., key-value stores like Redis, DynamoDB, RocksDB), and Search Engines (e.g., Elasticsearch). Ability to decide, based on the use case, when to use one over the other.
- Familiarity with recommendation and search to personalize the experience for millions of users across million items.
Please send the resume to or you can reach me at Directlt;/p>