Overview
Remote
Depends on Experience
Full Time
Skills
Data Architecture
Databricks
PySpark
SQL Azure
NoSQL
Data Modeling
Apache Kafka
Microsoft Azure
Apache Spark
Job Details
Responsibilities:
- Design, code, test, document, and maintain high-quality and scalable data pipelines/solutions in cloud .
- Ingest and transform data using variety of technologies from variety of sources (API s, streaming, Files, Databases .).
- Develop reusable patterns and encourage innovation that will increase team s velocity.
- Design and develop applications in an agile environment, deploy using CI/CD.
- Participate with prototypes as well as design and code reviews, own or assist with incident and problem management.
- The ideal candidate will be a self-starter who can learn things quickly, who is enthusiastic and actively engaged.
experience with the following technologies:
- Databricks, Python, Spark, pyspark, SQL, Azure Data factory.
- Design and Implementation of Datawarehouse/Datalake [Databricks/snowflake].
- Data architecture, Data modelling.
- Github actions/Jenkins or similar CICD tool, Cloud CICD, GitHub.
- Scheduling tools like Airflow / Tivoli Work scheduler.
- NoSQL and relational databases.
- Apache Kafka.
- Data ingestions from variety of APIs.
- Working in Agile/Scrum environment.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.