Primary Skills: Data Architect, Azure Databricks, Data Modeling, SQL, Pyspark/Python
Job Description:
We are seeking a highly skilled Cloud Data Architect (Azure/AWS) to join our team for an exciting data project. As a Technical Data Architect, you will be responsible for designing Azure data platforms with large-scale implementation experience. You will have hands-on coding skills in Azure Stacks, SQL, PySpark/Python, and proven experience with Databricks projects.
Responsibilities:
- Design Azure data platforms with large-scale implementation experience.
- Develop enterprise solutions using frameworks, enterprise patterns, database design, data modeling, and development in Azure.
- Utilize Azure Data Factory for ingestion, Data Lake Gen 2 and Azure SQL Server for storage, Azure Analysis Services for transformations, and Azure Databricks & HDInsight.
- Utilize Spark, Python, and PySpark ETL for data processing.
- Analyze the quality and granularity of real-time streaming data from IoT devices.
- Compare and evaluate various ETL tools like Databricks and ADF.
- Validate and suggest best practices for HDInsight ETL.
- Utilize Collibra as a data governance tool and validate the existing quality of the metadata catalog.
- Perform data analysis, profiling, and data modeling using modern technologies, methodologies, patterns, and industry standards.
- Consult and implement data governance and data quality management.
- Collaborate with Scrum Master and PMO to organize deliverables and contribute actively to planning and scheduling the solution implementation.
- Support the presales team in building RFPs.
Requirements:
- 10+ years of IT experience, including 5+ years of experience in Azure cloud technology.
- Proficiency in Azure Databricks, SQL, and PySpark/Python.
- Strong experience in Spark, Python, and PySpark ETL.
- Understanding of Event Hub configuration.
- Knowledge of HDInsight ETL and ability to suggest best practices.
- Familiarity with Collibra as a data governance tool.
- Experience in data analysis, profiling, and data modeling.
- Strong analytical and quantitative problem-solving abilities.
- Exposure to frameworks, reusable components, accelerators, and CI/CD automation.
- Excellent presentation, written, and verbal communication skills.
- Ability to collaborate and communicate effectively with different levels of the company.
- Experience with machine learning on Azure ML.
Never miss an opportunity! Create an alert based on the job you applied for.
