Overview
Hybrid
Depends on Experience
Contract - W2
Contract - 60 Month(s)
No Travel Required
Skills
Data Governance
Planning
Redshift
S3
Stakeholder Communication
cloud computing
cost - efficiency
data engineering
solutions architecture
Job Details
- Architecture Design: Designing end-to-end data solutions architecture on AWS leveraging Databricks capabilities to meet business requirements.
- Infrastructure Planning: Planning and architecting Databricks clusters, storage, and networking components on AWS to ensure scalability, reliability, and performance.
- Data Governance: Establishing data governance frameworks, access controls, and data lifecycle management policies within Databricks deployments on AWS to ensure compliance and security.
- Solution Implementation: Leading the implementation of Databricks solutions on AWS, including data ingestion, processing, storage, and analytics capabilities.
- Integration: Integrating Databricks with other AWS services such as S3, Glue, Redshift, Athena, and IAM to build comprehensive data platforms and analytics ecosystems.
- Performance Optimization: Optimizing Databricks workloads, Spark jobs, and AWS infrastructure configurations for performance, cost-efficiency, and scalability.
- Best Practices Advocacy:Advocating best practices in data engineering, data architecture, cloud computing, and Databricks usage across the organization.
- Team Leadership: Providing technical leadership and guidance to data engineering teams, architects, and developers working on Databricks projects on AWS.
- Vendor Management: Collaborating with AWS account teams and Databricks representatives to optimize licensing, support, and service agreements.
- Training and Enablement: Providing training, mentoring, and knowledge sharing sessions to enable teams to effectively utilize Databricks and AWS services.
- Continuous Improvement: Monitoring industry trends, evaluating new technologies, and driving continuous improvement initiatives to enhance Databricks architecture and AWS infrastructure.
- Documentation and Governance: Documenting architecture designs, implementation details, and best practices, and establishing governance mechanisms to ensure consistency and alignment with organizational standards.
- Stakeholder Communication: Communicating with business stakeholders, IT leadership, and other teams to understand requirements, provide updates, and gather feedback on Databricks architecture and AWS integration.