Overview
On Site
$100-110/hr on C2C Or 90-100/hr on W2
Accepts corp to corp applications
Contract - Independent
Contract - W2
75% Travel
Skills
databricks
Job Details
Job Title: Senior Databricks Solution Architect
Location: Raritan, NJ (Onsite)
Duration: Long Term Contract
Responsibilities:
- Architecture Design: Design scalable and reliable data analytics and machine learning architectures leveraging the Databricks platform to meet business requirements and objectives.
- Platform Implementation: Implement and configure Databricks clusters, workspaces, notebooks, and other components to support data processing, analytics, and model development workflows.
- Data Engineering: Develop and optimize data pipelines for ingesting, processing, and transforming large volumes of data from diverse sources using Databricks and related technologies.
- Data Modeling: Design and implement data models, schemas, and storage structures within Databricks for efficient data organization and querying.
- Machine Learning: Build and deploy machine learning models using Databricks ML frameworks and libraries, and integrate them into production workflows for predictive analytics and decision support.
- Performance Optimization: Optimize Databricks jobs, queries, and workflows for performance, scalability, and cost-efficiency, leveraging best practices and optimization techniques.
- Security and Governance: Implement security controls, access management policies, and data governance frameworks within Databricks to ensure data privacy, compliance, and regulatory requirements.
- Collaboration: Collaborate with data engineers, data scientists, business analysts, and stakeholders to understand requirements, design solutions, and deliver actionable insights using Databricks.
- Documentation and Training: Document Databricks architecture, design decisions, configurations, and best practices, and provide guidance and training to team members on Databricks usage and methodologies.
- Continuous Improvement: Stay abreast of the latest developments in Databricks and related technologies, evaluate new features and capabilities, and recommend enhancements to optimize data analytics workflows.
Qualifications:
- Proven experience designing, implementing, and managing data analytics and machine learning solutions using Databricks.
- Strong proficiency in Apache Spark, SQL, Python, and Scala, with hands-on experience developing data pipelines and machine learning models.
- Expertise in cloud platforms such as AWS, Azure, or Google Cloud Platform, with a focus on data services and analytics offerings.
- Experience with data engineering tools and frameworks such as Apache Kafka, Apache Airflow, and Apache Hadoop.
- Knowledge of containerization technologies (e.g., Docker, Kubernetes) and infrastructure-as-code tools (e.g., Terraform) is a plus.
- Excellent problem-solving and analytical skills, with the ability to troubleshoot complex issues and optimize performance in distributed systems.
- Effective communication and collaboration skills, with the ability to work with cross-functional teams in a dynamic environment.
- Databricks certification (e.g., Databricks Certified Associate Developer, Databricks Certified Associate Data Scientist) is desirable but not mandatory.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.