DataBricks Solution Architect

  • Raritan, NJ
  • Posted 60+ days ago | Updated 10 hours ago

Overview

On Site
$100-110/hr on C2C Or 90-100/hr on W2
Accepts corp to corp applications
Contract - W2
Contract - Independent
75% Travel

Skills

databricks

Job Details

Job Title: Senior Databricks Solution Architect
Location: Raritan, NJ (Onsite)
Duration: Long Term Contract

Responsibilities:
  • Architecture Design: Design scalable and reliable data analytics and machine learning architectures leveraging the Databricks platform to meet business requirements and objectives.
  • Platform Implementation: Implement and configure Databricks clusters, workspaces, notebooks, and other components to support data processing, analytics, and model development workflows.
  • Data Engineering: Develop and optimize data pipelines for ingesting, processing, and transforming large volumes of data from diverse sources using Databricks and related technologies.
  • Data Modeling: Design and implement data models, schemas, and storage structures within Databricks for efficient data organization and querying.
  • Machine Learning: Build and deploy machine learning models using Databricks ML frameworks and libraries, and integrate them into production workflows for predictive analytics and decision support.
  • Performance Optimization: Optimize Databricks jobs, queries, and workflows for performance, scalability, and cost-efficiency, leveraging best practices and optimization techniques.
  • Security and Governance: Implement security controls, access management policies, and data governance frameworks within Databricks to ensure data privacy, compliance, and regulatory requirements.
  • Collaboration: Collaborate with data engineers, data scientists, business analysts, and stakeholders to understand requirements, design solutions, and deliver actionable insights using Databricks.
  • Documentation and Training: Document Databricks architecture, design decisions, configurations, and best practices, and provide guidance and training to team members on Databricks usage and methodologies.
  • Continuous Improvement: Stay abreast of the latest developments in Databricks and related technologies, evaluate new features and capabilities, and recommend enhancements to optimize data analytics workflows.
Qualifications:
  • Proven experience designing, implementing, and managing data analytics and machine learning solutions using Databricks.
  • Strong proficiency in Apache Spark, SQL, Python, and Scala, with hands-on experience developing data pipelines and machine learning models.
  • Expertise in cloud platforms such as AWS, Azure, or Google Cloud Platform, with a focus on data services and analytics offerings.
  • Experience with data engineering tools and frameworks such as Apache Kafka, Apache Airflow, and Apache Hadoop.
  • Knowledge of containerization technologies (e.g., Docker, Kubernetes) and infrastructure-as-code tools (e.g., Terraform) is a plus.
  • Excellent problem-solving and analytical skills, with the ability to troubleshoot complex issues and optimize performance in distributed systems.
  • Effective communication and collaboration skills, with the ability to work with cross-functional teams in a dynamic environment.
  • Databricks certification (e.g., Databricks Certified Associate Developer, Databricks Certified Associate Data Scientist) is desirable but not mandatory.