Databricks Consultant

Databricks, Azure, Java, Spark, Scala, PySpark
Contract W2, Contract Corp-To-Corp, 12 Months
Depends on Experience
Travel not required

Job Description

  • Installation and Configuration
    • Install/Configure Databricks in Azure
    • Define Databricks, Spark, Java, Scala, PySpark versions
    • Assess and recommend memory and CPU requirements   
    • Model the workload on the cluster
    • Define and assess high data read and write volumes/storage
    • Estimate the expected size.       
    • Define plan/procedures to use Java/scala/pyspark
    • Define security standards
    • Define different user profiles and RBAC privileges using Azure AD
    • Provide CI/CD pipelines
    • EventHub, queue, log analytics, monitoring and other azure options for integrations
    • Provide storage considerations.
  • Data modeling and application solution architecture
    • Review and analyze existing data sources
    • Review and provide assessment and recommendations for data model and application integration to support American data requirements and to support additional features
    • Provide best practices for data modeling
    • Provide architecture guidance and assessment.
  • Data ingestion assessment and optimization
    • Assess and support updates to the current ingestion processing, while providing recommended best practices to include additional data sources and recommended best practices for recurrent updating of the existing data with updated data sources
  • Query optimization
    • Review and provide recommendations and best practices to assure reliability and performance expectations are met
  • Visualization (define output)
    • End user tooling considerations
  • Testing
    • Query and code development and unit testing
    • Performance testing and validation

Education:

Bachelor's degree in Computer Science/Engineering or related field

Dice Id : 10120357
Position Id : Databricks0921
Originally Posted : 1 month ago
Have a Job? Post it