Databricks Engineer

Overview

Remote
Depends on Experience
Accepts corp to corp applications
Contract - W2
Contract - 12 Month(s)

Skills

Databricks
Pyspark
Migration
ETL

Job Details

  • Data Analysis and Mapping: Analyzing the source Salesforce instance and target environment to understand data structures, relationships, and migration requirements.
  • Migration Strategy: Developing and documenting the overall data migration strategy, including data extraction, transformation, loading, and validation processes.
  • Data Modeling: Designing and implementing data models in Databricks that align with the target environment and business requirements.
  • Performance Optimization: Optimizing data pipelines and ETL processes for performance and scalability within the Databricks environment.
  • ETL Pipeline Development: Designing, developing, and testing ETL pipelines using Databricks, Spark, and other relevant technologies to extract, transform, and load data from Salesforce.
  • Data Integration: Integrating data from various sources, including Salesforce, into the Databricks environment.
  • Data Validation and Quality: Implementing data validation and quality checks throughout the migration process to ensure data accuracy and integrity.
  • Performance Monitoring: Monitoring the performance of data pipelines and ETL processes, identifying and resolving performance bottlenecks.
  • Automation: Implementing automated processes for data migration, validation, and monitoring.
  • Stakeholder Management: Collaborating with business stakeholders, product owners, and other vendors teams involved to understand requirements and ensure alignment.
  • Documentation:Creating and maintaining comprehensive documentation for data migration processes, data mappings, and transformation rules.

Post-Migration Support:Providing ongoing support for the migrated data and systems, addressing any issues that may arise.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.