Databricks Data Engineer Architect
Cyma Systems Inc
Dice Job Match Score™
🔢 Crunching numbers...
Job Details
Skills
- API
- Access Control
- Amazon Web Services
- Cloud Computing
- Change Data Capture
- Data Architecture
- Data Integration
- Data Engineering
- Data Flow
- Data Governance
- Conflict Resolution
- Data Migration
- ELT
- Enterprise Application Integration
- Google Cloud Platform
- IT Architecture
- Documentation
- Extract
- Transform
- Load
- Google Cloud
- Data Processing
- Data Quality
- Databricks
- Design Review
- Continuous Delivery
- Continuous Integration
- Data Modeling
- Microsoft Azure
- Offshoring
- PySpark
- Python
- SQL
Summary
Job Title: Databricks Data Engineer Architect
Location: Remote Role
Contract Duration: -6 -12 + months
Job Summary
We are seeking an experienced Databricks Data Engineer Architect to lead the architecture, design, and implementation of enterprise data engineering solutions on the Databricks platform. The ideal candidate will have extensive hands-on experience with Databricks, a proven track record of architecting scalable data solutions, and demonstrated experience migrating and integrating data from Salesforce and NetSuite into Databricks.
The candidate will be responsible for defining the data architecture, establishing data integration patterns, and ensuring the development of reliable, scalable, secure, and high-performing data pipelines. This role will work closely with business stakeholders, technical teams, and offshore data engineers to deliver the project's data engineering objectives.
Key Responsibilities
- Architect, design, and implement end-to-end data engineering solutions using Databricks.
- Lead the design and development of scalable data ingestion and transformation pipelines.
- Design and implement data migration solutions to ingest data from Salesforce and NetSuite into Databricks.
- Define data architecture, data models, integration patterns, and best practices for the Databricks platform.
- Develop and optimize ETL/ELT pipelines using Python, SQL, PySpark, and other relevant technologies.
- Establish data quality, validation, reconciliation, monitoring, and error-handling frameworks.
- Design secure and efficient data processing solutions, including appropriate access controls and governance practices.
- Optimize data pipelines for performance, scalability, reliability, and cost efficiency.
- Collaborate with business stakeholders to understand data requirements and translate them into technical architecture and implementation plans.
- Provide technical leadership and guidance to offshore data engineering resources.
- Review technical designs, code, and implementation approaches to ensure adherence to architectural standards.
- Support testing, deployment, troubleshooting, and production readiness activities.
- Document architecture decisions, data flows, technical specifications, and operational procedures.
Required Qualifications and Experience
- Extensive hands-on experience with Databricks and its data engineering capabilities.
- Proven experience architecting and delivering end-to-end data engineering solutions.
- Demonstrated experience migrating and integrating data from both Salesforce and NetSuite into Databricks.
- Strong proficiency in Python, SQL, and PySpark.
- Extensive experience building and managing ETL/ELT pipelines and complex data integration workflows.
- Strong understanding of data lakehouse architecture, data modeling, and modern cloud data platforms.
- Experience with Delta Lake and related Databricks capabilities.
- Knowledge of data governance, security, data quality, and pipeline monitoring.
- Experience with API-based integrations, incremental data loads, and change data capture where applicable.
- Strong analytical, problem-solving, communication, and technical leadership skills.
- Ability to work collaboratively with onshore stakeholders and offshore delivery teams.
Preferred Qualifications
- Databricks certifications in data engineering or solution architecture.
- Experience with cloud platforms such as Microsoft Azure, AWS, or Google Cloud.
- Experience with orchestration tools, CI/CD pipelines, and automated deployment practices.
- Familiarity with Salesforce APIs, NetSuite APIs, and enterprise application integration patterns.
- Experience delivering enterprise-scale data migration and modernization projects.
Key Deliverables
- Databricks solution architecture and technical design documentation.
- Data ingestion and migration architecture for Salesforce and NetSuite.
- Scalable and production-ready data pipelines.
- Data quality, validation, and reconciliation mechanisms.
- Technical standards and implementation guidelines for the data engineering team.
- Technical leadership, design reviews, and support for project delivery.
- Dice Id: RTL327349
- Position Id: 9130433
- Posted 1 hour ago
Company Info
Cyma Systems is a technology solutions firm based in Connecticut serving mid-sized and Fortune 1000 businesses. We work with a range of corporate commercial clients as well as with the government sector. As a full-service consulting firm, we engage with our clients to share ideas on how we can help their businesses to become more profitable, to get a jump start on competitors by bringing products to market quickly, reducing costs, maximizing the use of existing technologies, driving more sales through the channel, and to bring additional value by forming a mutually beneficial partnership.
We think that the essence of a successful business is its ability to adapt and transform keeping ground realities in sight. At Cyma we work on leveraging the right synergies between technology and people that work towards transforming the outlook and results of your business.
