Databricks Architect - Tarrytown, NY [Hybrid]
OMG Technologies
Dice Job Match Score™
🔢 Crunching numbers...
Job Details
Skills
- Apache Spark
- Amazon Redshift
- Amazon S3
- Amazon Web Services
- Apache NiFi
- Artificial Intelligence
- Change Data Capture
- Customer Facing
- Data Analysis
- Data Governance
- Data Migration
- Data Quality
- Databricks
- Electronic Health Record (EHR)
- Machine Learning (ML)
- Optimization
- Pharmaceutics
- Extract, Transform, Load
- IT Management
- Migration
- SQL
- Scalability
- Software Design
- Solution Architecture
- JD
- PySpark
- Python
- Real-time
- Roadmaps
- Streaming
- Workflow
Summary
Job Details:
Location: Tarrytown, NY
Work Arrangement: Hybrid — 3 days onsite per week
Contract Duration: 1 year
Role Summary
We are looking for a strong Databricks Architect to lead enterprise-scale Lakehouse architecture and implementation on AWS. The role involves designing scalable data solutions, modernizing existing data platforms, establishing governance standards, and leading architecture discussions with customers.
Key Responsibilities
Lead enterprise-scale Databricks Lakehouse architecture and implementation on AWS.
Drive migration of EMR, NiFi, Redshift, and legacy ETL workloads to Databricks.
Evaluate newer Databricks capabilities, including Serverless, Lakehouse Federation, Genie, Databricks Apps, Lakebase, and AI/ML.
Architect scalable batch, streaming, CDC, and real-time data solutions.
Design reusable frameworks using Delta Lake, PySpark, Auto Loader, DLT, and Databricks Workflows.
Define governance, security, lineage, and data-quality standards using Unity Catalog.
Establish best practices for performance, scalability, and cost optimization.
Lead customer workshops, architecture reviews, and solution assessments.
Define target-state architecture, migration roadmaps, and reusable patterns.
Required Skills & Qualifications
15+ years of overall Data & Analytics experience.
4+ years of hands-on Databricks experience as specified in the JD.
Strong experience with Databricks, Delta Lake, Unity Catalog, DLT, and Auto Loader.
Proficiency in PySpark, Spark SQL, Python, and Structured Streaming.
AWS experience with S3, IAM, Glue, Redshift, EMR, and Lambda.
Experience with Airflow, CDC, and enterprise data migration.
Knowledge of data governance, security, and observability.
Strong customer-facing, solution architecture, and technical leadership skills.
Preferred Qualifications
Healthcare or pharmaceutical domain experience is preferred but not mandatory.
Experience with enterprise-scale data platform modernization and migration is beneficial.
- Dice Id: 91093052
- Position Id: 9129498
- Posted 3 hours ago
Company Info
About OMG Technologies
From our humble beginnings as a small IT service provider in 2006, OMG Technologies has grown to become a reliable provider of management and IT consulting services, while still maintaining agility of a small company. With multiple clients in various industries, OMG Technologies continues to grow at a good, steady and consistent pace based on our client focused approach.
Whatever it Takes to deliver value is our CORE approach