Job Summary:
We are seeking a highly experienced Senior Databricks Architect & Developer to design, develop, and optimize scalable data solutions on the Databricks platform. The ideal candidate will have strong expertise in data architecture, big data processing, cloud platforms, and hands-on development experience using Apache Spark and Databricks.
Key Responsibilities:
•Design and implement scalable data architectures using Databricks.
•Architect end-to-end data pipelines using Apache Spark, Delta Lake, and cloud-native services.
•Develop and optimize ETL/ELT workflows for structured and unstructured data.
•Lead migration of on-premise data platforms to cloud-based Databricks environments.
•Implement Lakehouse architecture using Azure Databricks, AWS Databricks, or Google Cloud Databricks.
•Build and manage data models, data governance, and security frameworks.
•Optimize performance and cost efficiency of big data workloads.
•Collaborate with data engineers, data scientists, DevOps, and business stakeholders.
•Provide technical leadership, code reviews, and best practices guidance.
•Implement CI/CD pipelines and DevOps practices for data engineering workflows.
•Ensure data quality, integrity, and compliance standards.
Required Skills & Qualifications:
•7+ years of experience in Data Engineering / Big Data Development.
•3+ years of hands-on experience with Databricks architecture and development.
•Strong programming skills in Python, Scala, or SQL.
•Deep expertise in Apache Spark (Core, SQL, Structured Streaming).
•Experience implementing Lakehouse architecture with Delta Lake.
•Strong knowledge of data warehousing concepts (Star/Snowflake schema, SCD, partitioning).
•Amazon Web Services EMR, S3, Lambda, Glue, Redshift, Cloud infrastructure
•Experience with orchestration tools (Airflow, ADF, etc.).
•Knowledge of data governance, security, and access control frameworks.
•Experience with version control and CI/CD tools (Git, Azure DevOps, Jenkins).
“Taras Technology, LLC is an EEO/AA Employer: women, minorities, the disabled and veterans are encouraged to apply”
Never miss an opportunity! Create an alert based on the job you applied for.
