Spark Data Engineer

Remote • Posted 6 hours ago • Updated 3 hours ago

Contract W2

Contract Independent

No Travel Required

Remote

Depends on Experience

Resourcesoft, Inc.

Fitment

Dice Job Match Score™

🔢 Crunching numbers...

Job Details

Skills

Big Data
Azure
AWS
SQL
ETL
GCP
CI/CD
GitHub
Databricks
Apache Spark
PySpark
Data Modeling
Unity Catalog
ELT
Medallion Architecture
Delta Lake
Databricks Runtime Apache Spark (PySpark / Spark SQL) Delta Lake Auto Loader Structured Streaming
Spark SQL
Structured Streaming
Auto Loader
Jobs API

Summary

Requirements

5 or more years of experience in data engineering and large-scale software development.
Proficiency in Databricks Runtime and professional-level Databricks data engineering.
Experience with Apache Spark, PySpark, and Spark SQL for distributed data processing.
Experience in Delta Lake architecture, Structured Streaming, and medallion patterns.
Experience with Unity Catalog for enterprise data governance and workspace security.
Experience in CI/CD automation, GitHub, and cloud-based data lake environments.
Excellent verbal and written communication skills.

Responsibilities

Design and optimize scalable data pipelines using Spark and Databricks clusters.
Implement governed data access policies and audit logging via Unity Catalog.
Integrate complex data from REST APIs, relational databases, and cloud object stores.
Manage production workflows and automated jobs using the Databricks Jobs API.
Perform performance tuning and cost optimization for big data processing workloads.
Build curated, production-ready datasets for downstream analytics and data science teams.
Deploy Databricks infrastructure using automated CI/CD and infrastructure-as-code practices.

Founded in 1999, Resourcesoft, Inc. is a leading Technology Consulting and Professional Services organization. Headquartered in Marlborough, MA, the company serves the technology needs of its clients nationwide. Resourcesoft has often been recognized by prominent rating agencies for its exemplary growth and stability. With over two decades of industry experience, the Company has evolved as a front runner in enabling project optimization. We partner with leading organizations to provide technology solutions within the financial, insurance, education, government, publishing, healthcare and pharmaceutical domains. We take pride in mentoring a workforce that is well positioned to respond to the emerging IT trends and needs. With the employees at the crux of every business endeavor, our success is driven by our expertise in pairing the right talent with the best jobs in the technology sector. We forge long term, personalized relationships with our employees to advance their career to the next level. We engage them in technology centric client projects that provide opportunities for them to evolve, innovate and deliver world class products and services. Our career opportunities offer challenging assignments and exposure to emerging and cutting edge technologies. We are committed to providing our employees with the tools necessary to accelerate their career progression, while maintaining a healthy, work-life balance. We are deeply committed to providing a workplace ambience that is both challenging and fulfilling.

Resourcesoft is an Equal Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, pregnancy, sexual orientation, gender identity, national origin, age, protected veteran status, or disability status.

For more information about Resourcesoft and our services, please visit our website.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 10117734
Position Id: 20314-27990-
Posted 6 hours ago

Company Info

About Resourcesoft, Inc.

Founded in 1999, Resourcesoft, Inc. is a leading Technology Consulting and Professional Services organization. Headquartered in Marlborough, MA, the company serves the technology needs of its clients nationwide. Resourcesoft has often been recognized by prominent rating agencies for its exemplary growth and stability. With over two decades of industry experience, the company has evolved as a front runner in enabling project optimization. We partner with leading organizations to provide technology solutions within the financial, insurance, education, government, publishing, healthcare and pharmaceutical domains. We take pride in mentoring a workforce that is well positioned to respond to the emerging IT trends and needs. With the employees at the crux of every business endeavor, our success is driven by our expertise in pairing the right talent with the best jobs in the technology sector. We forge long term, personalized relationships with our employees to advance their career to the next level. We engage them in technology centric client projects that provide opportunities for them to evolve, innovate and deliver world class products and services. Our career opportunities offer challenging assignments and exposure to emerging and cutting edge technologies. We are committed to providing our employees with the tools necessary to accelerate their career progression, while maintaining a healthy, work-life balance. We are deeply committed to providing a workplace ambience that is both challenging and fulfilling.

Go to company profile

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.