Databricks Data Engineer

Job Title: Databricks Data Engineer
Location: Washington, DC (Remote)
Type: Contract
Compensation: $61.67/HR on W2


Responsibilities

• Develop and support new and existing data pipelines within Azure cloud-based data lake environments.
• Translate business requirements into scalable data engineering solutions supporting an enterprise-scale Microsoft Azure data platform.
• Maintain ETL operations and develop new pipelines to ensure data quality and management.
• Design, build, and optimize scalable data solutions using Databricks and Medallion Architecture.
• Manage ingestion routines for processing multi-terabyte datasets across multiple projects and Databricks workspaces.
• Integrate data from various structured and unstructured sources to enable high-quality business insights.
• Implement data management strategies to ensure data integrity, availability, and accessibility.
• Identify opportunities for cost optimization in storage, processing, and analytics operations.
• Monitor platform performance, address requests, and support cluster stability and Spark optimization.
• Collaborate with teams to enable advanced AI-driven analytics and data science workflows.
• Integrate with Azure services including Functions, Storage, Data Factory, Log Analytics, and User Management.
• Provision and manage infrastructure using Infrastructure-as-Code (IaC).
• Apply best practices for data security, governance, and compliance supporting federal regulations.
• Gather requirements from technical and non-technical teams to translate business needs into effective data solutions.

• BS degree in Computer Science or related field; 3+ years of experience or Master’s degree with 2+ years of experience.
• 3+ years developing and designing ingestion flows (structured, streaming, unstructured) using cloud platform services with focus on data quality.
• Databricks Data Engineer certification.
• 2+ years maintaining Databricks platform and developing in Spark.
• Ability to work directly with clients and provide front-line support.
• Proficient in Python, Spark, and R; .NET experience is a plus.
• Knowledge of data governance, metadata management, enterprise data catalogs, data quality, and security.
• Experience with Agile, CI/CD automation, and cloud development (Azure, AWS).
• U.S. citizen.
• On-site availability in the Washington, DC area or willingness to relocate for periodic on-site activities.  

System One, and its subsidiaries including Joulé and Mountain Ltd., are leaders in delivering outsourced services and workforce solutions across North America. We help clients get work done more efficiently and economically, without compromising quality. System One not only serves as a valued partner for our clients, but we offer eligible employees health and welfare benefits coverage options including medical, dental, vision, spending accounts, life insurance, voluntary plans, as well as participation in a 401(k) plan.

System One is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, age, national origin, disability, family care or medical leave status, genetic information, veteran status, marital status, or any other characteristic protected by applicable federal, state, or local law.

#M-M2

#LI-RF1

Ref: #851-Rockville-S1