Data Bricks Data Engineer

Hiring!!!

Greetings from Modern Agile Technologies.

Position:  Data Bricks Data Engineer 
Location: Seattle, WA / Bellevue, WA / Everett, WA / Renton, WA / Richardson, TX / Plano, TX / Dallas, TX / St. Louis, MO / Charleston, SC / Arlington, VA/Onsite
Duration: Full Time

Experience required: 8 – 15 Years

Job Description

Must Have Technical/Functional Skills

•       Awareness of IBM Data Stage ETL/ELT data integration tool to understand existing code.
Develop , Test , Deploy ,Optimize, and monitor large-scale data processing workloads in Azure  Data Bricks ETL.

•       Ensure and lead the efforts to review Legacy Data Stage legacy code and migrated Data bricks code to ensure functionality is not deviated

•       Strong programming skills in Python and PySpark.

•       Advanced proficiency writing SQL for analytics and ETL processes.

•       Proven experience building and optimizing complex data pipelines in Azure.

•       Hands-on experience with Azure Databricks: cluster management, job scheduling, workspace governance.

•       Strong working knowledge of core Azure services: Storage Account, Synapse, Key Vault, VMSS,

•       Function Apps, Web Apps, Log Analytics Workspace, service principals, and managed identities.

•       Experience with container services (ACA, container instances) and containerized data workloads.

•       Familiarity with Azure networking concepts and secure network integration for data platforms.

•       Experience creating Azure infrastructure using ARM templates.

•       Proficient with GitLab and Azure DevOps for CI/CD and source control workflows.

•       Strong analytical, problem-solving, and communication skills; proven ability to work cross-functionally.

•       Experience working in Agile teams and understanding of data governance frameworks.

•       Hands-on experience provisioning Databricks resources with Terraform; ability to author and maintain Terraform templates and modules.

•       Demonstrated experience implementing cluster autoscaling and autoscaling policies through Terraform.

•       Experience creating reusable Terraform modules and implementing infrastructure-as-code best practices (module structure, state management, remote backends).

•       Proven experience working on Databricks platform operations, including cluster configuration, job orchestration, and platform optimization.

•       Experience configuring high-availability Databricks deployments and operating across multiple availability zones/regions.

•       Familiarity with Metastore/Unity Catalog configuration and metadata governance in Databricks.

•       Hands-on experience building data pipelines and ingestion workflows into medallion-layer architectures (bronze/silver/gold).

•       Strong scripting skills (Python, Bash, or similar) and familiarity with CI/CD for Terraform and Databricks deployments.

•       Strong troubleshooting, performance tuning, and cost optimization skills.