Location: - Minneapolis, MN (3-4 days hybrid)
contract
Note : Candidate should have hands-on coding experience in Apache Spark, strong proficiency in Databricks, and proven expertise in designing, developing, and maintaining scalable ETL pipelines.”
Job Summary:
We are looking for a skilled Databricks Engineer with strong hands-on experience in Apache Spark, Azure Databricks, PySpark, and SQL. The ideal candidate will have proven expertise in designing, developing, and maintaining scalable ETL pipelines and working with large-scale data processing systems.
Key Responsibilities:
• Design, develop, and optimize ETL pipelines using Azure Databricks, PySpark, and SQL
• Perform hands-on coding in Apache Spark for large-scale data processing and transformation
• Implement end-to-end data engineering solutions on Azure platform
• Work with structured and unstructured data across multiple sources
• Optimize performance and troubleshoot issues in Spark jobs and Databricks workflows
• Collaborate with cross-functional teams including data analysts, data scientists, and business stakeholders
• Ensure data quality, integrity, and governance across pipelines
• Develop reusable and scalable data frameworks and best practices
Required Skills:
• Extensive hands-on coding experience in Apache Spark (PySpark)
• Strong expertise in Azure Databricks implementation
• Proficiency in SQL and data transformation techniques
• Experience in building and maintaining scalable ETL pipelines
• Good understanding of data warehousing concepts and data modeling
• Experience working with Azure Data Services (e.g., ADLS, ADF is a plus)
• Strong problem-solving and analytical skills
Preferred Skills:
• Experience with CI/CD pipelines and DevOps practices
• Knowledge of Delta Lake and data lake architecture
• Familiarity with performance tuning and optimization in Spark
• Experience working in Agile environments
Never miss an opportunity! Create an alert based on the job you applied for.
