Our client is seeking a senior-level, hands-on Data Engineer with deep experience designing and operating production-grade data platforms on Microsoft Azure, with a strong focus on Azure Databricks.
This role goes beyond pipeline development. The successful candidate will help own platform design, performance, reliability, and cost efficiency, while partnering closely with research teams, brokers, and technical stakeholders to deliver trusted, scalable data solutions.
Key Responsibilities:
- Serve as a senior technical contributor responsible for designing and maintaining scalable, production-ready data pipelines using Azure Databricks.
- Partner with business and research teams to translate complex data requirements into robust technical designs aligned with enterprise architecture standards.
- Lead design decisions around data ingestion, transformation, modeling, and orchestration for large and complex datasets.
- Optimize performance, reliability, and cost through effective Spark tuning, query optimization, and pipeline design.
- Build and maintain batch and near real-time data processing workflows.
- Establish and follow best practices for data quality, monitoring, and operational support.
- Support analytics and data science teams by delivering well-modeled, reusable data assets.
- Ensure data security, governance, and compliance with organizational standards.
- Collaborate with platform, infrastructure, and security teams to support and evolve the broader data ecosystem.
- Mentor junior and mid-level engineers and raise overall engineering standards.
Qualifications:
- Bachelor's degree or higher in Computer Science, Engineering, Mathematics, or a related field.
- 8+ years of experience in data engineering or related roles, with significant hands-on experience in Azure Databricks.
- Strong expertise with Azure Databricks, Azure Data Factory, Azure Data Lake Storage, and related Azure services.
- Advanced proficiency in Python and SQL, experience with Spark internals strongly preferred.
- Demonstrated experience designing, deploying, and supporting production-grade data pipelines at scale.
- Solid understanding of distributed data processing, data modeling, and performance optimization.
- Experience with CI/CD, version control, and structured deployment practices is a strong plus.
- Excellent problem-solving skills and ability to operate independently in complex environments.
- Strong communication skills and comfort working with both technical and business stakeholders.
- Experience in real estate, financial services, or similarly data-intensive domains is a plus.
Salary/Rate: $65-$70/Hour (depends on experience level). This is a contract position with candidates expected to work 40 hours/ week.
About Us
Peterson Technology Partners (PTP) is an Equal Opportunity Employer committed to creating a transparent, inclusive, and human-centered hiring experience.
For more than 27 years, PTP has operated as a premier Information Technology (IT) staffing, consulting, and recruiting firm built on trust, long-term partnerships, and technical excellence.
Based in the Chicago suburb of Park Ridge, IL, our team of more than 500 employees and consultants is dedicated to helping every client make the best hiring decisions possible.
As part of that commitment, we believe in providing clear information about how our hiring technologies work and how your data is used. The following section outlines our AI-assisted interview process and your rights as a candidate.
AI-Assisted Interview Experience (Pete & Gabi Rebecca):
To provide a consistent, fair, and flexible experience for all candidates, we use AI-assisted tools including our proprietary platform Pete & Gabi Rebecca to support parts of the interview process. These tools help us:
- transcribe interviews,
- summarize candidate responses,
- generate job-related insights
- streamline communication and scheduling.
Please note that
- AI does NOT make hiring decisions.
- All decisions are made by our human recruiters, hiring managers, or client partners.
- The AI does not evaluate facial expressions, emotions, or physical traits.
- It is used only to support fairness, consistency, and efficiency.
If you prefer a non-AI interview format, we will gladly provide an alternative.
Technical or Case Interviews (Role-Dependent):
For certain positions, you may participate in:
- a technical interview
- a coding challenge
- a case study
- a client-specific assessment.
We will always explain what to expect in advance so you can prepare with confidence.
Human Review & Selection:
Every candidate s profile including interviews, conversations, and assessments is reviewed by experienced recruiters and hiring leaders.
AI insights may assist with organization and evaluation, but final decisions are always human-driven.
Your Rights as a Candidate
At PTP, every candidate has the right to:
- Request a non-AI interview path
- Ask how your data is being used
- Request access to transcripts or interview recordings
- Request deletion of your AI-recorded interview
- Receive clear, timely communication
Our goal is to ensure you feel respected, informed, and supported throughout your experience
Our Commitment:
For more than 27 years, PTP has focused on putting people first candidates, consultants, employees, and clients.
We re committed to a hiring process that is:
- transparent
- compliant
- equitable
- powered by innovative technology that enhances not replaces human judgment.
Welcome to the future of hiring at Peterson Technology Partners.
We re excited to learn more about you
Equal Employment Opportunity
Peterson Technology Partners is an Equal Opportunity Employer. All qualified applicants will receive consideration without regard to race, color, religion, national origin, gender identity, sexual orientation, disability, veteran status, or any other protected characteristic.
Never miss an opportunity! Create an alert based on the job you applied for.
