DataBricks Data Engineer

A client of Innova Solutions is immediately hiring a Databricks Data Engineer

Position type: Contract, Fulltime
Location: New York New York 10036
Duration: 6+ months
Schedule: Will be required to work onsite 4 days a week; 40 hours per week

As a Databricks Data Engineer, you will:
As a Databricks Data Engineer, you will be responsible for designing, developing, and implementing data engineering solutions using the Databricks platform to support data ingestion, transformation, performance reporting, analytics, sales incentive compensation processing, and enterprise data pipeline processes.

Responsibilities:

• Collaborate with business stakeholders to understand their data requirements and translate them into scalable technical solutions.
• Design and develop Databricks-based data pipelines, including data models, Delta Lake tables, and dashboards/notebooks.
• Responsible for the overall Databricks architecture to ensure new pipelines and enhancements to existing workflows are designed in an integrated and optimal manner.
• Maintain and optimize all current Databricks workspaces and pipelines; capture and deliver requirements for new or improved functionality to achieve further automation and integration of data and processes.
• Design and implement data pipelines to support Sales Incentive Compensation (SIC) processes, including quota management, attainment calculations, commission processing, and payout reporting.
• Partner with Sales Operations and Finance teams to understand incentive plan rules and translate them into accurate, auditable data transformations and models.
• Perform testing and debugging of data pipelines and notebooks to identify and resolve issues and ensure optimal performance.
• Provide technical support and troubleshooting for Databricks-based solutions, working closely with end-users and analysts to address their needs and resolve any data-related problems.
• Stay up to date with Databricks features and enhancements (Unity Catalog, Delta Live Tables, Photon, etc.), and proactively identify opportunities to leverage them to improve data quality, performance, and analytics capabilities.
• Collaborate with cross-functional teams, including finance, IT, and business intelligence teams, to ensure seamless integration and alignment of Databricks with other systems and processes.
• Document technical specifications, data models, pipeline configurations, and development processes to ensure knowledge transfer and maintain system documentation.

The ideal candidate will have:
Requirements:

• At least 7 years of experience in data engineering, with a minimum of 3 years hands-on experience building and managing Databricks solutions.
• Extensive experience with Databricks platform components: Delta Lake, Delta Live Tables, Unity Catalog and Databricks Workflows.
• Strong proficiency in Python and/or Scala for data engineering; SQL expertise required.
• Strong experience with core data engineering practices including data ingestion, transformation (ETL/ELT), data modeling, and pipeline orchestration.
• Functional and technical experience with Sales Incentive Compensation (SIC) including incentive plan design, quota allocation, attainment tracking, commission calculations, and payout processing.
• Familiarity with SIC platforms (e.g., Anaplan, Oracle ICM, Varicent) and experience integrating them with data engineering platforms is a strong plus.
• Ability to understand complex incentive compensation rules and model them accurately in data pipelines and transformation logic.
• Experience integrating Databricks with cloud platforms (Azure, AWS, or Google Cloud Platform) and data sources such as Azure Data Factory, Event Hubs, Kafka, or equivalent.
• Passion and ability to create holistic, end-to-end, integrated data solutions.
• Stakeholder management skills; ability to challenge constructively and translate complex business requirements into scalable data architectures - not just gather requirements.
• Strong analytical and problem-solving skills, with the ability to understand complex data requirements and implement reliable, efficient solutions.
• Excellent communication and collaboration skills, with the ability to work effectively with cross-functional teams and stakeholders.
• Must work well in a team environment while being a resourceful, independent self-starter able to work effectively with minimal direction.
• Skilled at documenting complex data architectures and pipelines in a professional manner.
• Ability to work independently and manage multiple tasks and priorities in a fast-paced environment.
• Passion for data quality, performance optimization, and delivering excellent end-user data experiences.
• Databricks Certified Data Engineer Associate or Professional certification is a plus.

Qualified candidates should APPLY NOW for immediate consideration!
This position is only open to applicants who can be engaged on a W-2 basis.
Please hit APPLY to provide the required information, and we will be back in touch as soon as possible.
We are currently interviewing to fill this and other similar positions. If this role is not a fit for you, we do offer a referral bonus program for referrals that we successfully place with our clients, subject to program guidelines. ASK ME HOW.
Thank you!

Mohd Saqib



PAY RANGE AND BENEFITS:
Pay Range*: $70- $75 per hour
*Pay range offered to a successful candidate will be based on several factors, including the candidate's education, work experience, work location, specific job duties, certifications, etc.

Benefits: Innova Solutions offers benefits (based on eligibility) that include the following: Medical & pharmacy coverage, Dental/vision insurance, 401(k), Health saving account (HSA) and Flexible spending account (FSA), Life Insurance, Pet Insurance, Short term and Long term Disability, Accident & Critical illness coverage, Pre-paid legal & ID theft protection, Sick time, and other types of paid leaves (as required by law), Employee Assistance Program (EAP).

ABOUT INNOVA SOLUTIONS:Founded in 1998 and headquartered in Atlanta, Georgia, Innova Solutions employs approximately 50,000 professionals worldwide and reports an annual revenue approaching $3 Billion. Through our global delivery centers across North America, Asia, and Europe, we deliver strategic technology and business transformation solutions to our clients, enabling them to operate as leaders within their fields.

Recent Recognitions:

• Named One of America's Best Employers for New Grads by Forbes (2024
• Named One of the Best Companies to Work for by U.S. News & World Report (Private Companies List, 2024-2025)
• One of the Largest IT Staffing Firms in the US - Ranked #3 by Staffing Industry Analysts (SIA, 2024)
• One of the Largest Staffing Firms in the US - Ranked #13 by Staffing Industry Analysts (SIA, 2024; includes Innova Solutions, Volt, & HireGenics)
• Named One of the Top Ten Private Companies in Atlanta by the Atlanta Business Chronicle (2023)
• One of the Largest Certified Minority Business Enterprises (MBEs) in the United States (NMSDC, 2024)
• AWS Advanced Tier Services Partner with 100+ certifications

Website:

Innova Solutions offers a referral bonus for qualified Ambassadors who refer eligible referrals. Submit referrals at and earn $250-$1,000 per referral.
Innova Solutions is an Equal Opportunity Employer and prohibits any kind of unlawful discrimination and harassment. Innova Solutions is committed to the principle of equal employment opportunity for all employees and to providing employees with a work environment free of discrimination and harassment on the basis of race, color, religion or belief, national origin, citizenship, social or ethnic origin, sex, age, physical or mental disability, veteran status, marital status, domestic partner status, sexual orientation, or any other status protected by the statutes, rules, and regulations in the locations where it operates. If you are an individual with a disability and need a reasonable accommodation to assist with your job search or application for employment, please contact us at or . Please indicate the specifics of the assistance needed. Innova Solutions encourages all interested and qualified candidates to apply for employment opportunities. Innova Solutions (HireGenics/Volt) does not discriminate against applicants based on citizenship status, immigration status, or national origin, in accordance with 8 U.S.C. 1324b.
The company will consider for employment qualified applicants with arrest and conviction records in a manner that complies with the San Francisco Fair Chance Ordinance, the Los Angeles Fair Chance Initiative for Hiring Ordinance, and other applicable laws.
By applying for this job, you agree to receive calls, AI-generated calls, text messages, or emails from Innova Solutions and its affiliates, and contracted partners. Frequency varies for text messages. Message and data rates may apply. Carriers are not liable for delayed or undelivered messages. You can reply STOP to cancel and HELP for help. You can access our privacy policy at .
Pandologic. Keywords: Data Engineer, Location: New York, NY - 10060