Databricks Engineer - Hybrid

Title:             Databricks Engineer - Hybrid

Mandatory skills:

big data technologies, cloud platforms, AWS, Azure,
Databricks platform, scalable analytics solutions,
Spark, Unstructured format, Unity Catalog, Genie, Immuta, lambda, dynamodb, aws data sync, aws sftp,
programming languages, Java, Python, Bash, Unix shell, Shell Scripting,
VBA, automation, scripting,
ETL processes, SQL query optimization, database design,
Data Warehouses, Oracle, Microsoft SQL Server databases,
big data solutions, data workflows, cutting-edge cloud platforms,
data pipelines, ETL workflows, data ingestion, data transformation

Description:

Overview:
We are seeking a dynamic and highly skilled Databricks Engineer to join our data team. In this role, you will be at the forefront of designing, developing, and maintaining scalable big data solutions using Databricks and related technologies. Your expertise will enable us to harness the power of large datasets, optimize data workflows, and deliver actionable insights that drive strategic decision-making. This position offers an exciting opportunity to work with cutting-edge cloud platforms and innovative data tools in a fast-paced, collaborative environment.

Responsibilities

Design, develop, and optimize data pipelines leveraging Databricks Unified Analytics Platform to process large-scale datasets efficiently.
Implement robust ETL workflows using Spark, Python, SQL, and shell scripting to ensure high-quality data ingestion and transformation.
Collaborate with cross-functional teams to integrate diverse data sources such as AWS, Azure Data Lake, Hadoop, Oracle, Microsoft SQL Server, and Informatica into unified data warehouses.
Develop and maintain scalable data models and database schemas aligned with best practices in database design for analytical applications.
Utilize analytics tools like Looker for creating dashboards and visualizations that facilitate insightful business analysis.
Conduct model training and analysis to support machine learning initiatives, ensuring data readiness and quality.
Write RESTful APIs for seamless data access across platforms and applications while adhering to security standards.
Support agile development processes by participating in sprint planning, code reviews, and continuous integration efforts.
Maintain comprehensive documentation of data workflows, system configurations, and technical specifications to ensure operational transparency.

Experience:

8+ Years of experience
Proven experience working with big data technologies and cloud platforms like AWS or Azure.
Hands-on experience with Databricks platform for developing scalable analytics solutions in a cloud environment.
Experience working with Spark, Unstructured format, Unity Catalog, Genie, Immuta, lambda, dynamodb, aws data sync, aws sftp
Strong proficiency in programming languages including Java, Python, Bash (Unix shell), Shell Scripting, and VBA for automation and scripting tasks.
Extensive knowledge of ETL processes, SQL query optimization, database design principles for Data Warehouses, and familiarity with Oracle and Microsoft SQL Server databases.

Notes:

Hybrid 3 Days/Week on-site

VIVA USA is an equal opportunity employer and is committed to maintaining a professional working environment that is free from discrimination and unlawful harassment. The Management, contractors, and staff of VIVA USA shall respect others without regard to race, sex, religion, age, color, creed, national or ethnic origin, physical, mental or sensory disability, marital status, sexual orientation, or status as a Vietnam-era, recently separated veteran, Active war time or campaign badge veteran, Armed forces service medal veteran, or disabled veteran. Please contact us at for any complaints, comments and suggestions.

Contact Details :

Account co-ordinator: Renish Kuttampoil

VIVA USA INC.
3601 Algonquin Road, Suite 425
Rolling Meadows, IL 60008
|