Sr. Databricks Developer

Role : Sr. Databricks Developer

Location: Columbus, OH

Duration: 12 Months

MOI : Telephonic & WebCam

Primary Skills: SQL, Python, AWS & PYSPARK

Need Candidates with a Minimum of 9 Years of Experience. This is a 100% Onsite role, and we need candidates on our W2 or who have their Own Corporation.

Job Description:

We are seeking an experienced Databricks Developer to design, develop, and optimize large-scale data processing solutions for a financial services project. The ideal candidate will have strong expertise in Databricks, Apache Spark, cloud-based data platforms, and financial data processing. This role involves building scalable data pipelines, supporting analytics initiatives, ensuring data quality, and enabling regulatory and business reporting across financial systems.

Key Responsibilities:

• Design, develop, and maintain scalable ETL/ELT data pipelines using Databricks and Apache Spark.
• Develop and optimize data ingestion frameworks for structured, semi-structured, and streaming data.
• Build and maintain data lakes and lakehouse architectures using Databricks.
• Implement Delta Lake solutions for data reliability, governance, and performance optimization.
• Collaborate with business analysts, data architects, and finance stakeholders to understand data requirements.
• Integrate data from trading platforms, market data feeds, risk systems, accounting platforms, and regulatory reporting systems.
• Develop reusable notebooks, workflows, and automation processes.
• Monitor, troubleshoot, and optimize Spark jobs for performance and scalability.
• Implement data quality, lineage, security, and governance controls.
• Support regulatory reporting, risk management, compliance, and financial analytics initiatives.
• Participate in code reviews, testing, deployment, and production support activities.

Required Technical Skills:

• Strong experience with Databricks and Apache Spark (PySpark/Scala Spark).
• Proficiency in Python, SQL, and Spark optimisation techniques.
• Experience with Delta Lake, Databricks Workflows, Unity Catalogue, and Delta Live Tables.
• Strong understanding of data warehousing concepts and dimensional modelling.
• Experience with cloud platforms such as Amazon Web Services (AWS)