Title: Data Engineer (Databricks)
Duration: 12 month contract
Location: Atlanta, GA - hybrid onsite and would prefer local to Atlanta
Other: banking or financial services experience
Data Engineering:
Design and maintain scalable data pipelines using Spark, Python, and Databricks for real-time and batch processing.
Build and manage ELT workflows using Fivetran and integrate data into Snowflake for analytics and reporting.
Develop and optimize data models and warehouse structures to support business intelligence and machine learning use cases.
Collaborate with cross-functional teams to understand data needs and deliver reliable, governed, and high-quality data solutions.
Implement cloud-native data solutions and ensure performance, security, and compliance across platforms.
Architect and deploy data solutions on cloud platforms (e.g., Azure, AWS, or Google Cloud Platform) using Databricks and Snowflake.
Implement CI/CD pipelines for data workflows and infrastructure as code (IaC) using tools like Terraform or Azure DevOps.
Additional notes from today s call with the hiring manager Primary Platform: The new data layer is entirely built on Databricks; candidates must have strong, hands-on Databricks experience and be comfortable working heads-down in that environment. Core Responsibilities:
Build scalable data pipelines to ingest data from legacy systems into Databricks.
Support Bronze Silver Gold data transformation architecture.
Ensure data is available for reporting in Power BI and/or ThoughtSpot.
Preferred Skills:
Proficient in Python/PySpark for data engineering tasks.
Experience implementing CI/CD pipelines (no dedicated DevOps support, but doesn't need to build from scratch).
Familiarity with Databricks-native data quality checks.
Presentation Layer:
Snowflake is used for reporting by some clients knowledge of Snowflake is helpful but deep expertise isn t required.
Additional Context:
Collaboration and current team includes 3 front-end and 3 back-end developers.
Future direction includes LLM integration for reasoning models and multi-tenant, on-prem connectivity (e.g., SQL Server, Azure SQL).
Some components will connect to Foundry from the Silver layer.
Never miss an opportunity! Create an alert based on the job you applied for.
