Long Finch Technologies
Long Finch Technologies

Senior Data Engineer

Atlanta, GA, US • Posted 3 hours ago • Updated 32 minutes ago
Full Time
No Travel Required
On-site
Depends on Experience
Fitment

Dice Job Match Score™

🔢 Crunching numbers...

Job Details

Skills

  • Agile
  • Amazon S3
  • Amazon Web Services
  • Analytics
  • Apache Hadoop
  • Apache Hive
  • Apache Kafka
  • Business Intelligence
  • Data Quality
  • Data Governance
  • Data Engineering
  • Data Warehouse
  • Data Processing
  • Databricks
  • Extract
  • Transform
  • Load
  • Git
  • Google Cloud Platform
  • Machine Learning (ML)
  • Management
  • Microsoft Power BI
  • Mainframe
  • Kanban
  • IT Management
  • SAFE
  • SQL
  • Scala
  • Scrum
  • TIBCO Software
  • Tableau
  • Teradata

Summary

Job Description

We are looking for an experienced Senior Data Engineer with strong expertise in Databricks, Python, PySpark, Apache Spark, and large-scale data engineering. The ideal candidate will be responsible for designing and developing scalable ETL pipelines, implementing data processing solutions, managing data ingestion frameworks, and optimizing cloud-based data platforms. The role also involves technical leadership, data governance, and collaboration with analytics and business intelligence teams.

Required Technical Skills

  • Strong hands-on experience with Databricks, Delta Lake, Delta Live Tables, and Unity Catalog.

  • Proficiency in Python, PySpark, Spark SQL, and Apache Spark.

  • Strong experience in ETL development, data pipeline design, and data transformation.

  • Knowledge of Hadoop, Kafka, Hive, HDFS, and Sqoop.

  • Expertise in SQL, data modeling, and data warehousing.

  • Experience with batch and real-time data processing using Kafka and Spark Streaming.

  • Cloud experience with AWS and/or Google Cloud Platform.

  • Strong understanding of data quality, validation, cleansing, and governance.

  • Experience working in Agile environments (Scrum, Kanban, SAFe).

Preferred Skills

  • Scala / Scala-Spark, IBM DataStage, TIBCO.

  • AWS S3, Glue, Lambda, Kinesis/MSK.

  • Tableau, Power BI, Spotfire, OAC.

  • Teradata, DB2, BigQuery, Mainframe.

  • Git, data governance, and retention modeling.

Roles & Responsibilities

  • Design, develop, and maintain scalable data pipelines and ETL solutions.

  • Build data ingestion and transformation frameworks for structured and unstructured data.

  • Develop and optimize batch and streaming data processing solutions.

  • Implement data quality, validation, and governance standards.

  • Integrate heterogeneous data sources into enterprise data platforms.

  • Support analytics, reporting, BI, and machine learning initiatives.

  • Collaborate with Data Scientists, BI teams, and business stakeholders.

  • Lead technical discussions, mentor junior engineers, and drive data engineering best practices.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 90887977
  • Position Id: 9134382
  • Posted 3 hours ago
Create job alert
Never miss an opportunity! Create an alert based on the job you applied for.
More jobs at Long Finch Technologies in Atlanta, GA