Data Engineer - Plano TX (On-site)

Overview

On Site
Depends on Experience
Accepts corp to corp applications
Contract - W2
Contract - 18 Month(s)

Skills

Streaming
Pyhton
Spark
Databricks
SQL
API

Job Details

Title- Data Engineer

Location- Plano TX (On-site)

Duration: 18 months

Key Responsibilities:

  • Design, develop, and maintain scalable data pipelines using PySpark, Spark, and Structured Streaming.
  • Collaborate with data scientists, analysts, and product teams to understand data requirements and deliver reliable data solutions.
  • Build and manage data workflows using Databricks.
  • Develop robust APIs using Python to support data integrations and operational workflows (API development specific to certain roles).
  • Write and optimize complex SQL queries for large-scale data processing and analytics.
  • Ensure data quality, data governance, and observability throughout the pipeline lifecycle.
  • Contribute to architectural decisions and best practices for data engineering solutions.
  • Continuously monitor and enhance data pipeline performance and scalability.

Required Skills & Qualifications:

  • Proficient in Python with hands-on experience in data engineering and backend scripting.
  • Strong experience in Apache Spark and PySpark for large-scale data processing.
  • Expertise in Structured Streaming for real-time data pipeline development.
  • Solid knowledge of SQL and working with large datasets.
  • Experience working in a Databricks environment.
  • Experience in API development using Python (applies to specific positions).
  • Familiarity with modern data architecture principles and cloud-based data platforms (e.g., Azure, AWS, or Google Cloud Platform) is a plus.
  • Strong problem-solving skills and ability to work independently and collaboratively.
  • Bachelor s or Master s degree in Computer Science, Engineering, or a related field.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.