Highbrow Technology Inc
Highbrow Technology Inc

Data Engineer - PySpark

Hybrid in Durham, NC, US • Posted 5 hours ago • Updated 5 hours ago
Contract W2
12 Months
Occasional Travel Required
Hybrid
Depends on Experience
Fitment

Dice Job Match Score™

🔢 Crunching numbers...

Job Details

Skills

  • Agile
  • Amazon Redshift
  • Amazon Web Services
  • Analytical Skill
  • Apache Airflow
  • Apache Hive
  • Apache Kafka
  • Apache Spark
  • Big Data
  • Cloud Computing
  • Conflict Resolution
  • Continuous Delivery
  • Continuous Integration
  • Data Engineering
  • Data Governance
  • Data Lake
  • Data Modeling
  • Data Processing
  • Data Quality
  • Data Warehouse
  • Databricks
  • Distributed Computing
  • Docker
  • ELT
  • Extract, Transform, Load
  • Git
  • Good Clinical Practice
  • Google Cloud Platform
  • Kubernetes
  • Microsoft Azure
  • Orchestration
  • Problem Solving
  • PySpark
  • Python
  • Query Optimization
  • RTP
  • SQL
  • Snow Flake Schema
  • Streaming
  • Workflow

Summary

Job Title: Data Engineer – PySpark

Location: RTP Area, NC

Experience: 8+ Years

Work Arrangement: Hybrid – Onsite

Employment Type: W2 Only

Job Description

We are looking for an experienced Data Engineer with strong PySpark expertise to design, develop, and maintain scalable data processing solutions. The ideal candidate will have extensive experience in big data technologies, distributed computing, ETL/ELT development, and cloud-based data platforms.

The candidate should be comfortable working with large datasets, building efficient data pipelines, optimizing Spark workloads, and collaborating with cross-functional teams to deliver reliable data solutions.

Required Skills & Qualifications

  • 8+ years of experience in Data Engineering or related roles.
  • Strong hands-on experience with Python and PySpark.
  • Extensive experience with Apache Spark, Spark SQL, and distributed data processing.
  • Strong proficiency in SQL, including complex queries, joins, aggregations, and query optimization.
  • Experience developing and maintaining large-scale ETL/ELT data pipelines.
  • Strong understanding of data warehousing concepts, data modeling, and data lake architectures.
  • Experience working with cloud data platforms such as AWS, Azure, or Google Cloud Platform.
  • Experience with technologies such as Databricks, Delta Lake, Hive, or similar big data frameworks.
  • Knowledge of workflow orchestration tools such as Apache Airflow, Azure Data Factory, or AWS Glue.
  • Experience with Git, CI/CD pipelines, and Agile development methodologies.
  • Strong analytical, troubleshooting, and problem-solving skills.

Preferred Qualifications

  • Experience with streaming data processing using Kafka or Spark Streaming.
  • Exposure to Snowflake, Redshift, BigQuery, or Azure Synapse.
  • Experience implementing data governance and data quality frameworks.
  • Familiarity with Docker, Kubernetes, and cloud-native data engineering solutions.

Work Requirements

  • Location: Raleigh, NC.
  • Hybrid work arrangement with onsite presence required.
  • Must be willing to work on W2.
  • 8+ years of relevant professional experience required.

How to Apply

Interested candidates can share their updated resumes at:

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 91125331
  • Position Id: 9136303
  • Posted 5 hours ago
Contact the job poster
SRINIVASAN RAMALINGAM

SRINIVASAN RAMALINGAM

Talent Head @ Highbrow Technology Inc

View Profile
Create job alert
Never miss an opportunity! Create an alert based on the job you applied for.
More jobs at Highbrow Technology Inc in Durham, NC