Data Engineer - PySpark, Azure Data Factory

Houston, TX, US • Posted 2 hours ago • Updated 2 hours ago

Full Time

On-site

$140,000 - $160,000/yr

Fitment

Dice Job Match Score™

📋 Comparing job requirements...

Job Details

Skills

PySpark
Azure
Data Factory

Summary

Data Engineer

Houston, TX

Full time

As an Data Engineer, you will contribute to the BI development to optimize energy operations, enhance forecasting, and improve operational efficiency. This role is ideal for professionals with 10+ years of experience who are passionate comprehensive data and BI solutions development and solving real-world energy sector challenges using PySpark and Azure Data Factory.

Job Responsibilities

Engage with business users to gather requirements and translate them into comprehensive data and BI solutions.
Design and implement scalable data pipelines using PySparkand Azure Data Factory, integrating with Azure Data Lake, Synapse Analytics, DataBricks, and Microsoft Fabric.
Develop and optimize data models and visualizations using Power BI, ensuring performance and usability across business units.
Lead modernization efforts by migrating legacy BI systems to cloud-based platforms.
Collaborate with cross-functional teams to ensure alignment with architectural standards and business objectives.
Oversee system testing, user acceptance testing (UAT), and production deployments.
Maintain and enhance existing BI solutions, ensuring reliability and scalability.
Document data flows, transformation logic, and architectural decisions.
Communicate effectively with stakeholders regarding upcoming changes and coordinate change management activities.

Requirements

Job Requirements

Minimum 10 years of experience in data engineering and BI development.
Proven expertise in PySpark for data transformation and pipeline development.
Extensive experience with Azure Data Services, including Data Factory, Data Lake, Synapse,Microsoft Fabricand DataBricks.
Strong proficiency in Power BI dashboard development and data modeling.
Solid understanding of cloud architecture, data governance, and performance optimization.
Advanced SQL skills with a focus on query performance and scalability.
Experience with dimensional modeling (Star and Snowflake schemas).
Excellent communication and collaboration skills in cross-functional environments.
Ability to manage multiple priorities in a dynamic and fast-paced setting.
Familiarity with SSIS, SSRS, and SSAS(tabular and multi-dimensional models).
Experience with DAX and MDX query languages.
Exposure to Power Apps, Data Flows
Knowledge of machine learning (ML)and artificial intelligence (AI) concepts and their application in data analytics.
Have worked with Azure ML Services, and understand OpenAI LLM Model.
Experience implementing role-based and row-level security in BI platforms.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.

Dice Id: 10383289
Position Id: 8873850
Posted 2 hours ago

Create job alert

Never miss an opportunity! Create an alert based on the job you applied for.

Data Engineer - PySpark, Azure Data Factory

Dice Job Match Score™

Job Details

Skills

Summary

Similar Jobs