RESPONSIBILITIES:
Kforce has a client that is seeking a Senior Data Engineer - PySpark/ETL in Greenwood Village, CO.
Summary:
We're looking for a Senior Data Engineer to join a highly technical Data Engineering team supporting Data Science initiatives. This role will focus on building and maintaining large-scale data pipelines, implementing ETL processes, and making sure data is accurate, reliable, and ready for analytics and machine learning.
The ideal candidate is comfortable working with Python/PySpark, Spark DataFrames, SQL, and large datasets and enjoys digging into the data-not simply moving it from one place to another. You'll work closely with Data Scientists and other engineers on projects involving network telemetry and other complex datasets.
What You'll Do:
* Build and maintain scalable ETL pipelines using PySpark and Spark
* Develop Spark jobs from scratch and support existing production workflows
* Work primarily with batch processing, including daily and hourly jobs
* Use Spark DataFrames to transform, join, aggregate, and prepare large datasets
* Perform data validation and quality-control checks to ensure accurate downstream analytics
* Partner with Data Scientists to understand data requirements and deliver reliable datasets for modeling and analytics
* Troubleshoot and improve existing data pipelines and processing jobs
* Help ensure data is processed in the right format and according to business and analytical requirements
* Work with AWS-based data processing environments, including EMR and Athena
* Work within an orchestration environment such as Airflow or another workflow orchestration tool
* Collaborate with engineering, Data Science, and platform teams to deliver new and updated data solutions
REQUIREMENTS:
* Strong experience with Python for data engineering/analytics
* Hands-on experience with PySpark, specifically Spark DataFrames
* Experience building batch Spark jobs from scratch
* Experience with data validation and quality checks
* Strong SQL skills, including joins, aggregations, groupings, and window functions
* Experience with a workflow/orchestration tool such as Airflow, Step Functions, or a similar technology
* Strong understanding of ETL and data transformation
* Working knowledge of AWS and cloud-based data processing
Nice to Have:
* Experience with Scala/Spark
* Experience working directly with Data Science teams
* Experience with Databricks
* Experience working with telemetry, network, or other highly technical datasets
* Experience with AWS EMR and Athena
* Familiarity with MLflow or machine-learning model workflows
The pay range is the lowest to highest compensation we reasonably in good faith believe we would pay at posting for this role. We may ultimately pay more or less than this range. Employee pay is based on factors like relevant education, qualifications, certifications, experience, skills, seniority, location, performance, union contract and business needs. This range may be modified in the future.
We offer comprehensive benefits including medical/dental/vision insurance, HSA, FSA, 401(k), and life, disability & ADD insurance to eligible employees. Salaried personnel receive paid time off. Hourly employees are not eligible for paid time off unless required by law. Hourly employees on a Service Contract Act project are eligible for paid sick leave.
Note: Pay is not considered compensation until it is earned, vested and determinable. The amount and availability of any compensation remains in Kforce's sole discretion unless and until paid and may be modified in its discretion consistent with the law.
This job is not eligible for bonuses, incentives or commissions.
Kforce is an Equal Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, pregnancy, sexual orientation, gender identity, national origin, age, protected veteran status, or disability status.
By clicking ?Apply Today? you agree to receive calls, AI-generated calls, text messages or emails from Kforce and its affiliates, and service providers. Note that if you choose to communicate with Kforce via text messaging the frequency may vary, and message and data rates may apply. Carriers are not liable for delayed or undelivered messages. You will always have the right to cease communicating via text by using key words such as STOP.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
- Dice Id: kforcecx
- Position Id: ITTVT2189416
- Posted 30+ days ago
