Overview
Hybrid
$60 - $65
Contract - W2
Contract - 12 Month(s)
Skills
Apache Spark
Data Flow
Data Quality
Data Validation
Database
Debugging
Mainframe
Scala
Unstructured Data
Job Details
Job Title: Data Engineer (Databricks / Python / SQL)
Location Houston, TX 77077 - Hybrid.
Duration: 12+ Months
Must Have Skills:
- Databricks
- SQL
- Python
Nice to Have Skills:
- Leadership
- Accountability
Detailed Job Description:
- Develop and maintain data pipelines using Apache Spark and Scala to process large volumes of data efficiently.
- Implement data transformation logic to cleanse, normalize, and aggregate data from various sources, ensuring high data quality and consistency.
- Integrate data from different systems and sources, including structured and unstructured data, databases, and Mainframe systems, ensuring seamless data flow and compatibility.
- Implement data validation and quality checks within data pipelines to ensure data accuracy, consistency, and integrity, and handle exception handling and error logging.
- Develop and execute unit tests, integration tests, and end-to-end tests to ensure the reliability and correctness of Spark applications.
- Debug and troubleshoot issues during development and production phases.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.