Overview
On Site
Depends on Experience
Contract - W2
Contract - Independent
Contract - 12 Month(s)
Skills
Bigdata
SQL
Python
Job Details
Job Title: Big Data engineer
Work Location : Houston, TX 77077 (Onsite Hybrid)
Contract duration: 12+ months Contract
Job Description
Must Have Skills:
Databricks
SQL
Python
Nice to Have Skills:
Leadership
Accountability
Detailed Job Description:
Develop and maintain data pipelines using Apache Spark and Scala to process large volumes of data efficiently.
Implement data transformation logic to cleanse, normalize, and aggregate data from various sources, ensuring high data quality and consistency.
Integrate data from different systems and sources, including structured and unstructured data, databases, and Mainframe systems, ensuring seamless data flow and compatibility.
Implement data validation and quality checks within data pipelines to ensure data accuracy, consistency, and integrity, and handle exception handling and error logging.
Develop and execute unit tests, integration tests, and end-to-end tests to ensure the reliability and correctness of Spark applications.
Debug and troubleshoot issues during development and production phases.
Minimum Years of Experience:
8-10 years
Top 3 responsibilities you would expect the Subcon to shoulder and execute:
Lead and Design
Development
Lead for reporting track
Interview Process (Is face to face required?) Yes
Databricks
SQL
Python
Nice to Have Skills:
Leadership
Accountability
Detailed Job Description:
Develop and maintain data pipelines using Apache Spark and Scala to process large volumes of data efficiently.
Implement data transformation logic to cleanse, normalize, and aggregate data from various sources, ensuring high data quality and consistency.
Integrate data from different systems and sources, including structured and unstructured data, databases, and Mainframe systems, ensuring seamless data flow and compatibility.
Implement data validation and quality checks within data pipelines to ensure data accuracy, consistency, and integrity, and handle exception handling and error logging.
Develop and execute unit tests, integration tests, and end-to-end tests to ensure the reliability and correctness of Spark applications.
Debug and troubleshoot issues during development and production phases.
Minimum Years of Experience:
8-10 years
Top 3 responsibilities you would expect the Subcon to shoulder and execute:
Lead and Design
Development
Lead for reporting track
Interview Process (Is face to face required?) Yes
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.