Overview
On Site
Full Time
Skills
Writing
Internal Communications
Integrated Circuit
IC
HTML
SFTP
Data Integration
Collaboration
Documentation
Systems Architecture
Artificial Intelligence
Extract
Transform
Load
Management
Apache Spark
Databricks
Workflow
Python
SQL
Data Manipulation
Cloud Computing
Amazon Web Services
Microsoft Azure
Job Details
Job Description
Base-2 Solutions is seeking a Data Engineer/Integrator who will be collaborating closely with a team of developers to fulfill data integration requirements. This involves writing and maintaining code using an Extract-Transform-Load (ETL) platform to ensure data is transformed into suitable formats as defined by IC ITE initiatives. You will interface with external teams and systems, employing various protocols including HTML and SFTP to collect data efficiently. Additionally, your responsibilities include enhancing the ETL platform by adding features aimed at shortening timelines for future data integration efforts. Beyond coding tasks, you'll develop and maintain software, ensuring seamless integration into a fully functional system. Collaboration with external teams will be necessary to validate data ingest processes. Finally, you'll be responsible for providing comprehensive documentation covering system architecture, development, and any enhancements made throughout the process.
Required Skills
Qualifications
Desired Skills
Base-2 Solutions is seeking a Data Engineer/Integrator who will be collaborating closely with a team of developers to fulfill data integration requirements. This involves writing and maintaining code using an Extract-Transform-Load (ETL) platform to ensure data is transformed into suitable formats as defined by IC ITE initiatives. You will interface with external teams and systems, employing various protocols including HTML and SFTP to collect data efficiently. Additionally, your responsibilities include enhancing the ETL platform by adding features aimed at shortening timelines for future data integration efforts. Beyond coding tasks, you'll develop and maintain software, ensuring seamless integration into a fully functional system. Collaboration with external teams will be necessary to validate data ingest processes. Finally, you'll be responsible for providing comprehensive documentation covering system architecture, development, and any enhancements made throughout the process.
Required Skills
- Specialization using the Databricks platform for building, deploying, and managing data and AI solutions.
- Experience building ETL pipelines, managing data pipelines, and working with large datasets using tools like Spark, Python, and SQL.
- Experience with technologies like Delta Lake, Delta Live Tables, and Databricks Workflows.
Qualifications
- 5 years of professional experience with a Bachelor's degree or equivalent.
- 3 years of professional experience with a Master's degree or equivalent.
- 10 years of professional experience.
- Additional experience will be considered in lieu of a degree.
Desired Skills
- Experience collaborating with data scientists.
- Familiarity with Advana.
- Strong Python programming skills.
- Solid SQL knowledge for querying and data manipulation.
- Cloud platform experience, such as AWS, Azure, etc.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.