Overview
Remote
Depends on Experience
Full Time
Skills
Bigdata
Spark
Job Details
As a Data Engineer, you will develop a new data engineering platform that leverage a new cloud architecture, and will extend or migrate our existing data pipelines to this architecture as needed. You will also be assisting with integrating the SQL data warehouse platform as our primary processing platform to create the curated enterprise data model for the company to leverage. You will be part of a team building the next generation data platform and to drive the adoption of new technologies and new practices in existing implementations. You will be responsible for designing and implementing the complex ETL pipelines in cloud data platform and other solutions to support the rapidly growing and dynamic business demand for data, and use it to deliver the data as service which will have an immediate influence on day-to-day decision making.
"
General Duties/Responsibilities (May Include But Are Not Limited To)
"
General Duties/Responsibilities (May Include But Are Not Limited To)
- Interfacing with business customers, gathering requirements and developing new datasets in data platform
- Building and migrating the complex ETL pipelines from on premise system to cloud and Hadoop/Spark to make the system grow elastically
- Identifying the data quality issues to address them immediately to provide great user experience
- Extracting and combining data from various heterogeneous data sources
- Designing, implementing and supporting a platform that can provide ad-hoc access to large datasets
- Modelling data and metadata to support machine learning and AI
Supervisory Responsibilities
N/A
Minimum Requirements
To perform this job successfully, an individual must be able to perform each essential duty satisfactorily.The requirements listed below are representative of the knowledge, skill, and/or ability required.Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.
Minimum Experience
3+ years relevant experience in cloud based data engineering.
Demonstrated ability in data modeling, ETL development, and data warehousing.
Data Warehousing Experience with SQL Server, Oracle, Redshift, Teradata, etc.
Experience with Big Data Technologies (NoSQL databases, Hadoop, Hive, Hbase, Pig, Spark, Elasticsearch etc.)
Experience in using Python, .net, Java and/or other data engineering languages
Education/Licensure
Bachelors or Masters in Computer Science, Engineering, Mathematics, Statistics, or related field
Other
Knowledge and experience of SQL Sever and SSIS.
Excellent communication, analytical and collaborative problem-solving skills
Preferred
Healthcare domain and data experience
Healthcare EDI experience is a plus
API development experience is a plus
Industry experience as a Data Engineer or related specialty (e.g., Software Engineer, Business Intelligence Engineer, Data Scientist) with a track record of manipulating, processing, and extracting value from large datasets.
Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets
Experience building data products incrementally and integrating and managing datasets from multiple sources
Experience leading large-scale data warehousing and analytics projects, including using Azure or AWS technologies SQL Server, Redshift, S3, EC2, Data-pipeline, Data Lake, Data Factory and other big data technologies
Experience providing technical leadership and mentor other engineers for the best practices on the data engineering space
Linux/UNIX including to process large data sets.
Experience with Azure, AWS or Google Cloud Platform is a plus
Microsoft Azure Certification is a plus
Demonstrable track record dealing well with ambiguity, prioritizing needs, and delivering results in an agile, dynamic startup environment
Problem solving skills and Ability to meet deadlines are a must
Microsoft Azure Certification is a plus
N/A
Minimum Requirements
To perform this job successfully, an individual must be able to perform each essential duty satisfactorily.The requirements listed below are representative of the knowledge, skill, and/or ability required.Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.
Minimum Experience
3+ years relevant experience in cloud based data engineering.
Demonstrated ability in data modeling, ETL development, and data warehousing.
Data Warehousing Experience with SQL Server, Oracle, Redshift, Teradata, etc.
Experience with Big Data Technologies (NoSQL databases, Hadoop, Hive, Hbase, Pig, Spark, Elasticsearch etc.)
Experience in using Python, .net, Java and/or other data engineering languages
Education/Licensure
Bachelors or Masters in Computer Science, Engineering, Mathematics, Statistics, or related field
Other
Knowledge and experience of SQL Sever and SSIS.
Excellent communication, analytical and collaborative problem-solving skills
Preferred
Healthcare domain and data experience
Healthcare EDI experience is a plus
API development experience is a plus
Industry experience as a Data Engineer or related specialty (e.g., Software Engineer, Business Intelligence Engineer, Data Scientist) with a track record of manipulating, processing, and extracting value from large datasets.
Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets
Experience building data products incrementally and integrating and managing datasets from multiple sources
Experience leading large-scale data warehousing and analytics projects, including using Azure or AWS technologies SQL Server, Redshift, S3, EC2, Data-pipeline, Data Lake, Data Factory and other big data technologies
Experience providing technical leadership and mentor other engineers for the best practices on the data engineering space
Linux/UNIX including to process large data sets.
Experience with Azure, AWS or Google Cloud Platform is a plus
Microsoft Azure Certification is a plus
Demonstrable track record dealing well with ambiguity, prioritizing needs, and delivering results in an agile, dynamic startup environment
Problem solving skills and Ability to meet deadlines are a must
Microsoft Azure Certification is a plus
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.