Data Engineer With Data Lake Services

Role: Data Engineer with Date lake Services

Location: Remote

Duration: Long Term

Requirements:

• Data engineers will be responsible for designing and implementing data ingestion workflows to ingest data from a variety of sources, including government operations, business data, workforce data, and health data.
• Domain expertise in one or more of these areas is preferred.
• They will build robust, scalable, and efficient ELT (Extract, Load, Transform) patterns using AWS services such as S3, Glue, Athena, and Redshift.
• The data engineers will also be responsible for designing the data models and managing the schemas for the bronze and silver layers of the medallion architecture, as well as implementing data quality checks, data lineage tracking, and metadata management to ensure data governance and compliance.
• Additionally, they will automate data pipeline workflows using tools like AWS Glue, AWS Step Functions, or Apache Airflow, and collaborate closely with technical staff, data analysts, data scientists, and business stakeholders.
• The selected data engineers should have a minimum of 3-5 years of experience in building and maintaining large-scale, enterprise-grade data lake solutions on the AWS platform and will be required to maintain comprehensive documentation of the data lake architecture, data models, and data pipeline workflows
• Engineers will also support a few existing Azure Data factory workloads.
• The Contractor understands this is a shared environment and agrees to comply with all applicable platform functional and nonfunctional requirements, standards, and procedures as defined by the State. Contractor will provide constructive feedback for the ongoing development and enhancement of the platform.

Roles and Responsibilities.

State Personnel (Director of Data Services) will:

• Provide an introductory process
• Assign work
• Set the schedule
• Provide high-level oversight
• Provide performance feedback
• Establish a deliverable review process
• Approve deliverables submitted for work performed
• Establish invoice process
• Approve invoices submitted for work performed
• Provide necessary resources
• Resolve issues