COOLSOFT
COOLSOFT

Data Engineer Lead / Data Warehouse Architect

Hybrid in Lansing, MI, US • Posted 7 hours ago • Updated 7 hours ago
Contract Independent
Contract Corp To Corp
Contract W2
11 Months
25% Travel Required
Hybrid
$65 - $70/hr
Fitment

Dice Job Match Score™

🔢 Crunching numbers...

Job Details

Skills

  • Cloud
  • Data Lake
  • Lakehouse
  • Python
  • PySpark
  • SQL
  • Medallion Architecture
  • Data Products
  • Data Governance

Summary

Job Summary
 
The role will lead the design and delivery of enterprise Data Lake, Lakehouse, Data/ML products and distributed data architectures using Agile/DevOps practices. The position requires strong hands-on experience with cloud data platforms, data modeling, data pipelines, Medallion Architecture and ML/data engineering technologies.
 
Required Experience:
  • 4+ years hands-on experience building enterprise Data Lakes on AWS, Azure or Google Cloud.
  • Experience ingesting relational, semi-structured and unstructured data.
  • 4+ years transforming Data Lake datasets into data products.
  • Experience maintaining traceability from data-product attributes/features back to critical data elements.
  • 2+ years enforcing data-sharing policies and access controls for data products.
  • 4+ years designing and implementing data, analytics and AI solutions for large enterprises.
  • Strong Python / PySpark / SQL experience.
  • Experience extracting data from relational databases, flat files, CSV, HTML and JSON sources.
  • Experience implementing Lakehouse Medallion Architecture.
  • Experience working with Parquet data formats.
  • Experience designing Star Schema-based operational data stores.
  • Power BI or Tableau experience.
Key Technologies:
  • Databricks
  • SageMaker
  • Apache Spark / PySpark
  • Apache Kafka
  • Kubernetes
  • SQL / NoSQL
  • Graph Databases
  • Python / R
  • Terraform
  • Docker
  • YAML
  • DAGs
  • GitHub
  • Jenkins
  • dbt
  • AWS / Azure / Google Cloud
  • Parquet
  • Power BI / Tableau
  • AWS Glue
  • Redshift Spectrum
  • DataHub
  • Machine Learning
Key Responsibilities:
  • Design, develop and maintain enterprise data pipelines.
  • Architect and optimize data infrastructure.
  • Design storage solutions and orchestration frameworks.
  • Develop ETL/ELT processes.
  • Transform and prepare data for analytics and operational systems.
  • Establish data quality, reliability and security standards.
  • Implement validation and governance practices.
  • Lead and mentor data engineering teams.
  • Review technical designs and code.
  • Collaborate with data scientists, analysts, product teams and business stakeholders.
  • Support the State of Michigan technology environment.
  • Work with the University of Michigan proof-of-concept team to promote data engineering best practices.
  • Provide technical training and guidance to data engineers.
Preferred Skills:
  • GitHub
  • Jenkins
  • Terraform
  • YAML
  • Docker
  • DAGs
  • Databricks
  • SageMaker
  • Data governance/security tools
  • Data-sharing policy enforcement
  • Data engineering standards and best practices
  • Lakehouse/Medallion Architecture
  • Data/ML pipeline development
Important:The ideal candidate should be a hands-on senior Data Engineer/Architect, not just a data analyst. Strong experience with Cloud + Data Lake/Lakehouse + Python/PySpark/SQL + Medallion Architecture + Data Products + Data Governance is critical.Please send me your updated resume
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: nm13509
  • Position Id: 9141532
  • Posted 7 hours ago
Contact the job poster
NH

Nishant Hs

Recruiter @ COOLSOFT

View Profile
Create job alert
Never miss an opportunity! Create an alert based on the job you applied for.
More jobs at COOLSOFT in Lansing, MI