Job Title: Lead Data Engineer
Location: Atlanta, GA(Onsite)
Contract Duration: Long term Contract
Job Description:
Senior hands-on Data Engineer responsible for designing, building and operating scalable batch/streaming data pipelines and data products across Azure and Google Cloud Platform.
Own end-to-end engineering covering ingestion, transformation, data modeling, orchestration, testing, deployment and production support.
Build reusable engineering frameworks and support analytics/ML workloads while providing technical leadership and mentoring.
Relevant Experience: 10+ years
Mandatory skills:
- SQL, Python, PySpark;
- strong ETL/ELT, data modeling and distributed processing;
- hands-on Azure + Google Cloud Platform experience;
- Azure Data Factory/Fabric/Databricks/Data Lake and Google Cloud Platform BigQuery/Dataflow/Dataproc/Cloud Storage/Pub/Sub;
- GitHub, CI/CD, automated testing, IaC;
- DevOps/MLOps;
- production support and performance tuning.
Desired skills:
- Applied data science/ML understanding including feature engineering, statistics and model lifecycle;
- streaming pipelines;
- observability and cloud-cost optimization;
- experience with consumer, media, customer, commerce or marketing data;
- data governance, lineage, privacy and access controls;
- mentoring/code reviews.
Domain (Industry):
Consumer Packaged Goods (CPG) / Beverage Alcohol/ Retail / Consumer Goods