Data Engineer AI/ML Focus

The Data Engineer with AI/ML expertise is responsible for designing, building, and optimizing scalable data pipelines, cloudnative data platforms, and AIdriven data processing systems. This role combines advanced data engineering skills with practical machine learning knowledge to support enterprise analytics, automation, and AI model deployment.

The engineer will collaborate with data scientists, software engineers, and business stakeholders to deliver highperformance data solutions that power predictive analytics, generative AI, and realtime decision systems.

Key Responsibilities

Data Engineering & Architecture

• Design, build, and maintain scalable ETL/ELT pipelines using modern data engineering frameworks.

• Develop and optimize data lakes, data warehouses, and lakehouse architectures on cloud platforms (AWS, Azure, Google Cloud Platform).

• Implement data ingestion, transformation, and orchestration using tools such as Spark, Databricks, Airflow, Kafka, or Snowflake.

• Ensure data quality, lineage, governance, and metadata management.

AI/ML Engineering

• Build data pipelines that support machine learning training, inference, and model monitoring.

• Collaborate with data scientists to operationalize ML models using MLflow, SageMaker, Vertex AI, Azure ML, or similar platforms.

• Implement feature engineering, feature stores, and realtime data feeds for AI workloads.

• Support deployment of LLMs, NLP models, and generative AI solutions into production environments.

Cloud & Platform Development

• Develop cloudnative data solutions using AWS (Glue, EMR, Lambda, Redshift), Azure (ADF, Synapse, Databricks), or Google Cloud Platform (Dataflow, BigQuery).

• Build APIs, microservices, and integration layers to support AIdriven applications.

• Implement CI/CD pipelines for data and ML workflows.

Data Security & Compliance

• Ensure compliance with data privacy, governance, and security standards.

• Implement rolebased access, encryption, and secure datasharing mechanisms.

• Support audit, logging, and monitoring for data and AI systems.

CrossFunctional Collaboration

• Work with product owners, architects, and business teams to translate requirements into technical solutions.

• Provide technical leadership on data engineering and AI best practices.

• Document architecture, data flows, and operational procedures.

Required Qualifications

• Bachelor s degree in Computer Science, Data Engineering, Information Systems, or related field.

• 5 10+ years of handson experience in data engineering.

• Strong expertise in Python, SQL, Spark, and distributed data processing.

• Experience with cloud data platforms (AWS, Azure, or Google Cloud Platform).

• Practical experience supporting AI/ML pipelines, model deployment, and MLOps.

• Strong understanding of data modeling, data warehousing, and realtime streaming.

• Experience with orchestration tools (Airflow, Prefect, Dagster).

Preferred Qualifications

• Experience with LLMs, vector databases (Pinecone, FAISS, Chroma), and embeddings.

• Handson experience with Databricks, Snowflake, or similar platforms.

• Knowledge of Docker, Kubernetes, and containerized ML workloads.

• Experience with feature stores and ML observability tools.