Data Engineer with AI/ML (W2 role)

Role: Data Engineer with AI/ML

Exp: 10+ Years

Location: Remote

U-.S-C/G=C

MOI: 1-2 video round interviews

Skills Required:

• looking for a highly skilled Data Engineer with deep expertise in Databricks and a strong understanding of AI/ML workflows.
• This role is central to building and optimizing scalable data platforms that support advanced analytics and machine learning initiatives.
• You will work closely with data engineers, data scientists, ML engineers, and business stakeholders to enable intelligent data-driven solutions.
• Strong hands-on experience with Databricks, including Spark, Delta Lake, and MLflow.

• Proficiency in Python, SQL, and distributed data processing.
• Experience with cloud-native data services (e.g., AWS Glue, Azure Data Factory, Google Cloud Platform Dataflow).
• Familiarity with machine learning lifecycle and integration of models into data pipelines.
• Understanding of data warehousing, data lakehouse architecture, and real-time streaming (Kafka, Spark Structured Streaming).
• Experience with version control, CI/CD, and infrastructure-as-code tools.
• Excellent communication and collaboration skills.

Tech Stack - Core Tools:

• Databricks (Spark, Delta Lake, MLflow, Notebooks)
• Python & SQL
• Apache Spark (via Databricks)
• Delta Lake (for lakehouse architecture)
• Cloud Platforms
• Azure, AWS, or Google Cloud Platform
• Cloud Storage (ADLS, S3, GCS)
• Data Integration
• Kafka or Event Hubs (streaming)
• Auto Loader (Databricks file ingestion)
• REST APIs
• AI/ML
• MLflow (model tracking/deployment)
• Hugging Face Transformers
• LangChain / LlamaIndex (LLM integration)
• LLMs: Anthropic Claude, Meta LLaMA, Google Gemini
• DevOps
• Git (GitHub, GitLab, Azure Repos)
• Databricks Repos
• CI/CD: GitHub Actions, Azure DevOps
• Security & Governance
• Unity Catalog
• RBAC

Regards,

Sundeep B.

(732)-(790)-(5650)

sundeep.b (at) smg-LLC (dot) us