Sr AI Engineer

Chicago, IL, US • Posted 2 hours ago • Updated 2 hours ago
Full Time
No Travel Required
On-site
Depends on Experience
Fitment

Dice Job Match Score™

🔢 Crunching numbers...

Job Details

Skills

  • Artificial Intelligence
  • Machine Learning (ML)
  • Databricks
  • PySpark
  • SQL
  • Python
  • PyTorch
  • TensorFlow
  • scikit-learn
  • LangChain
  • Amazon Web Services
  • Google Cloud Platform

Summary

Role: Sr AI Engineer                    

Location:Onsite. Chicago, IL 

Type: C2C

Send profiles to

Job Description: 

  We are seeking a Senior AI Engineer with deep, hands-on expertise in Databricks to design, build, and deploy production-grade AI and machine learning solutions at scale. You will lead the end-to-end lifecycle of data-intensive AI applications—from building resilient, large-scale data ingestion and transformation pipelines to training, fine-tuning, and operationalizing ML and Generative AI models in a unified lakehouse environment.

Key Responsibilities:

  • Large-Scale Data Engineering & Transformation: Architect and optimize high-throughput batch and streaming ETL/ELT pipelines using Apache Spark, PySpark, Delta Lake, and Databricks Lakehouse architectures to process multi-terabyte datasets.

  • AI & Machine Learning Model Development: Design, train, evaluate, and fine-tune predictive ML models, deep learning architectures, and Generative AI/LLM workflows (e.g., RAG pipelines, agentic systems) tailored to business-critical use cases.

  • MLOps & Production Deployment: Operationalize models at scale using MLflow, Databricks Model Serving, and CI/CD best practices, ensuring automated retraining, drift monitoring, and low-latency inference.

  • Data Governance & Feature Engineering: Build reusable feature pipelines using Databricks Feature Store and enforce data quality, lineage, and access controls via Unity Catalog.

  • Performance & Cost Optimization: Tune Spark jobs, cluster configurations, Photon engine workloads, and Delta table partitioning/Z-ordering to maximize compute efficiency and minimize cloud infrastructure costs.

  • Cross-Functional Technical Leadership: Partner with data scientists, software engineers, and product managers to translate complex business requirements into scalable technical architectures, while mentoring junior engineers on coding and MLOps standards.

Required Qualifications:

  • Experience: 5+ years of hands-on experience in AI/ML engineering, data science, or big data engineering, with at least 3+ years building production systems on Databricks.

  • Databricks Ecosystem Mastery: Proven expertise in PySpark, Spark SQL, Delta Lake, MLflow, Unity Catalog, and Databricks Workflows.

  • Programming & Software Engineering: Strong proficiency in Python and SQL, with solid software engineering fundamentals (Git, modular design, unit testing, CI/CD).

  • AI/ML Frameworks: Deep familiarity with modern ML and deep learning libraries (e.g., PyTorch, TensorFlow, Scikit-learn, Hugging Face, LangChain).

  • Cloud Infrastructure: Hands-on experience deploying data and AI workloads on major cloud platforms (AWS, Azure, or Google Cloud Platform).

Preferred Qualifications:

  • Experience building and deploying Generative AI / LLM applications using Databricks Mosaic AI, Vector Search, and Retrieval-Augmented Generation (RAG).

  • Familiarity with infrastructure-as-code tools (Terraform, Databricks Asset Bundles) for automated environment provisioning.

  • Experience with real-time streaming technologies (Apache Kafka, Spark Structured Streaming).

  • Bachelor’s or Master’s degree in Computer Science, Data Science, Engineering, or a related quantitative field.

 

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
  • Dice Id: 91074842
  • Position Id: 20261119
  • Posted 2 hours ago
Contact the job poster
MR

Maya Reddy

Recruiter @ ISOFT

Create job alert
Never miss an opportunity! Create an alert based on the job you applied for.