Position: Senior Cloud Data Architect
Location : Remote(USA)
Position Overview
We are seeking a highly experienced Senior Cloud & Data Architect with 10 + years of expertise in designing, building, and modernizing large-scale Big Data, Cloud, and Analytics platforms. The ideal candidate brings deep, hands-on architectural expertise across AWS, Azure, Google Cloud Platform, and Snowflake, combined with strong emerging capability in Generative AI and Agentic AI solutions. This role is central to modernizing enterprise data platforms, driving cloud migration strategy, and embedding intelligent automation into data engineering and DevOps workflows.
Key Responsibilities
Cloud & Data Platform Architecture
- Design end-to-end cloud and data architectures across AWS, Azure, and Google Cloud Platform, aligned with business and technical requirements.
- Architect Data Lakes, Lakehouse, Data Warehousing, Dimensional Modeling, and Data Vault 2.0 solutions.
- Define reference architectures, solution blueprints, and landing zones following cloud well-architected frameworks.
- Evaluate legacy/on-premises systems and design modernization and migration strategies (re-host, re-platform, re-architect).
- Ensure all solutions meet scalability, availability, performance, security, and resiliency requirements.
Generative AI / Agentic AI
- Design and prototype Retrieval-Augmented Generation (RAG) pipelines connecting enterprise data platforms to LLMs for natural-language querying and reporting.
- Build agentic automation workflows (LangChain/LangGraph or similar) for autonomous task execution, tool-calling, and multi-step reasoning in data and DevOps pipelines.
- Integrate LLM APIs (Azure OpenAI, AWS Bedrock, Databricks Mosaic AI/Genie) into existing platforms to enable natural-language analytics, automated documentation, and intelligent data quality checks.
- Implement vector databases and embeddings for semantic search and knowledge-retrieval use cases.
- Apply prompt-engineering and multi-agent orchestration patterns across ingestion, transformation, and validation workflows.
Big Data & Analytics Engineering
- Design and implement Hadoop ecosystem components (HDFS, MapReduce, Hive, Impala, Pig, Sqoop, Oozie, HBase) and Apache Spark/PySpark pipelines for large-scale, distributed data processing.
- Architect and manage Snowflake Data Cloud environments, including virtual warehouses, data sharing, cloning, time travel, credit optimization, and Snowpipe.
- Build and govern Databricks Lakehouse solutions, including Unity Catalog, Workflows, and Spark-based orchestration.
- Design event-driven, real-time streaming architectures using Kafka.
Cloud Platform Engineering
- Architect and secure AWS services (IAM, KMS, EMR, Lambda, API Gateway, EC2, VPC, S3, Redshift, RDS, Kinesis, CloudFormation).
- Design Azure IaaS/PaaS solutions (Virtual Networks, AKS, App Services, Azure SQL, Cosmos DB, Azure OpenAI Service) and Azure DevOps CI/CD pipelines.
- Implement Google Cloud Platform services (Compute Engine, BigQuery, Dataproc, Cloud Functions, Pub/Sub, Vertex AI).
- Migrate on-premises databases and platforms to cloud using tools such as AWS DMS.
DevOps, Automation & Governance
- Define and implement CI/CD pipelines (Jenkins, GitHub Actions, Azure DevOps) with automated build, test, security, and release workflows.
- Orchestrate and automate workflows using Apache Airflow, including custom operators, hooks, and sensors.
- Drive Infrastructure-as-Code practices using Terraform, Ansible, Chef, or Puppet.
- Establish and enforce data governance, lineage, metadata management, and data quality standards.
- Lead containerization and microservices migration efforts using Docker and Kubernetes.
Required Qualifications
- 10+ years of experience in Software Engineering, Data Engineering, and Cloud Architecture roles.
- Proven experience architecting solutions across AWS, Azure, and Google Cloud Platform.
- Strong hands-on expertise with Snowflake, Databricks, and Apache Spark/PySpark.
- Demonstrated experience with RAG pipelines, LLM integration, and agentic AI frameworks (LangChain, LangGraph).
- Deep knowledge of the Hadoop ecosystem and distributed data processing.
- Strong programming/scripting skills in Python, Bash, YAML, and SQL/HiveQL.
- Experience with RDBMS (Oracle, SQL Server, PostgreSQL, MySQL, Teradata) and NoSQL (MongoDB, DynamoDB, Cassandra).
- Strong background in CI/CD, DevOps, and Infrastructure-as-Code.
- Excellent communication skills and experience collaborating with cross-functional teams (product, data, DevOps, governance).
- Bachelor's or Master's degree in Computer Science, Computer Applications, or related field.
Preferred Qualifications / Certifications
- AWS Solutions Architect Certification.
- Azure DevOps Engineer Certification.
- Databricks Generative AI Fundamentals or equivalent GenAI certification.
- Cloudera Certified Administrator for Apache Hadoop (CCAH).
- MongoDB/MySQL Certified DBA.
- Experience with agile methodologies (Scrum/Kanban).
Never miss an opportunity! Create an alert based on the job you applied for.