Data Architect

Title- Data Architect (15+ Years Exp)
Location- Atlanta, GA/ Dallas, TX/ Seattle, WA - Onsite
Type- Contract- C2C/W2
Duration- 18+ Months

Job Description-

Role Description
Lead the design and implementation of data processing solutions using PySpark, Azure Databricks and Python.
Creates the high-level software design and dictates technical standards including software coding standards, tools, and platforms.
Responsible to build or define correct processes and tools to establish non functional governance of Data Solution ( including Monitoring , performance , cost optimization , SLA compliance, security, scalability , Logging ).
Act as a Strong partner for our internal/external stakeholders to provide effective solution for architectural requirements and non functional challenges.
Responsible for designing holistic technical architecture solutions with appropriate HW/SW specifications (i.e. size) in alignment with customer requirements, solution specifications and cost constraints.
Makes cross-component design decisions. Defines the technical design, examines new technologies, and defines technical components.
In charge of the architectural roadmap. Provides architecture solutions during the software development lifecycle.

Key responsibilities
Analyzing and designing effective and clear technical solutions for infrastructure and enterprise applications, providing high-level and/or detailed designs per need, responsible for solution updates according to project evolution and changes
Able to analyze performance and architecture issues in the data processing framework components ( eg Application , Infra structure , network , compute )
Interacting with a variety of internal and external stakeholders: presale leads, customer IT & business managers, Amdocs customer-facing managers, project managers R&D managers, procurement, IT , 3rd party Software support teams
Performance tuning of Cloudera/Horton work big data platforms.
Effectively engage third party software support teams for production non functional issues due to 3rd Party.
Performing capacity (including trending) and implementation design planning for all Data Center technical components (e.g. servers, storage, network)
Responsible to work as a point of contact for architecture governance with internal/external stakeholders.
Responsible to establish build, design and operate model for HA , DR , Security , scalability for all data pipelines and solutions.
Constantly reviewing new technologies in Data /CI/CD Domain and bring new practices to project.
Works with software engineers and other architects to define and refine the product structure to align with the business, development, deployment, and maintenance needs. Works with customers and product line management to identify, refine, and translate customer needs into concrete technical requirements. Understands and plans for architecture evolution including integration of new software technologies. Takes accountability for product and application architecture. Supports and leads architectural decisions and tasks within a product line or across multiple product lines (cross-portfolio).
Works with the software engineering teams to establish best practice guidelines and reusable and production-ready solutions. Reviews technical designs, and provides guidance to software engineers on technical and architectural design decisions. Is an ambassador for the architecture, ensuring that it is being implemented correctly. Gives feedback and inputs to the product management team to consider and improve the product line. Leadership is focused on the technical aspects of the job, rather than the people management aspects.
Standards and Best Practices - Researches, evaluates, and prototypes new methodologies, technologies, and products. Provides leadership in the use of processes and tools. Proposes and follows through with improvements in processes and tools. Ensures effective application of corporate standards and procedures. Proposes improvements, and tracks and provides updates about pertinent technology trends.
Customer Engagement - Obtains a good understanding of the customer context while making technical decisions and choices. Interacts effectively with customers as required. Provides the development or support team with inputs and requirements related to the technical aspects of the solution/product.

Critical Experiences
Strong experience in designing and delivering Azure based data platform solutions, technologies including Databricks, Snowflake, ADF, ADLS, AKS etc.
Strong understanding of data architecture principles, including data modelling, ETL processes, data governance, data quality, privacy, and security controls.
Strong foundation in data engineering principles, ETL/ELT processes, data modeling, and data warehousing techniques.
Good knowledge in real time streaming applications preferably with experience in Kafka Real time messaging, Spark processing and performance tuning.
Expertise in Structured Streaming, Delta Live Tables, Databricks Unity Catalog, and Databricks Serverless Computing, Databricks, machine learning pipelines.
Designing and implementing DataOps & Data Mesh aligned frameworks to optimize data platforms for cost, value, flexibility, and efficiency.
File formats partitioning for e.g. Delta,Iceberg, Parquet,avro,JSON,CSV.
Experience with production performance/capacity analysis.

Technical Skills
Data Management Services
Data Strategy, Data Governance, Data Architecture, Data Integration, Data Quality, Data Cataloging, Data Security, Data Operations
Big Data Analytics, Self-Service BI, Data Visualization
Data Science (AI/ML, Platform, Self Service)
BI Platform Migrations
DevOps, Automation
Big Data Technologies (6+ experience; Cloud experience is a must - Azure/Google Cloud Platform/AWS)
Databricks,PostGres,Snowflake,ADLS
PySpark, Structured Streaming, Kafka, Spark SQL, Hive, Python, Java, Scala
Performance Tuning of Databricks & Snowflake platforms
Kubernetes
Jenkins, Docker
Automation of cluster setup/configuration, code generation