AWS Data Architect

Title: AWS Data Architect

Location: Remote

Duration: 6 Months

Job Summary:

We are seeking a highly skilled and experienced AWS Data Architect who will be responsible for developing and managing data pipelines on AWS. The ideal candidate will bring strong expertise in AWS data services, big data processing, and data modeling to help us provide high-performance data solutions.

Responsibilities:

Data Pipeline Development and Management:

• Design, build, and maintain scalable and reliable data pipelines using AWS Glue, Kinesis, and Lambda.
• Develop ETL/ELT processes to process and transform large volumes of structured and unstructured data.
• Optimize data pipeline performance, scalability, and reliability.
• Ensure data processing and ingestion workflows are monitored and meet performance SLAs.

Data Storage and Management:

• Design and implement data storage solutions using Amazon S3, Redshift, DynamoDB, and RDS.
• Optimize data structures, partitioning, and indexing for performance and cost efficiency.
• Ensure data security, integrity, and availability across different AWS storage solutions.
• Manage data lifecycle policies and archiving processes.

Data Transformation and Processing:

• Develop data transformation processes using AWS Glue, EMR, and Athena.
• Implement data quality checks, validation rules, and monitoring solutions.
• Support real-time and batch data processing needs.

Data Integration and Automation:

• Integrate data from multiple sources, including APIs, databases, and third-party applications.
• Automate data ingestion, transformation, and export using tools like Lambda and Step Functions.
• Ensure data consistency across different environments and systems.

Collaboration and Stakeholder Engagement:

• Work closely with data scientists and analysts to understand data needs and business goals.
• Provide technical guidance and best practices to the data engineering and business teams.
• Collaborate with security and compliance teams to ensure data governance standards are met.

Performance Monitoring and Troubleshooting:

• Monitor data pipeline performance and troubleshoot issues in real-time.
• Analyze data pipeline failures and implement fixes to prevent recurrence.
• Set up logging and monitoring using CloudWatch and X-Ray.

Qualifications:

• Bachelor’s degree in Computer Science, Data Engineering, or a related field; Master’s degree is a plus.
• 10 years of experience as a data Architect.
• Strong proficiency with AWS services such as S3, Redshift, Glue, Athena, Kinesis, Lambda, and Step Functions.
• Hands-on experience with big data tools and frameworks such as Apache Spark, Hadoop, and Flink.
• Proficiency in programming languages such as Python, Java, or Scala.
• Strong knowledge of SQL, data modeling, and query optimization.
• Experience with CI/CD tools and version control (e.g., Git, CodePipeline).
• Strong understanding of data governance, security, and compliance requirements.
• Ability to manage large-scale data processing and real-time data pipelines.
• Excellent problem-solving, analytical, and communication skills.
• Experience with data lake architecture and implementation on AWS.
• Familiarity with Infrastructure as Code (IaC) tools such as CloudFormation and Terraform.
• Experience with NoSQL databases (e.g., DynamoDB) and key-value stores.
• Knowledge of containerization and orchestration using ECS and EKS.
• AWS Certified Data Analytics or AWS Certified Big Data certification is a plus.