Overview
On Site
Depends on Experience
Accepts corp to corp applications
Contract - Independent
Contract - W2
Contract - 12 Month(s)
No Travel Required
Skills
Amazon Kinesis
Amazon S3
Amazon SQS
Amazon Web Services
Apache Spark
Data Modeling
SQL
Python
Job Details
Data Engineer Onsite Position ( Houston,TX). Long term
Qualifications
10+ years of experience in data engineering with an emphasis on data analytics and reporting.
6+ years of experience with the cloud platforms Amazon Web Services (AWS), Azure.
2+ years of experience in building data pipelines to support AI and ML models, strong understanding of LLM's, prompt engineering, vibe coding using the GitHub copilot or similar AI tools, helping team to build faster using the enterprise approved AI tools.
10+ years of experience in SQL, data transformations, ETL/ELT and experience on Database Platform Snowflake ,Fabric and S3.
10+ years of experience in the design and build of data extraction, transformation, and loading processes by writing custom data pipelines.
6+ years of experience with one or more of the following scripting languages: Python, SQL, Shell Scripting.
6+ years of experience designing and building real-time data pipelines utilizing various Cloud services such as S3, Kinesis, RDS, Snowflake, Lambda, Glue, API Gateway, SQS, SNS, CloudWatch, cloud formation, DBT, etc.
5+ years of experience in REST API data integrations in pulling into Snowflake, posting from Snowflake, understanding of webhooks, and implementing using AWS gateway.
Bachelor's degree, preferably in Computer Science, Information Technology, Computer Engineering, or related IT discipline, or equivalent experience.
4+ years of experience in data modeling to support descriptive analytics in PBI.
4+ years of experience in data cataloging, empowering business, and digital partners to utilize data cataloging and metadata for self-service.
Strong understanding of data management principles and best practices.
understanding of data modeling, including conceptual, logical, and physical data models.
Responsibilities
Translate business requirements into technical specifications; establish and define details, definitions, and requirements of applications, components, and enhancements.
Participate in project planning; identifying milestones, deliverables, and resource requirements; track activities and task execution.
Generate design, development, test plans, detailed functional specifications documents, user interface design, and process flow charts for execution of programming.
Develop data pipelines / APIs using Python, SQL, potentially Spark, and AWS, Azure, or Google Cloud Platform Methods.
Use an analytical, data-driven approach to drive a deep understanding of fast-changing business.
Build large-scale batch and real-time data pipelines with data processing frameworks in AWS, Snowflake, and DBT.
Move data from on-prem to cloud and cloud data conversions.
Implement large-scale data ecosystems including data management, governance, and the integration of structured and unstructured data to generate insights leveraging cloud-based platforms.
Leverage automation, cognitive, and science-based techniques to manage data, predict scenarios, and prescribe actions.
Drive operational efficiency by maintaining their data ecosystems, sourcing analytics expertise, and providing As-a-Service offerings for continuous insights and improvements.
Job Description
Translate business requirements into technical specifications; establish and define details, definitions, and requirements of applications, components, and enhancements.
Participate in project planning; identifying milestones, deliverables, and resource requirements; track activities and task execution.
Generate design, development, test plans, detailed functional specifications documents, user interface design, and process flow charts for execution of programming.
Develop data pipelines / APIs using Python, SQL, AWS, Azure Fabric.
Use an analytical, data-driven approach to drive a deep understanding of fast-changing business.
Qualifications
10+ years of experience in data engineering with an emphasis on data analytics and reporting.
6+ years of experience with the cloud platforms Amazon Web Services (AWS), Azure.
2+ years of experience in building data pipelines to support AI and ML models, strong understanding of LLM's, prompt engineering, vibe coding using the GitHub copilot or similar AI tools, helping team to build faster using the enterprise approved AI tools.
10+ years of experience in SQL, data transformations, ETL/ELT and experience on Database Platform Snowflake ,Fabric and S3.
10+ years of experience in the design and build of data extraction, transformation, and loading processes by writing custom data pipelines.
6+ years of experience with one or more of the following scripting languages: Python, SQL, Shell Scripting.
6+ years of experience designing and building real-time data pipelines utilizing various Cloud services such as S3, Kinesis, RDS, Snowflake, Lambda, Glue, API Gateway, SQS, SNS, CloudWatch, cloud formation, DBT, etc.
5+ years of experience in REST API data integrations in pulling into Snowflake, posting from Snowflake, understanding of webhooks, and implementing using AWS gateway.
Bachelor's degree, preferably in Computer Science, Information Technology, Computer Engineering, or related IT discipline, or equivalent experience.
4+ years of experience in data modeling to support descriptive analytics in PBI.
4+ years of experience in data cataloging, empowering business, and digital partners to utilize data cataloging and metadata for self-service.
Strong understanding of data management principles and best practices.
understanding of data modeling, including conceptual, logical, and physical data models.
Responsibilities
Translate business requirements into technical specifications; establish and define details, definitions, and requirements of applications, components, and enhancements.
Participate in project planning; identifying milestones, deliverables, and resource requirements; track activities and task execution.
Generate design, development, test plans, detailed functional specifications documents, user interface design, and process flow charts for execution of programming.
Develop data pipelines / APIs using Python, SQL, potentially Spark, and AWS, Azure, or Google Cloud Platform Methods.
Use an analytical, data-driven approach to drive a deep understanding of fast-changing business.
Build large-scale batch and real-time data pipelines with data processing frameworks in AWS, Snowflake, and DBT.
Move data from on-prem to cloud and cloud data conversions.
Implement large-scale data ecosystems including data management, governance, and the integration of structured and unstructured data to generate insights leveraging cloud-based platforms.
Leverage automation, cognitive, and science-based techniques to manage data, predict scenarios, and prescribe actions.
Drive operational efficiency by maintaining their data ecosystems, sourcing analytics expertise, and providing As-a-Service offerings for continuous insights and improvements.
Job Description
Translate business requirements into technical specifications; establish and define details, definitions, and requirements of applications, components, and enhancements.
Participate in project planning; identifying milestones, deliverables, and resource requirements; track activities and task execution.
Generate design, development, test plans, detailed functional specifications documents, user interface design, and process flow charts for execution of programming.
Develop data pipelines / APIs using Python, SQL, AWS, Azure Fabric.
Use an analytical, data-driven approach to drive a deep understanding of fast-changing business.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.