Data Engineer - ONLY W2

Introduction:

This position is for a Data Engineer at AT&T in Plano, TX. The primary focus of this role is to work on a migration project of data systems and applications. The ideal candidate will be responsible for designing, building, and maintaining data pipelines using modern cloud-native technologies such as Databricks, Snowflake, and Azure.

Responsibilities:

• Design and implement end-to-end data pipelines using Palantir Foundry, Databricks, and Snowflake
• Develop scalable ETL/ELT workflows using Python, PySpark, and SQL for processing large-scale datasets
• Build and maintain data lake and data warehouse architectures on cloud platforms (Azure, Databricks, Snowflake)
• Implement data governance, quality checks, and metadata management frameworks
• Create and optimize Databricks notebooks, Delta Lake tables, and Unity Catalog implementations
• Collaborate with data scientists, analysts, and business stakeholders to deliver data products
• Optimize query performance and storage strategies in Snowflake
• Implement real-time streaming pipelines using Spark Streaming or similar technologies
• Develop CI/CD pipelines for data workflows using Git, Jenkins, or similar tools

Requirements:

Required Qualifications:

• 6+ years of hands-on data engineering experience
• Proficiency in Python and PySpark for data processing
• Good SQL skills with experience in complex query optimization
• Hands-on experience with Databricks, Snowflake
• Experience with cloud platforms (Azure) and their data services
• Knowledge of data modeling (dimensional modeling, star schema, snowflake schema)
• Experience with orchestration tools (Airflow, dbt, Luigi, or similar)
• Good understanding of data governance, security, and compliance requirements

Desired Qualifications:

• Experience with IBM WatsonX.data or similar AI data platforms
• Knowledge of Delta Lake, Iceberg, or similar lakehouse formats
• Experience with GenAI, RAG architectures, or LangChain
• Familiarity with MCP (Model Context Protocol) for AI agent integrations