Senior Data & Application Engineer (Local to TX Only)

Job Title

Senior Data & Application Engineer

Location & Work Model

Fort Worth, TX / Long Term Contract / Hybrid

About the Role

Team Structure & Leadership Impact

The engineering team is organized into distinct pods, with each pod managing different projects. In addition to technical execution, this role helps drive project planning and deliverable tracking across all pods while ensuring every pod strictly adheres to Agile methodologies.

Key Responsibilities

·       Design & Architecture: Participate and contribute to the end-to-end design and implementation of scalable data and application architectures using Python and Azure ML.

·       ETL/ELT Development: Build and implement complex ETL/ELT processes to transform and process large-scale datasets from diverse sources using Spark and Python.

·       Performance Optimization: Tune Python jobs for speed and cost-efficiency using partitioning, caching, cluster configuration tuning, and bottleneck troubleshooting.

·       Collaboration & Agile Leadership: Work alongside data scientists, analysts, and business stakeholders to translate complex business problems into technical solutions. Guide pods in Agile execution and deliverable management.

·       Data Governance & Security: Deploy data security strategies, compliance standards, and governance frameworks like Unity Catalog, including managing access controls, encryption, and data lineage.

·       DevOps & Automation: Champion high code quality and construct CI/CD pipelines using Azure DevOps, GitHub Actions, or Jenkins to automate the deployment and management of infrastructure and data assets.

·       Troubleshooting & Monitoring: Develop robust monitoring, logging, and alerting systems for production pipelines to proactively identify and resolve operational issues.

Required Qualifications & Experience

·       Overall Experience: 7 to 10 years of experience in the field, including 7+ years directly handling Python, ETL data processing, and SQL.

·       Core Technical Skills: Advanced proficiency in Python, Databricks, Azure Cloud, Spark, Azure ML, and strong application development skills.

·       Spark & Databricks: 5 years of dedicated programming experience in Spark and Databricks.

·       Monitoring & Reliability: 5 years of hands-on experience building logging, monitoring, and alerting systems.

·       Data Processing Languages: Advanced proficiency in Python, Scala, or SQL for data processing and application development.

·       Software Engineering: Deep experience across all phases of the Software Development Life Cycle (SDLC).

·       CI/CD Pipeline Design: Hands-on experience designing CI/CD pipelines that deploy code to development and production environments, preferably pushing to Azure using GitHub Actions.

·       AI Tooling: Experience using AI tools to boost daily work productivity.

Nice-to-Have Skills

·       Practical experience with Java and the Spring framework.

·       Broad knowledge across different Azure Cloud services.

·       Excellent verbal and written communication skills.

About VGAI Solutions

VGAI Solutions is an AI-native and technology solutions firm that helps organizations build high-performing engineering and technology teams. MBE-certified and ISO 9001-registered, we operate across the USA, Mexico, UAE, Mauritius, and India, with our Global Delivery Center in Hyderabad. From specialized IT to engineering talent to end-to-end AI solutions, we deliver the human capital that powers our Fortune 1000 clients.

Equal Opportunity Statement

VGAI Solutions is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. All qualified applicants will receive consideration without regard to race, color, religion, sex, national origin, disability, veteran status, or any other protected characteristic.