Software Guidance & Assistance, Inc., (SGA), is searching for an
Lead Data Engineer for a
CONTRACT/RIGHT TO HIRE assignment with one of our premier
Banking clients in
Irving, TX .
Responsibilities :
As a key member of our global development team, you will:
Innovate & Develop: - Partner closely with project managers, business stakeholders, and senior managers to translate complex business requirements into well-architected technical solutions. Consult with users and other technology groups, providing advanced programming insights and support.
- Drive cross-functional collaboration with diverse management teams to ensure seamless integration of functions, aligning efforts to achieve strategic organizational goals.
- Proactively identify, define, and implement necessary system enhancements to facilitate the successful deployment of new products and process improvements.
Complex Problem Resolution: - Lead the resolution of high-impact problems and critical projects through in-depth evaluation of intricate business processes, complex system architectures, and relevant industry standards.
- Employ advanced analytical and interpretive thinking to define issues, uncover root causes, and develop innovative, sustainable solutions.
- Consult with users, clients, and other technology groups on issues, and recommend programming solutions. Analyze complex technical and business challenges, and propose innovative solutions that enhance system functionality and business processes.
Technical Architecture & Standards Leadership: - Serve as a subject matter expert in application programming, ensuring that all application designs rigorously adhere to the overall architectural blueprint and strategic technology roadmap.
- Leverage an advanced understanding of system flow to develop and enforce robust standards for coding, testing, debugging, and implementation across development teams.
Mentorship & Talent Development: - Act as a trusted advisor and coach for mid-level developers and analysts, providing guidance, fostering skill development, and judiciously allocating work to maximize team potential and project success.
- Provide technical guidance, mentorship, and code reviews to junior data engineers, fostering a culture of excellence and continuous improvement.
- Operational Excellence: Ensure adherence to best practices and essential procedures.
- Autonomy & Ownership: Operate with a high degree of independence and judgment, taking ownership of critical initiatives and driving them to successful completion.
- Risk Management: Proactively assess and manage technical risks, demonstrating a strong commitment to regulatory compliance, ethical judgment, and transparent reporting of control issues.
- Design, develop, and maintain robust, scalable, and high-performance data pipelines using PySpark.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver high-quality data solutions.
- Optimize and tune Spark jobs for performance and efficiency.
- Implement data quality checks and ensure data integrity across all data pipelines.
- Data Architecture & Design: Design, develop, and optimize data architectures, pipelines, and data models to support various business needs, including analytics, reporting, and machine learning.
- ETL/ELT Development (Python/PySpark Focus): Build, test, and deploy highly scalable and efficient ETL/ELT processes using Python and PySpark to ingest, transform, and load data from diverse sources into data warehouses and data lakes. Develop and optimize complex data transformations using PySpark.
- Data Quality & Governance: Implement best practices for data quality, data governance, and data security to ensure the integrity, reliability, and privacy of our data assets.
- Performance Optimization: Monitor, troubleshoot, and optimize data pipeline performance, ensuring data availability and timely delivery, particularly for PySpark jobs.
Required Skills:
Experience: 6-10 years of progressive experience in systems analysis and programming of software applications, with a proven track record of implementing successful projects.
- Strong proficiency in Java application technologies, including deep experience with TDD (Test-Driven Development), Spring framework, and Microservices architecture.
- Extensive hands-on experience with PySpark and advanced Python programming skills.
- Proven experience with Big Data ecosystems, including Cloudera and/or Data Bricks.
- Hands-on experience with distributed query engines like Starburst (Trino/Presto).
- Proficient in designing and managing complex workflows using scheduling tools, particularly Apache Airflow.
- Strong expertise in SQL and experience with relational and non-relational databases
- Excellent knowledge of algorithms and data structures, design patterns. Experience in systems analysis and programming of software applications
- Strong Java experience : Java core, collections, concurrency, streams
- Frameworks and APIs: Spring (Core, Batch, Integration, MVC, Boot, Data), Hibernate, Jackson , JAX RS, JPA, JAXB
- Experience with distributed caches like Apache Gem fire will be a plus
- Messaging: JMS, Kafka
- Experience in Angular 21+ / ReactJS
- Testing: JUnit, Mocking frameworks (Mockito, Power Mock)
- Experience in performance enhancements using parallel processing, multithreading. Understanding locking/synchronization
- Understanding Docker and Kubernetes
- Experience in RESTful API development and integration, deployment framework and source control experience such as Git.
- Solid understanding and experience with SQL.
- Proficiency in Linux environments.
- Experience with job scheduling.
- Methodology: Working knowledge of project management techniques and methods, with a focus on agile methodologies.
- Adaptability: Ability to thrive in a fast-paced environment, manage multiple deadlines, and adapt quickly to evolving requirements and priorities.
- Collaboration: A strong team player with excellent communication skills, capable of working effectively with global teams to deliver integrated solution
- Experience with real-time data streaming and processing using PySpark Structured Streaming.
- Knowledge of machine learning concepts and MLOps practices, especially integrating ML workflows with PySpark.
- Familiarity with data visualization tools (e.g., Tableau, Power BI).
- Contributions to open-source data projects.
- Strong experience with SQL and NoSQL databases (e.g., PostgreSQL, MySQL, MongoDB, Cassandra).
Preferred Skills:
- Experience with AI development tools (eg. Copilot, Devin & Claude)
- Prior experience or a keen interest in the financial services industry
- Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements
- Experience of working in fast paced environment
- Flexible and adaptive, team player
- Excellent analytical and communication, interpersonal skills.
SGA is a technology and resource solutions provider driven to stand out. We are a women-owned business. Our mission: to solve big IT problems with a more personal, boutique approach. Each year, we match consultants like you to more than 1,000 engagements. When we say let's work better together, we mean it. You'll join a diverse team built on these core values: customer service, employee development, and quality and integrity in everything we do. Be yourself, love what you do and find your passion at work. Please find us at .
SGA is an Equal Opportunity Employer and does not discriminate on the basis of Race, Color, Sex, Sexual Orientation, Gender Identity, Religion, National Origin, Disability, Veteran Status, Age, Marital Status, Pregnancy, Genetic Information, or Other Legally Protected Status. We are committed to providing access, equal opportunity, and reasonable accommodation for individuals with disabilities in employment, and our services, programs, and activities. Please visit our company to request an accommodation or assistance regarding our policy.
Never miss an opportunity! Create an alert based on the job you applied for.