Overview
Skills
Job Details
Role: Lead Data engineer with Pyspark and Python
Duration: 6-12 Months
Work location: Bay Area, CA
Department Overview
The Data Solutions Architecture Team at client is responsible for driving long-term, enterprise-wide data solutions, target state architecture, and overall excellence with the application of data, analytics and information to critical business challenges and opportunities. This team is chartered to develop the strategy, roadmap, and accompanying standards that will enable better use of data and information and to develop analytics maturity at client. It is also the central coordinating team for technical leadership for our data product development efforts.
Position Summary
The Digital Utility runs on data and information. At Client we have many teams building data products that need guidance, and our Data Systems Architecture team provides experienced team leads to run data product delivery teams. The Expert Tech Lead leads one or more product delivery teams,
As an Expert Tech Lead, you will lead data product delivery teams on our Palantir Foundry data platform, and may be expected to branch out into our other key platforms, Snowflake and Informatica.
As a team leader, you will be responsible for bringing to life the vision of our business teams, driving requirements discussions, architecture, and code.
You will lead technical design and implementation of multiple use cases and drive technical feasibility discussions for our products as you collaborate with other technical leaders, product owners and development leads. You will be required to discover the boundaries of our platforms, provide solutions for technical problems and drive discussions within IT and Lines of Business. As a productive leader of a motivated group of engineers, product teams, and technical analysts, you will be producing ground-breaking, forward thinking, and revolutionary projects that will modernize critical infrastructure. You will also be responsible for mentoring engineers, as well as upholding standards of code and execution quality, documentation, reliability, and maintainability in the projects you lead.
The Expert Tech Lead will report to a Manager of Data Solutions Architecture in the Data Analytics & Insights department of Information Technology, and work closely with product delivery teams in our Analytic Product Delivery group.
Job Responsibilities
- Lead technical design and implementation of Palantir Foundry use cases
- Cross-train into Snowflake + Informatica implementation as needed
- Ensure best practices are followed for performance tuning, security and data governance
- Build awesome big data products within a fun team where culture is the foundation for success
- Lead and mentor the development team in a hands-on way
- Provide technical feasibility assessment while collaborating with Product Owner and Design team to guide features and provide technical know-how
- Define and address application non-functional requirements, and manage technical debt
- Debug and optimize complex applications
- Effectively communicate the vision and architecture to team members
- Maintain a productive working environment for development teams
- Be a strong advocate for a culture of quality across the team
- Collaborate with external teams, especially security, to design, implement and support solutions that are scalable and efficient
- Effectively communicate the solution to non-technical stakeholders
- Follow an agile development methodology
- Keep up-to-date with product capabilities of Palantir Foundry releases, features and best practices and recommend changes to improve efficiency, performance and scalability
- Lead the way in deploying cutting-edge Foundry capabilities against real business problems
- Advocate for appropriate data modeling techniques to enable agile development and the business use cases, specifically in the integration of event-driven with batch data processing
Qualifications
Minimum:
- High School or GED-General Educational Development-GED Diploma
- Experience as Technical Lead, 3 years
- Intermediate to Expert Level Data Engineering background including complex SQL, Spark and PySpark queries and data modeling experience
- Intermediate to Expert Level Development in Python
Desired:
- BA/BS degree in Computer Science, Engineering, Business or related field or equivalent experience
- 7 years of experience in IT working in application development and architecture, with 5 years project implementation experience and hands-on experience developing data products
- Expert in the solution architecture discipline, processes, concepts and best practices
- Exposure to a process improvement discipline (e.g. Six Sigma) used within a technology supported business process implementation
- Process design, information modeling and / or system solution architecture development with project or portfolio focus in at least one of the following: Cloud Architecture, Data Management, Security, Microservices, or Infrastructure technologies
- Experience providing operational support for applications, systems, or infrastructure
- Experience providing direction to design and engineering staff
- Experience providing direction to one or more architects and technical leads
- 3 years of experience developing solution architectures for complex data platforms and products
- 3 years of hands-on experience working with multiple technologies and platforms such as integrations with ERP systems (e.g. SAP), Cloud technologies (e.g. AWS) etc.
- 1 year experience on Palantir Foundry-specific use cases
- Demonstrated solid understanding of architecture best practices and enterprise-scale solution development
- Able to provide individual or project oversight on rapid prototyping/POC efforts and large-scale enterprise wide roll out planning
Best Regards,
Vishal
Truth Lies in Heart