Data Pipeline Engineer

What We Want to See

• Data Engineering Skills: Strong experience with PySpark, Spark, and/or Polars is highly desirable. Pandas is acceptable when it''s been used for data transformations and pipelining (more on when it''s not sufficient below). Databricks experience is okay but not a strong signal on its own. A note on "ETL pipelines"—this phrase appears on many resumes, but it needs to be backed up by specific tools like the ones listed above in the candidate''s skills section.
• Python Proficiency: This is essential. Java can work as a complement if the candidate is clearly strong in it, but Python experience is still expected. C++ is common in DoD backgrounds, but few candidates are truly proficient in it, and it''s never used in this role—so it shouldn''t be weighted heavily.
• Networking Experience: We value hands-on experience with setting up or debugging endpoints and APIs, tracing network traffic, and reverse-engineering backend services.
• SQL: We treat this as a baseline prerequisite, but it doesn''t carry a resume on its own. It''s rarely used day-to-day, though candidates should be able to understand it when it comes up in a transform.

The Role

• As a Subcontractor Engineer - USG you will help develop and deploy technical solutions to solve our customers’ hardest problems, using our platform to integrate data, transform insights, and build first-class applications for operational decisions. You will leverage everything around you: core company products, open-source technologies (e.g. GHE), and anything you and your team can build to drive real impact. In this role, you work with customers around the globe, where you gain rare insight into the world’s most important industries and institutions. Each mission presents different challenges, from the regulatory environment to the nature of the data to the user population. You will work to accommodate all aspects of an environment to drive real technical outcomes for our customers!

Core Responsibilities

• Setup transfers of data feeds from source systems into location accessible to Foundry
• Debug issues related to delayed or missing data feeds
• Write transformations and derive new datasets using Spark for distributed computation
• Monitor build progress and debug build problems
• Using Foundry’s application development framework to design applications that address operational questions
• Rapid development and iteration cycles with SME’s including testing and troubleshooting application issues
• Executing requests for information (RFI’s) surrounding the platform’s data footprint
• Iterate with product teams while debugging / understanding the code backing our core platform

What We Value

• Strong engineering background, preferably in fields such as Computer Science, Mathematics, Software Engineering, Physics, or Data Science.
• Proficiency with programming languages such as Python (PySpark, Pandas) SQL, R or similar languages.
• Strong understanding of algorithms and data structures as they pertain to development / problem solving
• Ability to work effectively in teams of technical and non-technical individuals.
• Skill and comfort working in a rapidly changing environment with dynamic objectives and iteration with users.
• Demonstrated ability to continuously learn, work independently, and make decisions with minimal supervision.
• Proven track-record of strong customer communications including feedback gathering, execution updates, and troubleshooting.

What We Require

Minimum Secret Clearance