Overview
On Site
Depends on Experience
Accepts corp to corp applications
Contract - W2
Contract - 12 Month(s)
Skills
VLA fine-tuning
VLA Models
Fine-tuning
Action Expert
Knowledge Insulation
Job Details
Hi,
Position: VLA fine-tuning
Location: Mountainview , CA
Duration: Long Term
Interview process: 1 video Interview
Core concepts and technologies
- VLA Models: VLAs combine a VLM (Vision-Language Model), a pre-trained model that understands images and text, with an action decoder. The VLM processes visual observations and language instructions, and the action decoder translates the VLM's output into the continuous movements and commands needed to operate a robot.
- Fine-tuning: This process adapts a generalist VLM for a specific set of robotic tasks. It is crucial for getting satisfactory performance out of VLAs when deploying them on new robots or in new environments.
- Action Expert: In modern VLA architectures, the action expert is a module that decodes continuous actions for the robot. Instead of generating actions one by one, newer techniques like flow matching allow the expert to generate a full "chunk" of continuous actions at once, significantly reducing computation time.
- Knowledge Insulation: This advanced technique fine-tunes the VLM backbone with discretized actions to learn high-quality representations while preventing the gradients from the action expert from flowing back into the VLM. This allows the action expert to be trained for fluent continuous actions separately.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.