ALT Prompt tuning combines the best features of model tuning and prompt design to enable pre-trained models to achieve strong task performance while being served efficiently in a multi-task setting. Like prompt design, the pre-trained model is fixed and shared across tasks. However, learned soft prompts are added to the model input, enabling stronger adaptation to specific tasks.