多模态模型如何把文本和图像“对齐”到同一套表示里？
How do multimodal models align text and images into one representation space?
通常分别用文本编码器和图像编码器把两者变成向量，再通过对比学习让“相配”的图片-文字向量靠近、不配的远离，于是同一语义在两种模态下落到相近的位置。
A text encoder and an image encoder turn each modality into vectors; contrastive learning pulls matching image-text pairs closer and pushes mismatches apart, so the same meaning lands near the same spot across modalities.
难度 进阶 · 适合 大众
CC BY 4.0 · 来源待核验 · Source pending review
来源 · Source
出处备注：本题原挂出处与答案内容不符，已撤回；具体书目待补，暂不展示来源。
本题原挂出处与答案内容不符，已撤回；具体书目待补。
Previous citation withdrawn as unsupported for this answer; a specific source is pending.
#多模态 #对齐
什么是视觉语言模型（VLM），它和普通大模型有何不同？
What is a vision-language model (VLM) and how does it differ from a plain LLM?
VLM 在语言模型之外接入了图像编码器，能把图片转成模型能读的“视觉词元”，于是你可以拿着一张图问它问题、让它描述或推理，而不只是处理纯文字。
A VLM adds an image encoder to a language model, turning a picture into 'visual tokens' the model can read, so you can ask about, describe, or reason over an image—not just plain text.
难度 进阶 · 适合 大众
CC BY 4.0 · 已核验 · Verified 2026-11-04
来源 · Source
Liu、Li、Wu 与 Lee,《Visual Instruction Tuning》,Advances in Neural Information Processing Systems 36,2023 (DOI: 10.52202/075280-1516)
Liu, Li, Wu & Lee, 'Visual Instruction Tuning', Advances in Neural Information Processing Systems 36, 2023, DOI: 10.52202/075280-1516
独立核验依据 · Independent verification: 核验依据 1 Crossref DOI 10.52202/075280-1516
#VLM #多模态
CLIP 的训练目标是什么？
What is the training objective of CLIP?
CLIP 用海量“图片-文字”对做对比学习：同一对的图文向量被拉到一起，不同对的被打散。训练后只需用文字向量就能去检索或分类图像，无需额外标注。
CLIP is trained contrastively on massive image-text pairs: matching pairs are pulled together, non-matching pairs pushed apart. After training, a text vector alone can retrieve or classify images without extra labels.
难度 进阶 · 适合 大众
CC BY 4.0 · 已核验 · Verified 2026-11-04
来源 · Source
Wen, Zhao, Chen, Pang, Qi,《What Makes CLIP More Robust to Long-Tailed Pre-Training Data? A Controlled Study for Transferable Insights》,Advances in Neural Information Processing Systems 37,2024 (DOI: 10.52202/079017-1153)
Wen, Zhao, Chen, Pang, Qi, 'What Makes CLIP More Robust to Long-Tailed Pre-Training Data? A Controlled Study for Transferable Insights', Advances in Neural Information Processing Systems 37, 2024, DOI: 10.52202/079017-1153
独立核验依据 · Independent verification: 核验依据 1 Crossref DOI 10.52202/079017-1153
#CLIP #对比学习