The Flow has not summarised this story yet — read it at arXiv Machine Learning.
The paper uncovers a systematic mismatch in modern text embeddings: while sentiment can be extracted with high accuracy from frozen encoders, the embeddings’ cosine similarity metric barely reflects polarity differences. Across eleven models from 2021–2026, a linear probe achieves up to 91.3% accuracy, yet cosine similarity margins are only 0.003–0.064, leading to poor retrieval performance (e.g., Qwen3-Embedding-4B’s 64.3 MAP). Introducing a lightweight supervised‑contrastive head (Sentiment‑Injected Sentence Embeddings) dramatically improves probe accuracy, retrieval MAP, and polarity margins, but at the cost of downstream semantic similarity. "whyItMatters":"The findings reveal that widely used retrieval and clustering systems may ignore valuable sentiment information encoded in embeddings, impacting their effectiveness."
arXiv:2610.34901v1 Announce Type: cross Abstract: Training-free few-shot adaptation of vision-language models typically constructs a classifier from CLIP's final projected image representation, leavi...
The paper introduces Spectral Transductive Refinement (STR), a training‑free method that refines class prototypes at test time using the geometry of a joint k‑nearest‑neighbour graph and a normalized‑Laplacian spectral coordinate system. STR operates solely on frozen visual embeddings, iteratively updating pseudo‑labelled queries to improve one‑shot and few‑shot classification under domain shift. Experiments on ResNet‑18 and ResNet‑10 backbones show STR outperforms single‑prototype baselines and rivals meta‑trained cross‑domain few‑shot methods, achieving the best 1‑shot average across eight target domains.
ORCA (Orthogonal Residual Compositional Alignment) is a method that improves text-to-image diffusion models by aligning the diffusion transformer’s latent space with a low‑rank target derived from a frozen visual encoder. It introduces an auxiliary loss that uses a predictor with an orthogonal basis parameterised by a learned residual between T5 and CLIP embeddings, providing a prompt‑dependent signal for selecting the visual readout subspace. Experiments on three diffusion‑transformer backbones show that ORCA improves FID and GenEval scores, especially on attribute binding, spatial relations, and multi‑object prompts, without adding inference‑time cost.
CoVA introduces a lightweight framework that composes VAE features online to provide stage‑wise guidance for flow‑matching training, improving semantic discrimination and spatial structure without extra inference overhead. By jointly training a small latent composer with class supervision, masked reconstruction, and flip consistency, CoVA aligns intermediate SiT features early and later uses detached composer posteriors for sample‑dependent soft‑label conditioning. On ImageNet 256×256, CoVA cuts SiT‑XL/2 FID from 17.2 to 7.4 in 400K steps and achieves an FID of 2.03 in only 400 epochs with classifier‑free guidance, adding just 8.5 M training‑only parameters.
Omni-Embed-Mini is a 0.9B‑parameter model that embeds text, speech, audio, images, video, and visually‑rich documents into a single shared cosine space without updating any text‑side parameters. It uses a dense cascaded caption as a teacher signal, allowing the teacher and student to share identical backbone weights and requiring only lightweight projectors and phased LoRA adapters for alignment. The model achieves strong text retrieval performance (49.57 nDCG@10 on MTEB‑v2 BEIR‑8) while extending to five additional modalities and is significantly smaller than other open omni‑modal embedders.