← Back to all news
arXiv Machine Learning By Utku Cicek, Mahip Singh, Ziyao Shang, Kimathi Kaai, C Thomas, Pedro Pablo Guerrero Vela, Alexander Wong, Sirisha Rambhatla

Pixels to Tokens: Token Space Efficient Active Learning for Low-Budget Semantic Segmentation

Read the original on arXiv Machine Learning →

Pixels to Tokens introduces TEAL, a token‑space efficient active learning framework that operates at the pixel level for semantic segmentation using a frozen Vision Transformer backbone. TEAL selects diverse candidate pixels via a two‑level MaxHerding strategy on multi‑layer token descriptors and refines uncertainty on a finer decoder grid, avoiding interpolation artifacts. Across multiple datasets, TEAL outperforms prior baselines under extreme label scarcity, achieving up to +24.36 mIoU with a single labeled pixel per image and reaching 97% of fully supervised performance with only ten pixels.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Computer Vision
Sep 4

Efficient Semantic Understanding from Digital Foveation

The paper proposes a lightweight active‑vision pipeline that mimics biological foveation to perform semantic segmentation more efficiently. By selecting salient fixations, observing high‑resolution foveal patches, and using low‑resolution context, the method achieves 95.9% of baseline Top‑1 accuracy with only 4.7% of the computational cost, and recovers 90.6% of baseline object recall using 58.6% of the computation. The results demonstrate that sparse, selectively allocated observations can yield substantial semantic understanding, suggesting active vision as a viable alternative to uniform dense processing.

By Caterina Caccavella, Vittorio Fra, Andreas Ziegler, Giulia D'Angelo, Yulia Sandamirskaya
computer-vision
More like this →
arXiv AI
Jun 16

ActiveSAM: Image-Conditional Class Pruning for Fast and Accurate Open-Vocabulary Segmentation

arXiv:2606. 16996v1 Announce Type: cross Abstract: Segment Anything Model 3 (SAM 3) provides a strong frozen backbone for concept-prompted segmentation, but applying it directly to open-vocabulary semantic segmentation (OVSS) is inefficient: full-resolution decoding is typically run over the entire dataset vocabulary, whereas each image contains only a small active subset of classes.

By Tran Dinh Tien, Zhiqiang Shen
agentscomputer-visionroboticsefficiencybenchmarks
More like this →
Hugging Face Trending Papers
Sep 3

Efficient Semantic Understanding from Digital Foveation

The paper proposes a lightweight active‑vision pipeline that mimics biological foveation to perform dense semantic segmentation more efficiently. By selecting salient fixations, it uses high‑resolution foveal views, low‑resolution context, and adaptive computation to accumulate semantic information. On ADE20K‑Object, a single foveated observation attains 95.9% of baseline Top‑1 accuracy with only 4.7% of the computational cost, and semantic accumulation recovers 90.6% of baseline recall using 58.6% of the computation.

computer-vision
More like this →
arXiv Computer Vision
Sep 17

Decoder-Agnostic Token Merging for Vision Transformers: A Systematic Study of G2TM

The paper studies Graph-Guided Token Merging (G2TM), a module that reduces token count in Vision Transformers. It evaluates G2TM across multiple segmentation frameworks and decoder types, finding that its performance gains are tied to the encoder rather than the decoder. The authors report consistent reductions in GFLOPs (22‑47%) and throughput improvements (up to 74%) on ADE20K, with optimal hyperparameters depending mainly on backbone pre‑training and target dataset.

By Victor Bercy, Martyna Poreba, Michal Szczepanski, Samia Bouchafa
llmscomputer-visionefficiencybenchmarks
More like this →