INCONVENIENT: A Benchmark to Assess LLM Generalization Capabilities from Convenience Samples
Read the original on arXiv Machine Learning →INCONVENIENT is a new benchmark that evaluates how machine‑learning models, especially large language models, generalize when trained on convenience samples that bias feature–target relationships. Unlike prior distribution‑shift tests that rely on natural domain splits, INCONVENIENT explicitly perturbs these relationships while keeping the deployment population unchanged, allowing controlled study of shift severity and direction. The benchmark covers 200 tabular classification and regression datasets and produces about 15,500 shifted training distributions, revealing significant performance degradation.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.