arXiv Computation and Language

In-Context Learning for Diagram Code Generation: What is the Effect on Faithfulness and Quality?

arXiv AI
Sep 1

Imag-Eval: a language-grounded framework for interpretable Text-to-Image instruction following evaluation

Imag‑Eval is a new language‑grounded benchmark for evaluating Text‑to‑Image models, focusing on how well they follow compositional natural‑language instructions. It disentangles prompt length from compositional difficulty by independently varying the number of instances and the combination of constraints (rules), providing 1,140 prompts and 8,842 rule combinations. The study shows that for structured skills, the difficulty is mainly driven by the number of grounded rules and their binding to instances rather than prompt length alone.

By Ibrahim Mohamed Serouis, David Jaramillo Duque
arXiv AI
Aug 5

Evaluating LLM-Based Goal Extraction in Requirements Engineering: Prompting Strategies and Their Limitations

arXiv:2604. 22207v2 Announce Type: replace-cross Abstract: Due to the textual and repetitive nature of many Requirements Engineering (RE) artefacts, Large Language Models (LLMs) have proven useful to automate their generation and processing.

By Anna Arnaudo, Riccardo Coppola, Maurizio Morisio, Flavio Giobergia, Andrea Bioddo, Angelo Bongiorno, Luca Dadone
arXiv AI
3d ago

Instruction-Tuning Open-Weight Language Models for BPMN Model Generation

The paper presents InstruBPM, an instruction‑tuned open‑weight large language model that generates BPMN process models from natural language descriptions. By preparing paired text‑diagram data and applying parameter‑efficient fine‑tuning with quantization, the authors demonstrate that the tuned model outperforms both untuned open‑weight baselines and strong proprietary models in sequence, structural, and guideline‑conformance metrics. The approach is cost‑effective, privacy‑preserving, and supports on‑prem deployment, with publicly released models and scripts for reproducibility.

By G\"okberk \c{C}elikmasat, Atay \"Ozg\"ovde, Fatma Ba\c{s}ak Aydemir
arXiv AI
Oct 2

PRISM: A Category-Theoretic Framework for Measuring and Refining Multimodal Analogies

PRISM is a modality‑agnostic, category‑theoretic framework that measures and refines multimodal analogies by representing them as explicit relational mappings. It introduces a pullback score to quantify relational alignment and an iterative refinement loop that uses this score as feedback to improve generated images. On the AnaloBench benchmark, PRISM’s pullback score alone achieves 82.5% accuracy, and human evaluations show a 57.65% preference for refined outputs, though refinement may sometimes favor visually crowded compositions.

By Mirella Zeisler, Ojas Shirekar, Mircea Lic\v{a}, Chirag Raman