NeSySpec is a neuro‑symbolic audit framework that uses Logic Tensor Networks to evaluate engineering specifications, detect violations, identify altered fields, propose minimal corrections, and link image evidence to parts. It learns tolerances from real records and integrates all audit outputs into a single verification report. The system was validated on 107 aircraft types, achieving superior macro‑F1 scores and higher accuracy in diagnosing constraints and locating altered fields compared to existing methods.
arXiv:2607. 23386v1 Announce Type: new Abstract: We document a failure class in frontier large language models -- exception chain collapse -- observed in eligibility evaluation under nested conditional rules of the form "A is required UNLESS B applies, UNLESS C overrides B".
The paper introduces Bounded-RS, a protocol for testing whether neural networks are representationally systematic. It evaluates explanations by checking if the claimed representation is recoverable and causally used, then classifies the explanation into one of four states: Unsupported, Present-only, Causal-use, or Undetermined. Applied to a semantic‑parsing benchmark, the protocol finds that T5’s role‑filler Binding explanation is Causal‑use while its Geometry explanation is Unsupported, and it demonstrates high reliability across different configurations.
The paper introduces BiasGuard‑D, a system that uses two adversarial language‑model agents from different families to exchange typed claim documents, with a third model quorum voting to adjudicate. A combined referential‑plus‑semantic invariant validates the verdict, and the iterative deliberation layer shifts remediation from stereotypes to hallucinations, improving right‑answer recovery by 49 percentage points on HaluEval QA. All claims, votes, and instructions are stored as typed records, providing an auditable provenance trail for operators under emerging AI regulations.
The paper introduces $A^2E^2$, a diagnostic framework that decomposes confusion in fine‑grained aircraft detection into four distinct, measurable sources: affinity, heterogeneity, contested, and collapsed. By attributing each source to either aleatoric or epistemic uncertainty and within‑ or between‑class distinctions, the method provides actionable remedies and experimentally verifies that targeted interventions reduce the identified source without affecting irreducible ones. This turns passive confusion matrices into a concrete, validatable diagnosis that can guide model improvement.
arXiv:2604. 27720v2 Announce Type: replace Abstract: Vision-language models (VLMs) are increasingly applied to medical visual question answering (Med-VQA), yet whether they can \emph{localize} the evidence behind their answers---a prerequisite for clinical auditability---is poorly characterized.
The paper investigates a failure mode in Graph-JEPA, a joint‑embedding predictive model trained on a large scientific‑reasoning graph. Despite achieving high linear‑probe accuracy and effective rank, the learned representation contains almost no usable instance information, as shown by retrieval metrics. The authors diagnose the issue to variance allocation in the objective, propose a repair that restores near‑perfect information recovery, and demonstrate that the problem persists even after repair, highlighting limitations in the evaluation metrics used.
arXiv:2608. 08786v1 Announce Type: new Abstract: Large language models (LLMs) increasingly serve as data-driven reasoners, yet their chains-of-thought (CoT) can be unfaithful even when final answers are correct.
arXiv:2608. 04975v1 Announce Type: cross Abstract: SciCode is the standard measure of the scientific-coding ability of language models: research-level problems that demand both frontier scientific theory and its implementation as working numerical code.
arXiv:2610.00111v1 Announce Type: new Abstract: Model judges now supervise multimodal systems at scale, filtering training data, selecting outputs, and supplying the reward that shapes multimodal rea...
arXiv:2608. 07946v1 Announce Type: cross Abstract: Text-to-SQL benchmarks ship schemas whose column names already say what the columns mean.
arXiv:2606. 13706v1 Announce Type: cross Abstract: We present HierSVA, an integrated suite that combines a pipeline, dataset, and benchmark for LLM-driven hierarchical hardware formal verification.
arXiv:2510. 14538v3 Announce Type: replace Abstract: Neuro-symbolic (NeSy) AI aims to develop deep neural networks whose predictions comply with prior knowledge encoding, e.