Beyond Consensus: Perspectivist Modeling and Evaluation of Annotator Disagreement in NLP
A new arXiv survey explores perspectivist NLP modeling, treating human annotator disagreement as meaningful signal rather than noise.
Use this view to inspect the underlying evidence corpus. For ranked developments, decision posture and interpretation, use Signals.
Raw feed or Signals?
Raw feed is chronological evidence. Signals ranks and interprets material change.
A new arXiv survey explores perspectivist NLP modeling, treating human annotator disagreement as meaningful signal rather than noise.
Researchers found large language models exhibit systemic pro-AI bias, over-recommending AI options and overestimating AI role salaries.
Academic research finds large language models differ significantly from human preferences when selecting goals in self-directed tasks.
Researchers introduced RubricEM, a reinforcement learning framework using rubrics to train deep research agents without verifiers.
Researchers propose a metric and preference optimisation method to address cross-lingual cultural inconsistencies in multilingual LLMs.
Researchers introduced PragReST, a self-supervised framework improving large language models in pragmatic language understanding.
Researchers propose direct timestep embedding and contrastive alignment to bypass tokenization bottlenecks in time-series question answering.
Researchers introduced EdgeWise, an edge-cloud framework for long-horizon agents that selectively delegates key steps to cloud LLMs.
A new research survey synthesizes transferable attack mechanisms in AI systems, establishing a unified taxonomy for cross-model security risks.
Researchers propose AutoPrunedRetriever, using persistent reasoning graphs to improve RAG efficiency across multi-step queries.
Researchers propose a framework to address proxy failure in LLM uncertainty estimation for better hallucination detection.
Researchers introduce MuPHI, a diagnostic benchmark for evaluating cross-modal reasoning in detecting implicit multimodal harm.
Researchers examine combining large language models with statistical methods for causal discovery, highlighting token costs and error risks.
Researchers propose group-graph policy optimization to address reward sparsity in long-horizon agentic reinforcement learning for LLMs.
Researchers introduced Audit-First VAPO, a technique to prevent harmful model updates under imperfect verification using finite-sample bounds.
SpaceCast-Bench evaluates predictive spatial reasoning in vision-language models, moving beyond static perception to intervention modeling.
Researchers benchmarked Jev, a source verifier for LLMs, to detect when financial calculations cite wrong identical numbers.
Researchers propose FAER, an auditable full-trajectory replay framework for language model post-training alignment and selector efficiency.
Researchers introduced an execution contract that cryptographically binds tool-agent claims to their underlying traces and sources.
ContractRL introduces a verifier-guided JSON repair protocol for structured tool calls, modelling schema error correction as a decision process.
Researchers propose using chess as a cheap, contamination-resistant benchmark to evaluate large language model prompt optimization.
Researchers release NARCBench, a benchmark using internal model representations to detect covert collusion between LLM agents.
Researchers propose VStress, a correlation-aware allocation policy to optimize repeated verifier calls and reduce inference costs.
Researchers propose CVaR anchor regression to improve out-of-distribution prediction robustness against rare, large shifts in data.
Researchers propose KaLM-Reranker-V1, a fast non-late-interaction reranker that decouples query and passage computation for retrieval systems.
New research shows LLM-as-a-judge systems exhibit bidirectional bias based on self- and other-labeling, independent of text quality.
Researchers propose predicting task difficulty from text descriptions to optimize agent evaluations without costly simulations.
Researchers introduced Poli-Bias, a counterfactual framework evaluating how LLMs alter legal and political reasoning based on country names.
Academic research demonstrates that accurate ML-based option pricing does not guarantee accurate recovery of latent risk-neutral densities.
TabRank introduces a chain-of-thought distillation method for improving table re-rankers, enhancing structured information retrieval via LLMs.