Testerpce
Testerpce
AI & ML interests
None yet
Recent Activity
updated a collection 37 minutes ago
Long context updated a collection 37 minutes ago
Meta learning and Evolution updated a collection 38 minutes ago
Agent Organizations
Robotics
Meta learning and Evolution
-
EvoMaster: A Foundational Agent Framework for Building Evolving Autonomous Scientific Agents at Scale
Paper • 2604.17406 • Published • 6 -
Solvita: Enhancing Large Language Models for Competitive Programming via Agentic Evolution
Paper • 2605.15301 • Published • 22 -
Learning to Foresee: Unveiling the Unlocking Efficiency of On-Policy Distillation
Paper • 2605.11739 • Published • 61 -
SkillsVote: Lifecycle Governance of Agent Skills from Collection, Recommendation to Evolution
Paper • 2605.18401 • Published • 130
RAI
Continual Learning
-
Online Experiential Learning for Language Models
Paper • 2603.16856 • Published • 60 -
On Token's Dilemma: Dynamic MoE with Drift-Aware Token Assignment for Continual Learning of Large Vision Language Models
Paper • 2603.27481 • Published • 35 -
Rethinking Continual Experience Internalization for Self-Evolving LLM Agents
Paper • 2606.04703 • Published • 26
Embedding
-
Scaling Embeddings Outperforms Scaling Experts in Language Models
Paper • 2601.21204 • Published • 105 -
jina-embeddings-v5-text: Task-Targeted Embedding Distillation
Paper • 2602.15547 • Published • 31 -
ManCAR: Manifold-Constrained Latent Reasoning with Adaptive Test-Time Computation for Sequential Recommendation
Paper • 2602.20093 • Published • 29 -
Distribution-Conditioned Transport
Paper • 2603.04736 • Published • 3
OCR
-
HunyuanOCR Technical Report
Paper • 2511.19575 • Published • 23 -
DeepSeek-OCR 2: Visual Causal Flow
Paper • 2601.20552 • Published • 73 -
CodeOCR: On the Effectiveness of Vision Language Models in Code Understanding
Paper • 2602.01785 • Published • 97 -
MinerU-Diffusion: Rethinking Document OCR as Inverse Rendering via Diffusion Decoding
Paper • 2603.22458 • Published • 139
Tokenization methods and language modelling
-
Continuous Autoregressive Language Models
Paper • 2510.27688 • Published • 74 -
Efficient Speech Language Modeling via Energy Distance in Continuous Latent Space
Paper • 2505.13181 • Published • 9 -
Long-Context Autoregressive Video Modeling with Next-Frame Prediction
Paper • 2503.19325 • Published • 73 -
Bridging Continuous and Discrete Tokens for Autoregressive Visual Generation
Paper • 2503.16430 • Published • 34
Tool
-
Provable Benefits of In-Tool Learning for Large Language Models
Paper • 2508.20755 • Published • 11 -
SimpleTIR: End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoning
Paper • 2509.02479 • Published • 84 -
How Can Input Reformulation Improve Tool Usage Accuracy in a Complex Dynamic Environment? A Study on τ-bench
Paper • 2508.20931 • Published • 16 -
THOR: Tool-Integrated Hierarchical Optimization via RL for Mathematical Reasoning
Paper • 2509.13761 • Published • 16
LLM judge
-
One Token to Fool LLM-as-a-Judge
Paper • 2507.08794 • Published • 32 -
Self-Improving VLM Judges Without Human Annotations
Paper • 2512.05145 • Published • 21 -
RubricBench: Aligning Model-Generated Rubrics with Human Standards
Paper • 2603.01562 • Published • 64 -
Xpertbench: Expert Level Tasks with Rubrics-Based Evaluation
Paper • 2604.02368 • Published • 12
3D
-
Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modeling
Paper • 2507.07982 • Published • 34 -
MeshLLM: Empowering Large Language Models to Progressively Understand and Generate 3D Mesh
Paper • 2508.01242 • Published • 11 -
Loc3R-VLM: Language-based Localization and 3D Reasoning with Vision-Language Models
Paper • 2603.18002 • Published • 15 -
Generation Models Know Space: Unleashing Implicit 3D Priors for Scene Understanding
Paper • 2603.19235 • Published • 95
Materials and structures
Vision
-
MiCo: Multi-image Contrast for Reinforcement Visual Reasoning
Paper • 2506.22434 • Published • 10 -
VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learning
Paper • 2507.13348 • Published • 80 -
RewardDance: Reward Scaling in Visual Generation
Paper • 2509.08826 • Published • 73 -
Grasp Any Region: Towards Precise, Contextual Pixel Understanding for Multimodal LLMs
Paper • 2510.18876 • Published • 37
Code
-
The Debugging Decay Index: Rethinking Debugging Strategies for Code LLMs
Paper • 2506.18403 • Published • 3 -
ReCode: Updating Code API Knowledge with Reinforcement Learning
Paper • 2506.20495 • Published • 10 -
SWE-Debate: Competitive Multi-Agent Debate for Software Issue Resolution
Paper • 2507.23348 • Published • 12 -
LoCoBench: A Benchmark for Long-Context Large Language Models in Complex Software Engineering
Paper • 2509.09614 • Published • 7
Data
-
Skywork-SWE: Unveiling Data Scaling Laws for Software Engineering in LLMs
Paper • 2506.19290 • Published • 53 -
Data Efficacy for Language Model Training
Paper • 2506.21545 • Published • 11 -
Easy Dataset: A Unified and Extensible Framework for Synthesizing LLM Fine-Tuning Data from Unstructured Documents
Paper • 2507.04009 • Published • 55 -
RefineX: Learning to Refine Pre-training Data at Scale from Expert-Guided Programs
Paper • 2507.03253 • Published • 19
Memory
-
Xolver: Multi-Agent Reasoning with Holistic Experience Learning Just Like an Olympiad Team
Paper • 2506.14234 • Published • 41 -
MoTE: Mixture of Ternary Experts for Memory-efficient Large Multimodal Models
Paper • 2506.14435 • Published • 7 -
Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
Paper • 2504.19413 • Published • 67 -
MemOS: A Memory OS for AI System
Paper • 2507.03724 • Published • 168
Applications and Uses
-
ComfyUI-R1: Exploring Reasoning Models for Workflow Generation
Paper • 2506.09790 • Published • 53 -
Saffron-1: Towards an Inference Scaling Paradigm for LLM Safety Assurance
Paper • 2506.06444 • Published • 73 -
DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents
Paper • 2506.11763 • Published • 74 -
Agentic Reasoning: Reasoning LLMs with Tools for the Deep Research
Paper • 2502.04644 • Published • 4
Adversarial
Multimodal
-
Qwen2.5-Omni Technical Report
Paper • 2503.20215 • Published • 174 -
Unsupervised Post-Training for Multi-Modal LLM Reasoning via GRPO
Paper • 2505.22453 • Published • 46 -
UniRL: Self-Improving Unified Multimodal Models via Supervised and Reinforcement Learning
Paper • 2505.23380 • Published • 22 -
More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models
Paper • 2505.21523 • Published • 13
Interpretable
-
I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders
Paper • 2503.18878 • Published • 121 -
Large Language Models are Locally Linear Mappings
Paper • 2505.24293 • Published • 14 -
Thought Anchors: Which LLM Reasoning Steps Matter?
Paper • 2506.19143 • Published • 14 -
Steered LLM Activations are Non-Surjective
Paper • 2604.09839 • Published • 15
Diffusion
-
Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models
Paper • 2503.09573 • Published • 77 -
Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective
Paper • 2505.15045 • Published • 56 -
Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding
Paper • 2505.16990 • Published • 22 -
D-AR: Diffusion via Autoregressive Models
Paper • 2505.23660 • Published • 34
Information_retrieval
-
Rank1: Test-Time Compute for Reranking in Information Retrieval
Paper • 2502.18418 • Published • 29 -
Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
Paper • 2503.09516 • Published • 41 -
Fixing Data That Hurts Performance: Cascading LLMs to Relabel Hard Negatives for Robust Information Retrieval
Paper • 2505.16967 • Published • 24 -
SitEmb-v1.5: Improved Context-Aware Dense Retrieval for Semantic Association and Long Story Comprehension
Paper • 2508.01959 • Published • 61
Attention
-
MiniMax-01: Scaling Foundation Models with Lightning Attention
Paper • 2501.08313 • Published • 304 -
Lizard: An Efficient Linearization Framework for Large Language Models
Paper • 2507.09025 • Published • 19 -
On the Expressiveness of Softmax Attention: A Recurrent Neural Network Perspective
Paper • 2507.23632 • Published • 6 -
Causal Attention with Lookahead Keys
Paper • 2509.07301 • Published • 21
Agent
-
Large Language Models Orchestrating Structured Reasoning Achieve Kaggle Grandmaster Level
Paper • 2411.03562 • Published • 70 -
Training Language Models for Social Deduction with Multi-Agent Reinforcement Learning
Paper • 2502.06060 • Published • 37 -
MLGym: A New Framework and Benchmark for Advancing AI Research Agents
Paper • 2502.14499 • Published • 196 -
SurveyX: Academic Survey Automation via Large Language Models
Paper • 2502.14776 • Published • 100
RAG
-
StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurization
Paper • 2410.08815 • Published • 47 -
SKETCH: Structured Knowledge Enhanced Text Comprehension for Holistic Retrieval
Paper • 2412.15443 • Published • 10 -
RAG-Star: Enhancing Deliberative Reasoning with Retrieval Augmented Verification and Refinement
Paper • 2412.12881 • Published • 2 -
AR-RAG: Autoregressive Retrieval Augmentation for Image Generation
Paper • 2506.06962 • Published • 28
Planning
-
Compositional Foundation Models for Hierarchical Planning
Paper • 2309.08587 • Published • 11 -
ALPINE: Unveiling the Planning Capability of Autoregressive Learning in Language Models
Paper • 2405.09220 • Published • 27 -
WALL-E 2.0: World Alignment by NeuroSymbolic Learning improves World Model-based LLM Agents
Paper • 2504.15785 • Published • 22 -
CODA: Coordinating the Cerebrum and Cerebellum for a Dual-Brain Computer Use Agent with Decoupled Reinforcement Learning
Paper • 2508.20096 • Published • 37
Multilingual
-
CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages
Paper • 2309.09400 • Published • 87 -
Tuning LLMs with Contrastive Alignment Instructions for Machine Translation in Unseen, Low-resource Languages
Paper • 2401.05811 • Published • 9 -
Is Preference Alignment Always the Best Option to Enhance LLM-Based Translation? An Empirical Analysis
Paper • 2409.20059 • Published • 16 -
Are Character-level Translations Worth the Wait? Comparing Character- and Subword-level Models for Machine Translation
Paper • 2302.14220 • Published
Partial layer training LLMs
-
Sorted LLaMA: Unlocking the Potential of Intermediate Layers of Large Language Models for Dynamic Inference Using Sorted Fine-Tuning (SoFT)
Paper • 2309.08968 • Published • 24 -
GraLoRA: Granular Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
Paper • 2505.20355 • Published • 37 -
Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding
Paper • 2505.22618 • Published • 47 -
Latent Zoning Network: A Unified Principle for Generative Modeling, Representation Learning, and Classification
Paper • 2509.15591 • Published • 46
Evaluation
-
GPT-Fathom: Benchmarking Large Language Models to Decipher the Evolutionary Path towards GPT-4 and Beyond
Paper • 2309.16583 • Published • 13 -
Prometheus: Inducing Fine-grained Evaluation Capability in Language Models
Paper • 2310.08491 • Published • 57 -
SO-Bench: A Structural Output Evaluation of Multimodal LLMs
Paper • 2511.21750 • Published • 6 -
LLM Swiss Round: Aggregating Multi-Benchmark Performance via Competitive Swiss-System Dynamics
Paper • 2512.21010 • Published • 4
Math
-
Transformers Can Do Arithmetic with the Right Embeddings
Paper • 2405.17399 • Published • 55 -
Solving Inequality Proofs with Large Language Models
Paper • 2506.07927 • Published • 20 -
Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoning
Paper • 2507.00432 • Published • 79 -
CriticLean: Critic-Guided Reinforcement Learning for Mathematical Formalization
Paper • 2507.06181 • Published • 45
Style transfer
Reinforcement learning
-
Diffusion Augmented Agents: A Framework for Efficient Exploration and Transfer Learning
Paper • 2407.20798 • Published • 24 -
Offline Reinforcement Learning for LLM Multi-Step Reasoning
Paper • 2412.16145 • Published • 38 -
REINFORCE++: A Simple and Efficient Approach for Aligning Large Language Models
Paper • 2501.03262 • Published • 103 -
SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution
Paper • 2502.18449 • Published • 75
Knowledge
-
Leveraging Open Knowledge for Advancing Task Expertise in Large Language Models
Paper • 2408.15915 • Published • 19 -
ReLearn: Unlearning via Learning for Large Language Models
Paper • 2502.11190 • Published • 30 -
ReaRAG: Knowledge-guided Reasoning Enhances Factuality of Large Reasoning Models with Iterative Retrieval Augmented Generation
Paper • 2503.21729 • Published • 29 -
Recitation over Reasoning: How Cutting-Edge Language Models Can Fail on Elementary School-Level Reasoning Problems?
Paper • 2504.00509 • Published • 24
Distillation
-
MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training
Paper • 2606.30406 • Published • 15 -
TurnOPD: Making On-Policy Distillation Turn-Aware for Efficient Long-Horizon Agent Training
Paper • 2607.05804 • Published • 18 -
Trust Region Policy Distillation
Paper • 2607.04751 • Published • 34 -
SEED: Self-Evolving On-Policy Distillation for Agentic Reinforcement Learning
Paper • 2607.14777 • Published • 93
Causal
-
LLM Explainability with Counterfactual Chains and Causal Graphs
Paper • 2606.05972 • Published • 18 -
CausalMix: Data Mixture as Causal Inference for Language Model Training
Paper • 2607.01104 • Published • 21 -
CausaLab: A Scalable Environment for Interactive Causal Discovery Toward AI Scientists
Paper • 2605.26029 • Published • 18
Audio
-
Audio Flamingo Next: Next-Generation Open Audio-Language Models for Speech, Sound, and Music
Paper • 2604.10905 • Published • 29 -
Native Audio-Visual Alignment for Generation
Paper • 2605.30073 • Published • 33 -
UnityShots: Memory-Driven Multi-Shot Audio-Video Generation with Boundary-Aware Gating
Paper • 2606.21661 • Published • 28 -
WanSong v1.0 Technical Report
Paper • 2607.14749 • Published • 17
Medical and Biology
Inference
-
DualPath: Breaking the Storage Bandwidth Bottleneck in Agentic LLM Inference
Paper • 2602.21548 • Published • 56 -
TAPS: Task Aware Proposal Distributions for Speculative Sampling
Paper • 2603.27027 • Published • 147 -
Draft Less, Retrieve More: Hybrid Tree Construction for Speculative Decoding
Paper • 2605.20104 • Published • 7 -
Domino: Decoupling Causal Modeling from Autoregressive Drafting in Speculative Decoding
Paper • 2605.29707 • Published • 152
Optimization
-
Learnable Multipliers: Freeing the Scale of Language Model Matrix Layers
Paper • 2601.04890 • Published • 44 -
MSign: An Optimizer Preventing Training Instability in Large Language Models via Stable Rank Restoration
Paper • 2602.01734 • Published • 34 -
On Surprising Effectiveness of Masking Updates in Adaptive Optimizers
Paper • 2602.15322 • Published • 11 -
Flash-KMeans: Fast and Memory-Efficient Exact K-Means
Paper • 2603.09229 • Published • 85
Model Arithmetic
-
Souper-Model: How Simple Arithmetic Unlocks State-of-the-Art LLM Performance
Paper • 2511.13254 • Published • 140 -
Token-Level LLM Collaboration via FusionRoute
Paper • 2601.05106 • Published • 40 -
Darwin Family: MRI-Trust-Weighted Evolutionary Merging for Training-Free Scaling of Language-Model Reasoning
Paper • 2605.14386 • Published • 67
Tabular ML
Foundation Models
-
GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models
Paper • 2508.06471 • Published • 213 -
Cognitive Kernel-Pro: A Framework for Deep Research Agents and Agent Foundation Models Training
Paper • 2508.00414 • Published • 96 -
Continuous Autoregressive Language Models
Paper • 2510.27688 • Published • 74 -
MiMo-Embodied: X-Embodied Foundation Model Technical Report
Paper • 2511.16518 • Published • 26
Test time
-
Skip a Layer or Loop it? Test-Time Depth Adaptation of Pretrained LLMs
Paper • 2507.07996 • Published • 36 -
Test-Time Scaling with Reflective Generative Model
Paper • 2507.01951 • Published • 108 -
Does More Inference-Time Compute Really Help Robustness?
Paper • 2507.15974 • Published • 7 -
TTCS: Test-Time Curriculum Synthesis for Self-Evolving
Paper • 2601.22628 • Published • 35
Physics and operators
-
Lost in Latent Space: An Empirical Study of Latent Diffusion Models for Physics Emulation
Paper • 2507.02608 • Published • 22 -
Physics of Language Models: Part 4.1, Architecture Design and the Magic of Canon Layers
Paper • 2512.17351 • Published • 29 -
RynnBrain: Open Embodied Foundation Models
Paper • 2602.14979 • Published • 46
Vision Language Action models
-
A Survey on Vision-Language-Action Models: An Action Tokenization Perspective
Paper • 2507.01925 • Published • 39 -
Zebra-CoT: A Dataset for Interleaved Vision Language Reasoning
Paper • 2507.16746 • Published • 36 -
MolmoAct: Action Reasoning Models that can Reason in Space
Paper • 2508.07917 • Published • 45 -
Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policies
Paper • 2508.20072 • Published • 32
World model
-
WorldVLA: Towards Autoregressive Action World Model
Paper • 2506.21539 • Published • 40 -
LatticeWorld: A Multimodal Large Language Model-Empowered Framework for Interactive Complex World Generation
Paper • 2509.05263 • Published • 11 -
VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified Rewards in World Simulators
Paper • 2510.00406 • Published • 68 -
GigaBrain-0: A World Model-Powered Vision-Language-Action Model
Paper • 2510.19430 • Published • 54
Compression
-
Outlier-Safe Pre-Training for Robust 4-Bit Quantization of Large Language Models
Paper • 2506.19697 • Published • 45 -
Winning the Pruning Gamble: A Unified Approach to Joint Sample and Token Pruning for Efficient Supervised Fine-Tuning
Paper • 2509.23873 • Published • 68 -
Efficient Multi-modal Large Language Models via Progressive Consistency Distillation
Paper • 2510.00515 • Published • 41 -
SINQ: Sinkhorn-Normalized Quantization for Calibration-Free Low-Precision LLM Weights
Paper • 2509.22944 • Published • 80
Process Reward Modelling
-
ReasonFlux-PRM: Trajectory-Aware PRMs for Long Chain-of-Thought Reasoning in LLMs
Paper • 2506.18896 • Published • 29 -
Web-Shepherd: Advancing PRMs for Reinforcing Web Agents
Paper • 2505.15277 • Published • 105 -
PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Models
Paper • 2501.03124 • Published • 14 -
Is PRM Necessary? Problem-Solving RL Implicitly Induces PRM Capability in LLMs
Paper • 2505.11227 • Published
SAE
-
Resa: Transparent Reasoning Models via SAEs
Paper • 2506.09967 • Published • 22 -
Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2
Paper • 2408.05147 • Published • 42 -
Train Sparse Autoencoders Efficiently by Utilizing Features Correlation
Paper • 2505.22255 • Published • 24 -
I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders
Paper • 2503.18878 • Published • 121
Theory and Representation learning
-
I-Con: A Unifying Framework for Representation Learning
Paper • 2504.16929 • Published • 31 -
SONAR-LLM: Autoregressive Transformer that Thinks in Sentence Embeddings and Speaks in Tokens
Paper • 2508.05305 • Published • 49 -
The Strong Lottery Ticket Hypothesis for Multi-Head Attention Mechanisms
Paper • 2511.04217 • Published • 17 -
Large Language Models as Markov Chains
Paper • 2410.02724 • Published • 33
Graph
-
NodeRAG: Structuring Graph-based RAG with Heterogeneous Nodes
Paper • 2504.11544 • Published • 44 -
On the Origin of LLMs: An Evolutionary Tree and Graph for 15,821 Large Language Models
Paper • 2307.09793 • Published • 48 -
GraphOmni: A Comprehensive and Extendable Benchmark Framework for Large Language Models on Graph-theoretic Tasks
Paper • 2504.12764 • Published • 42 -
Code Graph Model (CGM): A Graph-Integrated Large Language Model for Repository-Level Software Engineering Tasks
Paper • 2505.16901 • Published • 48
Search
-
Open Deep Search: Democratizing Search with Open-source Reasoning Agents
Paper • 2503.20201 • Published • 48 -
ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning
Paper • 2503.19470 • Published • 19 -
Spacer: Towards Engineered Scientific Inspiration
Paper • 2508.17661 • Published • 32 -
DeepResearch Arena: The First Exam of LLMs' Research Abilities via Seminar-Grounded Tasks
Paper • 2509.01396 • Published • 58
Diversity
-
Modifying Large Language Model Post-Training for Diverse Creative Writing
Paper • 2503.17126 • Published • 36 -
Guided Self-Evolving LLMs with Minimal Human Supervision
Paper • 2512.02472 • Published • 55 -
Scaling Text-to-Image Diffusion Transformers with Representation Autoencoders
Paper • 2601.16208 • Published • 55 -
Diversity-Preserved Distribution Matching Distillation for Fast Visual Synthesis
Paper • 2602.03139 • Published • 45
Self correction
-
Self-Taught Self-Correction for Small Language Models
Paper • 2503.08681 • Published • 16 -
Learning from Failures: Correction-Oriented Policy Optimization with Verifiable Rewards
Paper • 2605.14539 • Published • 8 -
SEED: Self-Evolving On-Policy Distillation for Agentic Reinforcement Learning
Paper • 2607.14777 • Published • 93 -
Self-Improvements in Modern Agentic Systems: A Survey
Paper • 2607.13104 • Published • 29
Speech
-
Slamming: Training a Speech Language Model on One GPU in a Day
Paper • 2502.15814 • Published • 69 -
LLMVoX: Autoregressive Streaming Text-to-Speech Model for Any LLM
Paper • 2503.04724 • Published • 72 -
Audio-Aware Large Language Models as Judges for Speaking Styles
Paper • 2506.05984 • Published • 15 -
Optimizing Multilingual Text-To-Speech with Accents & Emotions
Paper • 2506.16310 • Published • 26
Synthetic data
-
Evaluating Language Models as Synthetic Data Generators
Paper • 2412.03679 • Published • 47 -
Smaller Language Models Are Better Instruction Evolvers
Paper • 2412.11231 • Published • 29 -
How to Synthesize Text Data without Model Collapse?
Paper • 2412.14689 • Published • 52 -
Open Data Synthesis For Deep Research
Paper • 2509.00375 • Published • 74
MoE
-
Your Mixture-of-Experts LLM Is Secretly an Embedding Model For Free
Paper • 2410.10814 • Published • 51 -
Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment
Paper • 2502.16894 • Published • 33 -
Ring-lite: Scalable Reasoning via C3PO-Stabilized Reinforcement Learning for LLMs
Paper • 2506.14731 • Published • 8 -
SlimMoE: Structured Compression of Large MoE Models via Expert Slimming and Distillation
Paper • 2506.18349 • Published • 13
Prompt papers
-
Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers
Paper • 2309.08532 • Published • 53 -
Evolving Prompts In-Context: An Open-ended, Self-replicating Perspective
Paper • 2506.17930 • Published • 19 -
No Prompt Left Behind: Exploiting Zero-Variance Prompts in LLM Reinforcement Learning via Entropy-Guided Advantage Shaping
Paper • 2509.21880 • Published • 54 -
p1: Better Prompt Optimization with Fewer Prompts
Paper • 2604.08801 • Published • 9
Sparsity
-
Sparse Autoencoders Find Highly Interpretable Features in Language Models
Paper • 2309.08600 • Published • 15 -
Learning to Skip the Middle Layers of Transformers
Paper • 2506.21103 • Published • 18 -
Alleviating Sparse Rewards by Modeling Step-Wise and Long-Term Sampling Effects in Flow-Based GRPO
Paper • 2602.06422 • Published • 47
State space LLM
Reasoning
-
Contrastive Decoding Improves Reasoning in Large Language Models
Paper • 2309.09117 • Published • 40 -
Prometheus: Inducing Fine-grained Evaluation Capability in Language Models
Paper • 2310.08491 • Published • 57 -
Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewarding
Paper • 2411.04282 • Published • 37 -
Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models
Paper • 2411.14432 • Published • 26
Fine tuning
-
When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method
Paper • 2402.17193 • Published • 27 -
What Happened in LLMs Layers when Trained for Fast vs. Slow Thinking: A Gradient Perspective
Paper • 2410.23743 • Published • 64 -
Direct Preference Optimization Using Sparse Feature-Level Constraints
Paper • 2411.07618 • Published • 17 -
Transformer^2: Self-adaptive LLMs
Paper • 2501.06252 • Published • 55
Dataset and Data processing
-
Perplexed by Perplexity: Perplexity-Based Data Pruning With Small Reference Models
Paper • 2405.20541 • Published • 24 -
RedPajama: an Open Dataset for Training Large Language Models
Paper • 2411.12372 • Published • 59 -
Exploring Data Scaling Trends and Effects in Reinforcement Learning from Human Feedback
Paper • 2503.22230 • Published • 45 -
DAComp: Benchmarking Data Agents across the Full Data Intelligence Lifecycle
Paper • 2512.04324 • Published • 160
Video understanding
-
Wolf: Captioning Everything with a World Summarization Framework
Paper • 2407.18908 • Published • 33 -
Mixture of Nested Experts: Adaptive Processing of Visual Tokens
Paper • 2407.19985 • Published • 37 -
TPDiff: Temporal Pyramid Video Diffusion Model
Paper • 2503.09566 • Published • 45 -
DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware Regressive GRPO
Paper • 2506.07464 • Published • 14
Long context
-
Writing in the Margins: Better Inference Pattern for Long Context Retrieval
Paper • 2408.14906 • Published • 144 -
DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
Paper • 2410.10819 • Published • 7 -
LLMtimesMapReduce: Simplified Long-Sequence Processing using Large Language Models
Paper • 2410.09342 • Published • 39 -
PDFTriage: Question Answering over Long, Structured Documents
Paper • 2309.08872 • Published • 55
Skill
Distillation
-
MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training
Paper • 2606.30406 • Published • 15 -
TurnOPD: Making On-Policy Distillation Turn-Aware for Efficient Long-Horizon Agent Training
Paper • 2607.05804 • Published • 18 -
Trust Region Policy Distillation
Paper • 2607.04751 • Published • 34 -
SEED: Self-Evolving On-Policy Distillation for Agentic Reinforcement Learning
Paper • 2607.14777 • Published • 93
Robotics
Causal
-
LLM Explainability with Counterfactual Chains and Causal Graphs
Paper • 2606.05972 • Published • 18 -
CausalMix: Data Mixture as Causal Inference for Language Model Training
Paper • 2607.01104 • Published • 21 -
CausaLab: A Scalable Environment for Interactive Causal Discovery Toward AI Scientists
Paper • 2605.26029 • Published • 18
Meta learning and Evolution
-
EvoMaster: A Foundational Agent Framework for Building Evolving Autonomous Scientific Agents at Scale
Paper • 2604.17406 • Published • 6 -
Solvita: Enhancing Large Language Models for Competitive Programming via Agentic Evolution
Paper • 2605.15301 • Published • 22 -
Learning to Foresee: Unveiling the Unlocking Efficiency of On-Policy Distillation
Paper • 2605.11739 • Published • 61 -
SkillsVote: Lifecycle Governance of Agent Skills from Collection, Recommendation to Evolution
Paper • 2605.18401 • Published • 130
Audio
-
Audio Flamingo Next: Next-Generation Open Audio-Language Models for Speech, Sound, and Music
Paper • 2604.10905 • Published • 29 -
Native Audio-Visual Alignment for Generation
Paper • 2605.30073 • Published • 33 -
UnityShots: Memory-Driven Multi-Shot Audio-Video Generation with Boundary-Aware Gating
Paper • 2606.21661 • Published • 28 -
WanSong v1.0 Technical Report
Paper • 2607.14749 • Published • 17
RAI
Medical and Biology
Continual Learning
-
Online Experiential Learning for Language Models
Paper • 2603.16856 • Published • 60 -
On Token's Dilemma: Dynamic MoE with Drift-Aware Token Assignment for Continual Learning of Large Vision Language Models
Paper • 2603.27481 • Published • 35 -
Rethinking Continual Experience Internalization for Self-Evolving LLM Agents
Paper • 2606.04703 • Published • 26
Inference
-
DualPath: Breaking the Storage Bandwidth Bottleneck in Agentic LLM Inference
Paper • 2602.21548 • Published • 56 -
TAPS: Task Aware Proposal Distributions for Speculative Sampling
Paper • 2603.27027 • Published • 147 -
Draft Less, Retrieve More: Hybrid Tree Construction for Speculative Decoding
Paper • 2605.20104 • Published • 7 -
Domino: Decoupling Causal Modeling from Autoregressive Drafting in Speculative Decoding
Paper • 2605.29707 • Published • 152
Embedding
-
Scaling Embeddings Outperforms Scaling Experts in Language Models
Paper • 2601.21204 • Published • 105 -
jina-embeddings-v5-text: Task-Targeted Embedding Distillation
Paper • 2602.15547 • Published • 31 -
ManCAR: Manifold-Constrained Latent Reasoning with Adaptive Test-Time Computation for Sequential Recommendation
Paper • 2602.20093 • Published • 29 -
Distribution-Conditioned Transport
Paper • 2603.04736 • Published • 3
Optimization
-
Learnable Multipliers: Freeing the Scale of Language Model Matrix Layers
Paper • 2601.04890 • Published • 44 -
MSign: An Optimizer Preventing Training Instability in Large Language Models via Stable Rank Restoration
Paper • 2602.01734 • Published • 34 -
On Surprising Effectiveness of Masking Updates in Adaptive Optimizers
Paper • 2602.15322 • Published • 11 -
Flash-KMeans: Fast and Memory-Efficient Exact K-Means
Paper • 2603.09229 • Published • 85
OCR
-
HunyuanOCR Technical Report
Paper • 2511.19575 • Published • 23 -
DeepSeek-OCR 2: Visual Causal Flow
Paper • 2601.20552 • Published • 73 -
CodeOCR: On the Effectiveness of Vision Language Models in Code Understanding
Paper • 2602.01785 • Published • 97 -
MinerU-Diffusion: Rethinking Document OCR as Inverse Rendering via Diffusion Decoding
Paper • 2603.22458 • Published • 139
Model Arithmetic
-
Souper-Model: How Simple Arithmetic Unlocks State-of-the-Art LLM Performance
Paper • 2511.13254 • Published • 140 -
Token-Level LLM Collaboration via FusionRoute
Paper • 2601.05106 • Published • 40 -
Darwin Family: MRI-Trust-Weighted Evolutionary Merging for Training-Free Scaling of Language-Model Reasoning
Paper • 2605.14386 • Published • 67
Tokenization methods and language modelling
-
Continuous Autoregressive Language Models
Paper • 2510.27688 • Published • 74 -
Efficient Speech Language Modeling via Energy Distance in Continuous Latent Space
Paper • 2505.13181 • Published • 9 -
Long-Context Autoregressive Video Modeling with Next-Frame Prediction
Paper • 2503.19325 • Published • 73 -
Bridging Continuous and Discrete Tokens for Autoregressive Visual Generation
Paper • 2503.16430 • Published • 34
Tabular ML
Tool
-
Provable Benefits of In-Tool Learning for Large Language Models
Paper • 2508.20755 • Published • 11 -
SimpleTIR: End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoning
Paper • 2509.02479 • Published • 84 -
How Can Input Reformulation Improve Tool Usage Accuracy in a Complex Dynamic Environment? A Study on τ-bench
Paper • 2508.20931 • Published • 16 -
THOR: Tool-Integrated Hierarchical Optimization via RL for Mathematical Reasoning
Paper • 2509.13761 • Published • 16
Foundation Models
-
GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models
Paper • 2508.06471 • Published • 213 -
Cognitive Kernel-Pro: A Framework for Deep Research Agents and Agent Foundation Models Training
Paper • 2508.00414 • Published • 96 -
Continuous Autoregressive Language Models
Paper • 2510.27688 • Published • 74 -
MiMo-Embodied: X-Embodied Foundation Model Technical Report
Paper • 2511.16518 • Published • 26
LLM judge
-
One Token to Fool LLM-as-a-Judge
Paper • 2507.08794 • Published • 32 -
Self-Improving VLM Judges Without Human Annotations
Paper • 2512.05145 • Published • 21 -
RubricBench: Aligning Model-Generated Rubrics with Human Standards
Paper • 2603.01562 • Published • 64 -
Xpertbench: Expert Level Tasks with Rubrics-Based Evaluation
Paper • 2604.02368 • Published • 12
Test time
-
Skip a Layer or Loop it? Test-Time Depth Adaptation of Pretrained LLMs
Paper • 2507.07996 • Published • 36 -
Test-Time Scaling with Reflective Generative Model
Paper • 2507.01951 • Published • 108 -
Does More Inference-Time Compute Really Help Robustness?
Paper • 2507.15974 • Published • 7 -
TTCS: Test-Time Curriculum Synthesis for Self-Evolving
Paper • 2601.22628 • Published • 35
3D
-
Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modeling
Paper • 2507.07982 • Published • 34 -
MeshLLM: Empowering Large Language Models to Progressively Understand and Generate 3D Mesh
Paper • 2508.01242 • Published • 11 -
Loc3R-VLM: Language-based Localization and 3D Reasoning with Vision-Language Models
Paper • 2603.18002 • Published • 15 -
Generation Models Know Space: Unleashing Implicit 3D Priors for Scene Understanding
Paper • 2603.19235 • Published • 95
Physics and operators
-
Lost in Latent Space: An Empirical Study of Latent Diffusion Models for Physics Emulation
Paper • 2507.02608 • Published • 22 -
Physics of Language Models: Part 4.1, Architecture Design and the Magic of Canon Layers
Paper • 2512.17351 • Published • 29 -
RynnBrain: Open Embodied Foundation Models
Paper • 2602.14979 • Published • 46
Materials and structures
Vision Language Action models
-
A Survey on Vision-Language-Action Models: An Action Tokenization Perspective
Paper • 2507.01925 • Published • 39 -
Zebra-CoT: A Dataset for Interleaved Vision Language Reasoning
Paper • 2507.16746 • Published • 36 -
MolmoAct: Action Reasoning Models that can Reason in Space
Paper • 2508.07917 • Published • 45 -
Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policies
Paper • 2508.20072 • Published • 32
Vision
-
MiCo: Multi-image Contrast for Reinforcement Visual Reasoning
Paper • 2506.22434 • Published • 10 -
VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learning
Paper • 2507.13348 • Published • 80 -
RewardDance: Reward Scaling in Visual Generation
Paper • 2509.08826 • Published • 73 -
Grasp Any Region: Towards Precise, Contextual Pixel Understanding for Multimodal LLMs
Paper • 2510.18876 • Published • 37
World model
-
WorldVLA: Towards Autoregressive Action World Model
Paper • 2506.21539 • Published • 40 -
LatticeWorld: A Multimodal Large Language Model-Empowered Framework for Interactive Complex World Generation
Paper • 2509.05263 • Published • 11 -
VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified Rewards in World Simulators
Paper • 2510.00406 • Published • 68 -
GigaBrain-0: A World Model-Powered Vision-Language-Action Model
Paper • 2510.19430 • Published • 54
Code
-
The Debugging Decay Index: Rethinking Debugging Strategies for Code LLMs
Paper • 2506.18403 • Published • 3 -
ReCode: Updating Code API Knowledge with Reinforcement Learning
Paper • 2506.20495 • Published • 10 -
SWE-Debate: Competitive Multi-Agent Debate for Software Issue Resolution
Paper • 2507.23348 • Published • 12 -
LoCoBench: A Benchmark for Long-Context Large Language Models in Complex Software Engineering
Paper • 2509.09614 • Published • 7
Compression
-
Outlier-Safe Pre-Training for Robust 4-Bit Quantization of Large Language Models
Paper • 2506.19697 • Published • 45 -
Winning the Pruning Gamble: A Unified Approach to Joint Sample and Token Pruning for Efficient Supervised Fine-Tuning
Paper • 2509.23873 • Published • 68 -
Efficient Multi-modal Large Language Models via Progressive Consistency Distillation
Paper • 2510.00515 • Published • 41 -
SINQ: Sinkhorn-Normalized Quantization for Calibration-Free Low-Precision LLM Weights
Paper • 2509.22944 • Published • 80
Data
-
Skywork-SWE: Unveiling Data Scaling Laws for Software Engineering in LLMs
Paper • 2506.19290 • Published • 53 -
Data Efficacy for Language Model Training
Paper • 2506.21545 • Published • 11 -
Easy Dataset: A Unified and Extensible Framework for Synthesizing LLM Fine-Tuning Data from Unstructured Documents
Paper • 2507.04009 • Published • 55 -
RefineX: Learning to Refine Pre-training Data at Scale from Expert-Guided Programs
Paper • 2507.03253 • Published • 19
Process Reward Modelling
-
ReasonFlux-PRM: Trajectory-Aware PRMs for Long Chain-of-Thought Reasoning in LLMs
Paper • 2506.18896 • Published • 29 -
Web-Shepherd: Advancing PRMs for Reinforcing Web Agents
Paper • 2505.15277 • Published • 105 -
PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Models
Paper • 2501.03124 • Published • 14 -
Is PRM Necessary? Problem-Solving RL Implicitly Induces PRM Capability in LLMs
Paper • 2505.11227 • Published
Memory
-
Xolver: Multi-Agent Reasoning with Holistic Experience Learning Just Like an Olympiad Team
Paper • 2506.14234 • Published • 41 -
MoTE: Mixture of Ternary Experts for Memory-efficient Large Multimodal Models
Paper • 2506.14435 • Published • 7 -
Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
Paper • 2504.19413 • Published • 67 -
MemOS: A Memory OS for AI System
Paper • 2507.03724 • Published • 168
SAE
-
Resa: Transparent Reasoning Models via SAEs
Paper • 2506.09967 • Published • 22 -
Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2
Paper • 2408.05147 • Published • 42 -
Train Sparse Autoencoders Efficiently by Utilizing Features Correlation
Paper • 2505.22255 • Published • 24 -
I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders
Paper • 2503.18878 • Published • 121
Applications and Uses
-
ComfyUI-R1: Exploring Reasoning Models for Workflow Generation
Paper • 2506.09790 • Published • 53 -
Saffron-1: Towards an Inference Scaling Paradigm for LLM Safety Assurance
Paper • 2506.06444 • Published • 73 -
DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents
Paper • 2506.11763 • Published • 74 -
Agentic Reasoning: Reasoning LLMs with Tools for the Deep Research
Paper • 2502.04644 • Published • 4
Theory and Representation learning
-
I-Con: A Unifying Framework for Representation Learning
Paper • 2504.16929 • Published • 31 -
SONAR-LLM: Autoregressive Transformer that Thinks in Sentence Embeddings and Speaks in Tokens
Paper • 2508.05305 • Published • 49 -
The Strong Lottery Ticket Hypothesis for Multi-Head Attention Mechanisms
Paper • 2511.04217 • Published • 17 -
Large Language Models as Markov Chains
Paper • 2410.02724 • Published • 33
Adversarial
Graph
-
NodeRAG: Structuring Graph-based RAG with Heterogeneous Nodes
Paper • 2504.11544 • Published • 44 -
On the Origin of LLMs: An Evolutionary Tree and Graph for 15,821 Large Language Models
Paper • 2307.09793 • Published • 48 -
GraphOmni: A Comprehensive and Extendable Benchmark Framework for Large Language Models on Graph-theoretic Tasks
Paper • 2504.12764 • Published • 42 -
Code Graph Model (CGM): A Graph-Integrated Large Language Model for Repository-Level Software Engineering Tasks
Paper • 2505.16901 • Published • 48
Multimodal
-
Qwen2.5-Omni Technical Report
Paper • 2503.20215 • Published • 174 -
Unsupervised Post-Training for Multi-Modal LLM Reasoning via GRPO
Paper • 2505.22453 • Published • 46 -
UniRL: Self-Improving Unified Multimodal Models via Supervised and Reinforcement Learning
Paper • 2505.23380 • Published • 22 -
More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models
Paper • 2505.21523 • Published • 13
Search
-
Open Deep Search: Democratizing Search with Open-source Reasoning Agents
Paper • 2503.20201 • Published • 48 -
ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning
Paper • 2503.19470 • Published • 19 -
Spacer: Towards Engineered Scientific Inspiration
Paper • 2508.17661 • Published • 32 -
DeepResearch Arena: The First Exam of LLMs' Research Abilities via Seminar-Grounded Tasks
Paper • 2509.01396 • Published • 58
Interpretable
-
I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders
Paper • 2503.18878 • Published • 121 -
Large Language Models are Locally Linear Mappings
Paper • 2505.24293 • Published • 14 -
Thought Anchors: Which LLM Reasoning Steps Matter?
Paper • 2506.19143 • Published • 14 -
Steered LLM Activations are Non-Surjective
Paper • 2604.09839 • Published • 15
Diversity
-
Modifying Large Language Model Post-Training for Diverse Creative Writing
Paper • 2503.17126 • Published • 36 -
Guided Self-Evolving LLMs with Minimal Human Supervision
Paper • 2512.02472 • Published • 55 -
Scaling Text-to-Image Diffusion Transformers with Representation Autoencoders
Paper • 2601.16208 • Published • 55 -
Diversity-Preserved Distribution Matching Distillation for Fast Visual Synthesis
Paper • 2602.03139 • Published • 45
Diffusion
-
Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models
Paper • 2503.09573 • Published • 77 -
Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective
Paper • 2505.15045 • Published • 56 -
Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding
Paper • 2505.16990 • Published • 22 -
D-AR: Diffusion via Autoregressive Models
Paper • 2505.23660 • Published • 34
Self correction
-
Self-Taught Self-Correction for Small Language Models
Paper • 2503.08681 • Published • 16 -
Learning from Failures: Correction-Oriented Policy Optimization with Verifiable Rewards
Paper • 2605.14539 • Published • 8 -
SEED: Self-Evolving On-Policy Distillation for Agentic Reinforcement Learning
Paper • 2607.14777 • Published • 93 -
Self-Improvements in Modern Agentic Systems: A Survey
Paper • 2607.13104 • Published • 29
Information_retrieval
-
Rank1: Test-Time Compute for Reranking in Information Retrieval
Paper • 2502.18418 • Published • 29 -
Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
Paper • 2503.09516 • Published • 41 -
Fixing Data That Hurts Performance: Cascading LLMs to Relabel Hard Negatives for Robust Information Retrieval
Paper • 2505.16967 • Published • 24 -
SitEmb-v1.5: Improved Context-Aware Dense Retrieval for Semantic Association and Long Story Comprehension
Paper • 2508.01959 • Published • 61
Speech
-
Slamming: Training a Speech Language Model on One GPU in a Day
Paper • 2502.15814 • Published • 69 -
LLMVoX: Autoregressive Streaming Text-to-Speech Model for Any LLM
Paper • 2503.04724 • Published • 72 -
Audio-Aware Large Language Models as Judges for Speaking Styles
Paper • 2506.05984 • Published • 15 -
Optimizing Multilingual Text-To-Speech with Accents & Emotions
Paper • 2506.16310 • Published • 26
Attention
-
MiniMax-01: Scaling Foundation Models with Lightning Attention
Paper • 2501.08313 • Published • 304 -
Lizard: An Efficient Linearization Framework for Large Language Models
Paper • 2507.09025 • Published • 19 -
On the Expressiveness of Softmax Attention: A Recurrent Neural Network Perspective
Paper • 2507.23632 • Published • 6 -
Causal Attention with Lookahead Keys
Paper • 2509.07301 • Published • 21
Synthetic data
-
Evaluating Language Models as Synthetic Data Generators
Paper • 2412.03679 • Published • 47 -
Smaller Language Models Are Better Instruction Evolvers
Paper • 2412.11231 • Published • 29 -
How to Synthesize Text Data without Model Collapse?
Paper • 2412.14689 • Published • 52 -
Open Data Synthesis For Deep Research
Paper • 2509.00375 • Published • 74
Agent
-
Large Language Models Orchestrating Structured Reasoning Achieve Kaggle Grandmaster Level
Paper • 2411.03562 • Published • 70 -
Training Language Models for Social Deduction with Multi-Agent Reinforcement Learning
Paper • 2502.06060 • Published • 37 -
MLGym: A New Framework and Benchmark for Advancing AI Research Agents
Paper • 2502.14499 • Published • 196 -
SurveyX: Academic Survey Automation via Large Language Models
Paper • 2502.14776 • Published • 100
MoE
-
Your Mixture-of-Experts LLM Is Secretly an Embedding Model For Free
Paper • 2410.10814 • Published • 51 -
Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment
Paper • 2502.16894 • Published • 33 -
Ring-lite: Scalable Reasoning via C3PO-Stabilized Reinforcement Learning for LLMs
Paper • 2506.14731 • Published • 8 -
SlimMoE: Structured Compression of Large MoE Models via Expert Slimming and Distillation
Paper • 2506.18349 • Published • 13
RAG
-
StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurization
Paper • 2410.08815 • Published • 47 -
SKETCH: Structured Knowledge Enhanced Text Comprehension for Holistic Retrieval
Paper • 2412.15443 • Published • 10 -
RAG-Star: Enhancing Deliberative Reasoning with Retrieval Augmented Verification and Refinement
Paper • 2412.12881 • Published • 2 -
AR-RAG: Autoregressive Retrieval Augmentation for Image Generation
Paper • 2506.06962 • Published • 28
Prompt papers
-
Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers
Paper • 2309.08532 • Published • 53 -
Evolving Prompts In-Context: An Open-ended, Self-replicating Perspective
Paper • 2506.17930 • Published • 19 -
No Prompt Left Behind: Exploiting Zero-Variance Prompts in LLM Reinforcement Learning via Entropy-Guided Advantage Shaping
Paper • 2509.21880 • Published • 54 -
p1: Better Prompt Optimization with Fewer Prompts
Paper • 2604.08801 • Published • 9
Planning
-
Compositional Foundation Models for Hierarchical Planning
Paper • 2309.08587 • Published • 11 -
ALPINE: Unveiling the Planning Capability of Autoregressive Learning in Language Models
Paper • 2405.09220 • Published • 27 -
WALL-E 2.0: World Alignment by NeuroSymbolic Learning improves World Model-based LLM Agents
Paper • 2504.15785 • Published • 22 -
CODA: Coordinating the Cerebrum and Cerebellum for a Dual-Brain Computer Use Agent with Decoupled Reinforcement Learning
Paper • 2508.20096 • Published • 37
Sparsity
-
Sparse Autoencoders Find Highly Interpretable Features in Language Models
Paper • 2309.08600 • Published • 15 -
Learning to Skip the Middle Layers of Transformers
Paper • 2506.21103 • Published • 18 -
Alleviating Sparse Rewards by Modeling Step-Wise and Long-Term Sampling Effects in Flow-Based GRPO
Paper • 2602.06422 • Published • 47
Multilingual
-
CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages
Paper • 2309.09400 • Published • 87 -
Tuning LLMs with Contrastive Alignment Instructions for Machine Translation in Unseen, Low-resource Languages
Paper • 2401.05811 • Published • 9 -
Is Preference Alignment Always the Best Option to Enhance LLM-Based Translation? An Empirical Analysis
Paper • 2409.20059 • Published • 16 -
Are Character-level Translations Worth the Wait? Comparing Character- and Subword-level Models for Machine Translation
Paper • 2302.14220 • Published
State space LLM
Partial layer training LLMs
-
Sorted LLaMA: Unlocking the Potential of Intermediate Layers of Large Language Models for Dynamic Inference Using Sorted Fine-Tuning (SoFT)
Paper • 2309.08968 • Published • 24 -
GraLoRA: Granular Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
Paper • 2505.20355 • Published • 37 -
Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding
Paper • 2505.22618 • Published • 47 -
Latent Zoning Network: A Unified Principle for Generative Modeling, Representation Learning, and Classification
Paper • 2509.15591 • Published • 46
Reasoning
-
Contrastive Decoding Improves Reasoning in Large Language Models
Paper • 2309.09117 • Published • 40 -
Prometheus: Inducing Fine-grained Evaluation Capability in Language Models
Paper • 2310.08491 • Published • 57 -
Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewarding
Paper • 2411.04282 • Published • 37 -
Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models
Paper • 2411.14432 • Published • 26
Evaluation
-
GPT-Fathom: Benchmarking Large Language Models to Decipher the Evolutionary Path towards GPT-4 and Beyond
Paper • 2309.16583 • Published • 13 -
Prometheus: Inducing Fine-grained Evaluation Capability in Language Models
Paper • 2310.08491 • Published • 57 -
SO-Bench: A Structural Output Evaluation of Multimodal LLMs
Paper • 2511.21750 • Published • 6 -
LLM Swiss Round: Aggregating Multi-Benchmark Performance via Competitive Swiss-System Dynamics
Paper • 2512.21010 • Published • 4
Fine tuning
-
When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method
Paper • 2402.17193 • Published • 27 -
What Happened in LLMs Layers when Trained for Fast vs. Slow Thinking: A Gradient Perspective
Paper • 2410.23743 • Published • 64 -
Direct Preference Optimization Using Sparse Feature-Level Constraints
Paper • 2411.07618 • Published • 17 -
Transformer^2: Self-adaptive LLMs
Paper • 2501.06252 • Published • 55
Math
-
Transformers Can Do Arithmetic with the Right Embeddings
Paper • 2405.17399 • Published • 55 -
Solving Inequality Proofs with Large Language Models
Paper • 2506.07927 • Published • 20 -
Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoning
Paper • 2507.00432 • Published • 79 -
CriticLean: Critic-Guided Reinforcement Learning for Mathematical Formalization
Paper • 2507.06181 • Published • 45
Dataset and Data processing
-
Perplexed by Perplexity: Perplexity-Based Data Pruning With Small Reference Models
Paper • 2405.20541 • Published • 24 -
RedPajama: an Open Dataset for Training Large Language Models
Paper • 2411.12372 • Published • 59 -
Exploring Data Scaling Trends and Effects in Reinforcement Learning from Human Feedback
Paper • 2503.22230 • Published • 45 -
DAComp: Benchmarking Data Agents across the Full Data Intelligence Lifecycle
Paper • 2512.04324 • Published • 160
Style transfer
Video understanding
-
Wolf: Captioning Everything with a World Summarization Framework
Paper • 2407.18908 • Published • 33 -
Mixture of Nested Experts: Adaptive Processing of Visual Tokens
Paper • 2407.19985 • Published • 37 -
TPDiff: Temporal Pyramid Video Diffusion Model
Paper • 2503.09566 • Published • 45 -
DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware Regressive GRPO
Paper • 2506.07464 • Published • 14
Reinforcement learning
-
Diffusion Augmented Agents: A Framework for Efficient Exploration and Transfer Learning
Paper • 2407.20798 • Published • 24 -
Offline Reinforcement Learning for LLM Multi-Step Reasoning
Paper • 2412.16145 • Published • 38 -
REINFORCE++: A Simple and Efficient Approach for Aligning Large Language Models
Paper • 2501.03262 • Published • 103 -
SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution
Paper • 2502.18449 • Published • 75
Long context
-
Writing in the Margins: Better Inference Pattern for Long Context Retrieval
Paper • 2408.14906 • Published • 144 -
DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
Paper • 2410.10819 • Published • 7 -
LLMtimesMapReduce: Simplified Long-Sequence Processing using Large Language Models
Paper • 2410.09342 • Published • 39 -
PDFTriage: Question Answering over Long, Structured Documents
Paper • 2309.08872 • Published • 55
Knowledge
-
Leveraging Open Knowledge for Advancing Task Expertise in Large Language Models
Paper • 2408.15915 • Published • 19 -
ReLearn: Unlearning via Learning for Large Language Models
Paper • 2502.11190 • Published • 30 -
ReaRAG: Knowledge-guided Reasoning Enhances Factuality of Large Reasoning Models with Iterative Retrieval Augmented Generation
Paper • 2503.21729 • Published • 29 -
Recitation over Reasoning: How Cutting-Edge Language Models Can Fail on Elementary School-Level Reasoning Problems?
Paper • 2504.00509 • Published • 24