KRLabsOrg/lettucedetect-code-hallucination
Viewer • Updated • 74.3k • 439 • 2
AI & ML interests
Explainable AI, Rule-based models, Rule learning with LLMs, Hallucination detection, Fact checking LLMs
Recent Activity
View all activity
Papers
RuleChef: Grounding LLM Task Knowledge in Human-Editable Rules
ACL-Verbatim: hallucination-free question answering for research
adaamko
updated a
dataset about 12 hours ago
Post
88
🥬 LettuceDetect v2 — span-level hallucination detection for code, tool output, and structured documents.
Hallucination detectors are trained on document QA, but agents ground their answers in source code, tool output and markdown. On code-agent answers, existing detectors reach 0.17 span-F1 and even 550B zero-shot judges at most 0.22.
We built a unified span-level benchmark — 74,285 newly constructed examples (145K+ with RAGTruth and 14-language PsiloQA folded in), every span typed and character-labeled — and trained two detectors on it:
🤖 KRLabsOrg/lettucedect-v2-qwen-2b — generative, typed spans + explanations in one pass, 32K context, **0.689 span-F1** (0.60 on code-agent)
⚡ KRLabsOrg/lettucedect-v2-mmbert-base — 307M multilingual encoder for high-throughput setups
🏷️ KRLabsOrg/lettucedect-v2-taxonomy-head — types the spans of any binary detector
It also reaches the best reported English PsiloQA IoU (0.724) and 81.8 RAGTruth example-F1, so specializing on code didn't cost general RAG performance.
📚 Dataset: KRLabsOrg/lettucedetect-code-hallucination
📄 Paper: https://arxiv.org/abs/2607.00895
The models are now integrated natively into vLLM Semantic Router — joint blog post on how it works: https://vllm-sr.ai/blog/lettucedetect-v2-generative-hallucination-detection
Hallucination detectors are trained on document QA, but agents ground their answers in source code, tool output and markdown. On code-agent answers, existing detectors reach 0.17 span-F1 and even 550B zero-shot judges at most 0.22.
We built a unified span-level benchmark — 74,285 newly constructed examples (145K+ with RAGTruth and 14-language PsiloQA folded in), every span typed and character-labeled — and trained two detectors on it:
🤖 KRLabsOrg/lettucedect-v2-qwen-2b — generative, typed spans + explanations in one pass, 32K context, **0.689 span-F1** (0.60 on code-agent)
⚡ KRLabsOrg/lettucedect-v2-mmbert-base — 307M multilingual encoder for high-throughput setups
🏷️ KRLabsOrg/lettucedect-v2-taxonomy-head — types the spans of any binary detector
It also reaches the best reported English PsiloQA IoU (0.724) and 81.8 RAGTruth example-F1, so specializing on code didn't cost general RAG performance.
📚 Dataset: KRLabsOrg/lettucedetect-code-hallucination
📄 Paper: https://arxiv.org/abs/2607.00895
The models are now integrated natively into vLLM Semantic Router — joint blog post on how it works: https://vllm-sr.ai/blog/lettucedetect-v2-generative-hallucination-detection
- 3 replies
adaamko
updated a
collection 3 days ago
adaamko
updated a
model 4 days ago
adaamko
updated 12
models 17 days ago
KRLabsOrg/lettucedect-610m-eurobert-pl-v1
Token Classification • 0.6B • Updated • 81
KRLabsOrg/lettucedect-610m-eurobert-it-v1
Token Classification • 0.6B • Updated • 40
KRLabsOrg/lettucedect-610m-eurobert-fr-v1
Token Classification • 0.6B • Updated • 240 • 1
KRLabsOrg/lettucedect-610m-eurobert-es-v1
Token Classification • 0.6B • Updated • 22
KRLabsOrg/lettucedect-610m-eurobert-de-v1
Token Classification • 0.6B • Updated • 360 • 1
KRLabsOrg/lettucedect-610m-eurobert-cn-v1
Token Classification • 0.6B • Updated • 37 • 1
KRLabsOrg/lettucedect-210m-eurobert-pl-v1
Token Classification • 0.2B • Updated • 24
KRLabsOrg/lettucedect-210m-eurobert-it-v1
Token Classification • 0.2B • Updated • 23
KRLabsOrg/lettucedect-210m-eurobert-fr-v1
Token Classification • 0.2B • Updated • 92 • 1
KRLabsOrg/lettucedect-210m-eurobert-es-v1
Token Classification • 0.2B • Updated • 64
KRLabsOrg/lettucedect-210m-eurobert-de-v1
Token Classification • 0.2B • Updated • 115
KRLabsOrg/lettucedect-210m-eurobert-cn-v1
Token Classification • 0.2B • Updated • 57
adaamko
updated a
model about 1 month ago
adaamko
submitted a
paper to Daily Papers about 1 month ago
adaamko
updated a
collection about 1 month ago