Qwen2.5-Binary — Protocol 0 SFT
20/20 refusals. 0/20 fabrications.
Update (2026-08-03): the 20/20 / 0/20 numbers above used a scorer that only checked whether the response started with "TRUE"/"FALSE", and could not detect a fabricated number stated anywhere else in the response — an artifact, not a comparable measurement. A v2 control run (30 tokens, one money-regex scorer applied identically to base and fine-tuned models) gives 15/20 refusals, 5/20 fabrications for this model. Raw results: binary_sft_k20_v2.json.
Qwen2.5-7B-Instruct fine-tuned on the Protocol 0 Binary dataset.
Same data, same 3 epochs, same binary format. Result: perfect abstention without a single fabricated number.
See Hermes-3-binary for full methodology.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
model = PeftModel.from_pretrained(base, "SoulInPsyAbstract/binary-qwen25-lora")
Part of SIPA OS
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support