Article
Does Your LLM Know *When It's About to Be Wrong*?
ginigen-ai
โข โข 21ginigen-ai PRO
ginigen-ai
AI & ML interests
None yet
Recent Activity
upvoted an article about 4 hours ago
The Fast Gemma Challenge: our verified-SOTA recipe, in full reacted to SeaWolf-AI's post with ๐ฅ about 4 hours ago
We wrote up our run in The Fast Gemma Challenge โ as vidraft-darwin โ and wanted to share the recipe. ๐
https://huggingface.co/spaces/gemma-challenge/gemma-dashboard
Verified result: 510.58 TPS at PPL 2.3930 on a single A10G (fw188-ctk49-n64-patchbridge, re-run & VERIFIED). Honest note: on raw TPS there are faster runs (535+), but those went over the PPL bar and didn't verify โ what we're proud of is the fastest result that keeps quality.
The recipe is already open, so we explained each piece: sliding-window W188, CTK49 kernel tuning, noprecache (honest, verifiable measurement), and an N64 synthetic warmup bridge that shrinks the publicโprivate gap (~15 TPS), plus INT4 + MTP K=7 + CUDA-graph capture. One rule: only stack quality-neutral speedups.
Huge thanks to @firfir-cast, @gemma-slayer, @chiku-inu, @kenyan-duma, @dixie-flatline and everyone who shared their experiments. Full write-up
๐
https://huggingface.co/blog/FINAL-Bench/fast-gemma
updated a Space about 10 hours ago
ginigen-ai/Heatmap-LeaderboardOrganizations
None yet
published an article about 1 month ago