Article
The cheap oracle is lying to you: from GPU kernels to agent rewards
dipankarsarkar
• • 1Dipankar Sarkar PRO
dipankarsarkar
AI & ML interests
Building the AI-native stack. Agents as infrastructure, safety as architecture, performance as plumbing. I publish the receipts: papers, datasets, demos.
Recent Activity
upvoted a paper less than a minute ago
AISPA: User-Centric System Prompt Auditing for Large Language Model Applications repliedto Abhisek987's post 1 minute ago
New dataset: 24,922 real Python dependency compatibility experiments🧪
Every experiment installed a pair of Python packages into a fresh environment and tested three stages: resolution → import → runtime smoke test.
The finding that motivated this: resolvers only read published metadata. In this data, ~3,000 environments resolved successfully and still failed at import or smoke-test time. That gap is invisible to pip and uv, and this dataset makes it measurable.
What's inside:
- 21,490 development + 3,432 sealed validation experiments
- 41 unique packages, Python 3.8–3.14, Linux x86_64
- The 6 validation packages (boto3, botocore, s3transfer, celery, kombu, billiard) never appear in training — validation measures generalization to entirely unseen packages
- Full per-stage logs: exact commands, exit codes, stdout/stderr
- Normalized error text + Python exception types for every failure
- Wheel metadata with Python/ABI/platform tag compatibility decisions
- PEP 751 lockfile hashes
- 142 engineered features per experiment, ready for training
Outcome labels: pass, resolution_failure, import_failure, smoke_test_failure.
I used it to train two models for DepLab (a structured model + a fine-tuned ModernBERT on release notes). On the sealed validation set (entirely unseen packages) the structured model reached 90.0% accuracy and 100% precision when predicting failures.
CC BY 4.0. Load it directly:
```python
from datasets import load_dataset
ds = load_dataset("Abhisek987/deplab-dependency-compatibility", "experiments")
```
Dataset: https://huggingface.co/datasets/Abhisek987/deplab-dependency-compatibility
Zenodo DOI: https://doi.org/10.5281/zenodo.21729353
Code: https://github.com/Abhisek12378/DepLab
If you're working on ML for developer tools or software engineering research, I'd love to hear what you build with it. repliedto nwaughachukwuma's post 1 minute ago
Can a text-only model + a vision toolkit (mm-ctx) match a native vision model?
We benchmarked 4 setups on 23 multimodal tasks (image, video, audio, PDF):
• glm-5.2 (text-only) + mm-ctx: 88.4
• gemini-3.5-flash (vision): 83
• deepseek-v4-pro (text-only) + mm-ctx: 79.4
• qwen3.6-35b-a3b (vision): 44.3
The best text-only setup closes 89% of the gap to the top vision model.
Full report: https://huggingface.co/blog/vlm-run/text-only-models-with-mmOrganizations
published an article about 1 month ago