refactor(evals): consolidate runners under headroom/evals CLI hub
3f36c79
Tany Nguyencommited on
refactor: update swe_bench references to model_creds in evals files
c20e575
Tany Nguyencommited on
docs: update documentation to align with v0.21.x codebase
fa65fec
Tany Nguyencommited on
chore: add model_creds/ to .gitignore
a0dbb62
Tany Nguyencommited on
docs: update README.md to reflect model_creds directory rename
715ea6b
Tany Nguyencommited on
refactor: rename swe_bench/ to model_creds/ and update all references
28d24bc
Tany Nguyencommited on
fix(ccr): update defaults and docs for inject_retrieval_marker
910e81a
tudragon154203Claude Sonnet 4.6Happycommited on
feat(ccr): disable retrieval marker by default
3f1d055
tudragon154203Claude Sonnet 4.6Happycommited on
del deprecated evals
1340fe8
Tu Nguyencommited on
chore(transforms): remove twotrim files\n\n- Remove two_trim.py implementation file\n- Remove test_two_trim.py test file\n- Remove bench_two_trim.py benchmark file\n\nGenerated with [Claude Code](https://claude.ai/code)\nvia [Happy](https://happy.engineering)\n\nCo-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>\nCo-Authored-By: Happy <yesreply@happy.engineering>
c8d4763
tudragon154203commited on
feat(transforms): remove twotrim from pipeline\n\n- Remove TwoTrim transform implementation and configuration\n- Remove TwoTrim references from pipeline.py, config.py, __init__.py\n- Remove TwoTrim documentation from wiki\n- Remove test and benchmark files\n\nGenerated with [Claude Code](https://claude.ai/code)\nvia [Happy](https://happy.engineering)\n\nCo-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>\nCo-Authored-By: Happy <yesreply@happy.engineering>
f7fead9
tudragon154203commited on
feat(kompress): switch default to kompress-v2-base with weight-only int8 ONNX\n\n- Update HF_MODEL_ID and docstrings\n- Add HEADROOM_KOMPRESS_ONNX_FILENAME env var and fallback chain\n- Refactor ONNX loader to try candidates in order (int8-wo → fp32 → v1 int8)\n- Preserve existing CPU/GPU session options and logging\n\nCo-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>\nCo-Authored-By: Happy <yesreply@happy.engineering>
0f94236
tudragon154203commited on
fix(proxy): guard _hook_ctx in post_compress when hooks enabled in cache mode
4dccde9
tudragon154203ClaudeHappycommited on
merge dev: dockerignore Rust artifacts, cost tracker, image compressor close
6eb6419
tudragon154203ClaudeHappycommited on
chore: consolidate step 3.7 eval outputs into evals/
ef55e23
tudragon154203ClaudeHappycommited on
fix(evals): send compress_user_messages=true in BeforeAfterRunner
84a0b9e
tudragon154203ClaudeHappycommited on
docs(evals): record refreshed step 3.7 tier 2 results
b43fc86
tudragon154203ClaudeHappycommited on
fix(evals): send compress_user_messages=true to proxy in LiveCodeBenchRunner
e95dd6d
tudragon154203commited on
feat(evals): accept explicit env_file in SuiteRunner, honor OPENAI_MODEL in LiveCodeBench
8ee43f8
tudragon154203ClaudeHappycommited on
fix(evals): load repo envs and handle reasoning-only responses
453cf79
tudragon154203ClaudeHappycommited on
fix(evals): add 429 retry with exponential backoff, extract shared helper
c39f084
tudragon154203ClaudeHappycommited on
fix(evals): harden suite model selection and response extraction
ae4e2cf
tudragon154203ClaudeHappycommited on
fix(evals): strip trailing newline in LiveCodeBench stdout comparison
ae767be
tudragon154203Claude Sonnet 4.5commited on
fix(evals): extract OpenAI text util, harden proxy logging, support stdin JSON tests
d1c753f
tudragon154203Claude Sonnet 4.6commited on
fix(evals): bump default max_tokens to 2048 for NVIDIA NIM models
b72f0f6
tudragon154203ClaudeHappycommited on
fix(evals): harden LiveCodeBench runner semantics
4556dc4
tudragon154203ClaudeHappycommited on
fix(evals): harden benchmark runners and dataset loading
49457b8
tudragon154203ClaudeHappycommited on
feat(evals): add BFCL v2, LiveCodeBench, and Long-context QA benchmarks
3f8e7d9
tudragon154203ClaudeHappycommited on
fix(swe-bench): default max-workers to 1
9c58bf4
tudragon154203ClaudeHappycommited on
feat(swe-bench): add 25-instance benchmark subset, default to 1 worker