parallel-web/parallel-search-evals
Can I trust it?
Worth a closer look before you use it.
Warnings Check
Very new: it was created yesterday. Wait a little before installing it.
Stars from real accounts No data
We haven't checked its stars for fakes yet.
Active OK
Last change today.
Licence OK
MIT: you can use it for free, also in a commercial product. Keep the licence notice.
Quick checks from public data. They help, but they are no guarantee.
What it is
Python harness that evaluates language models with Parallel Search and Extract tools against a closed-book baseline, using DeepSearchQA and Humanity's Last Exam benchmarks with an OpenAI judge. Runs are set up via a CLI for one model, benchmark and mode at a time, with resume…
Who it's for
Researchers and developers benchmarking LLMs with web search tools on DeepSearchQA or HLE
How to use it
Install
uv sync --frozen --python 3.12From its README. Check it there before running it.
Ask your AI to set it up
Paste this into Claude, ChatGPT or your coding assistant.
Help me install and set up parallel-web/parallel-search-evals in my project. Follow its README: https://github.com/parallel-web/parallel-search-evals. Explain each step and check it is safe before running anything.
Star curve
Get alerts for repos like this
We'll email you when something in Models & inference starts rising. Free, once a week, only what matches.
What arrives: one email on Mondays with the repos that match, never more. See last week's. Privacy
For maintainers
Add this badge to your README. It shows the repo's place on the radar, updates every hour and links back to this page.
[](https://repoalerts.com/repo/parallel-web/parallel-search-evals?ref=badge)Similar repos
- awslabs/ai-eval-studio
A prototype end-to-end LLM evaluation platform. Users upload a CSV or Excel dataset, describe their goal in a chat with a coordinator agent, and the system runs evaluations across candidate models on
- antins-labs/VisSearch-Bench-v1
A benchmark of 204 human-written tasks for evaluating how multimodal agents keep track of visual evidence over long visual search sequences. Includes the dataset loader, reference answers, reference s
- apple-aiml-research/ml-rsi-inference-bench
A long-horizon benchmark for coding agents that improve the inference stack of large language models in vLLM or SGLang. Tasks cover supporting a model on day 0, optimizing it after launch, and repairi
- phonny40rowle/rag-search
A self-hosted FastAPI service for retrieval-augmented search. It combines web search via Serper, content extraction, embeddings stored in Zilliz, and answer generation through an OpenAI-compatible API
- Prophetlab/Investor-Last-Exam
A benchmark for evaluating AI agents on equity investment research using a time-frozen corpus, so models only see information available at a historical cutoff. It comprises 86 assignments covering res
- garrytan/g-harness-evals
A benchmark comparing AI agent harnesses (the system prompt, tools and memory wrapped around a model) on the same frontier model and tasks, measuring pass rate, cost per solved task and time. Reports