Quepid is an open source, engine-agnostic web app for test-driven search relevance tuning: rate query results, track scores over time, and collaborate with your team, against OpenSearch, Elasticsearch, Solr, Vectara, Algolia, Fusion, or any custom search API. It's free to try hosted, and free to self-host.
It's not the only tool in this space. Below is an honest comparison against the other active tools people evaluate alongside Quepid, followed by a few related projects that come up in searches but aren't really independent competitors.
| Tool | Form | Engine support | LLM as a Judge | Engagement Model |
|---|---|---|---|---|
| Quepid | Hosted SaaS + self-hosted web app | OpenSearch, Elasticsearch, Solr, Vectara, Algolia, Fusion, Vespa and any other HTTP API via custom mapper | Yes: AI Judge (OpenAI or self-hosted Ollama) | Open source; free hosted cloud option; consulting & training available (OpenSource Connections) |
| Search Relevance Workbench | Native OpenSearch Dashboards feature (3.1+) | OpenSearch only | Yes: built-in "LLM as a Judge" tool | Open source; self-hosted with your own OpenSearch cluster; AWS Managed OpenSearch, community support via forums/Slack |
| Elasticsearch Relevance Studio | Unofficial Elastic reference tool, self-hosted via Docker | Elasticsearch only | Yes, with a human review/adjust step: AI agents judge via MCP | Source-available (Elastic License 2.0, not OSI open source); self-hosted only, no dedicated paid tier |
| Rated Ranking Evaluator | Java library, with optional UI | Solr, Elasticsearch | No (human-managed ratings only) | Open source core; paid RRE Enterprise (contact Sease); consulting & training from Sease |
| Releval | Self-hosted (Docker) | Elasticsearch, OpenSearch, Solr, Vespa natively; any HTTP API or rendered search page generically | Yes: AI-powered judging, bring your own AI provider | Closed-source; free Individual tier (1 user, 1 endpoint); paid Team/Enterprise tiers (contact sales) |
| Rankquest Studio | Browser-only static app, no install | Any HTTP API returning a ranked list (built-in Elasticsearch client, demo configs for others) | No (AI only generates test case queries, not ratings) | Open source; free hosted instance available; independent consulting from the maintainer |
Built directly into OpenSearch Dashboards starting with OpenSearch 3.1, and co-developed by the OpenSearch project together with OpenSource Connections, the same team that builds Quepid. Judgments, query sets, and experiments live as OpenSearch indices, with no separate app to host. SRW can even import Quepid's CSV judgment exports directly, so a move from Quepid to SRW (or the reverse) isn't a rebuild from scratch.
Consider it if: you're already on OpenSearch and want relevance tooling built into your existing stack instead of a separate application.
Published by Elastic on GitHub as, in their own words, a "demonstrator / reference implementation," not an officially supported product. It's licensed under the Elastic License 2.0: source-available, with usage restrictions, and the same license Elastic itself moved to in 2021. It's not an OSI-approved open source license. The tool is Elasticsearch-specific and leans into agentic workflows via an MCP server for AI-driven relevance engineering.
Consider it if: you're Elasticsearch-only and want to try Elastic's own emerging approach, with the understanding that it's explicitly experimental and source-available rather than fully open source.
An open source Maven/JVM library from Sease for running relevance regression tests as part of a CI/CD pipeline, treating search quality like automated software testing. A paid "RRE Enterprise" tier adds a UI on top. It targets engineers building evaluation pipelines rather than analysts doing interactive, exploratory rating.
Consider it if: your team wants relevance checks embedded in the build pipeline as code, and you're comfortable in the JVM ecosystem.
A closed-source, self-hosted platform (first stable release July 2026) with native support for the UBI clickstream standard, meaning it can derive implicit judgments from real user behavior in addition to manual ratings. It ships with LLM-as-judge and MCP integrations built in. Pricing is tiered: a free "Individual" plan capped at one user and one search endpoint, with paid Team and Enterprise tiers (contact sales) for more users, endpoints, and CI/CD automation. It's new enough that there's no independent track record yet, and it requires accepting a EULA to run.
Consider it if: built-in clickstream-driven judgments matter more to you than open source, and you're comfortable adopting a brand-new commercial product.
A zero-install, browser-only tool: no server, no database, everything runs client-side, and configurations/test cases export as git-friendly JSON. It connects to any search API via a plugin model, similar in spirit to Quepid's engine-agnostic approach, and includes AI-assisted test case generation. Maintained by an individual developer (Jilles van Gurp) rather than a company.
Consider it if: you want the lowest-friction way to try relevance testing with nothing to deploy.
These come up in searches near Quepid, but aren't really independent competitors. Here's why, in their own words.
A single-query "explain" sandbox for Solr and Elasticsearch: paste in a query, see why results ranked the way they did. It's also from OpenSource Connections, and its own README says it plainly: Splainer "forms the core of the open source tool Quepid". It's a lighter-weight sibling tool for a narrower job.
A open source project whose own README describes it as "highly inspired by Quepid."
A Java-based relevance evaluation toolkit originally from MITRE. While no longer active developed, it has some interesting ideas. See the original and a successor fork.
Also from Sease, and actively maintained. It answers a narrower question than a general relevancy-tuning platform does: is your embedding model the problem, or your approximate nearest neighbor configuration? A useful companion to a broader tool like Quepid.
Free to try hosted, free to self-host, and built on open source software.
Free Service » Take a Tour »