Search Birds and Rerank them in Natural Language
Use Pinecone to search over a corpus of bird Wikipedia pages with full-text, then rerank the top 200 results down to 10 with TypeSafe.
First, a full-text query is sent to Pinecone. The Wikipedia articles are long, so full-text search can match a term anywhere in an article, across every field at once. Second, a TypeSafe call takes each result’s opening paragraph and grades it against the criteria above. That grading score is what reranks the candidate set. We can batch the grades because the results are independent and calibrated. No need to rescale afterwards.
We compare against a Claude baseline to showcase different kinds of natural-language reranking. The Claude baseline can reason between the corpus entries, but is more computationally expensive.
What each column is doing under the hood
Pinecone FTS (BM25)
One documents.search call. BM25 scores the full-text query against bird_name, intro and the body fields, and returns the top 200 of 2,155 articles. No embeddings are involved anywhere in this app.
Terms combine with OR semantics: an article matching any term participates, and rarer terms count for more. That is why this column has good recall and poor judgment — it ranks articles that use the words, which for a negation is close to the opposite of what was asked.
TypeSafe rerank
200 independent requests to the Jev model, 64 in flight at once — one per candidate. Each request carries only that bird’s name and snippet plus your criteria, and asks a single yes/no question. The answer comes back as a calibrated probability from 0 to 1, which the app sorts on directly.
Because no candidate ever sees another, the scores stay comparable across the whole set. When nothing matches, every probability is low and the column says so instead of inventing a top 10.
Claude rerank
One request containing all 200 snippets at once, returning the ten best ids as structured output. Unlike the per-candidate judgments, Claude sees the whole list and can reason across it — a genuine advantage the parallel approach gives up.
It pays for that with one long serial call at full token rates, which is what the timing and cost badges above each column are measuring.
The #N beside a reranked result is that bird’s original BM25 rank. A high number means the reranker pulled it up from deep in the retrieved set.