Best Web Search API — category guide

What a web search API does for an app

A web search API gives a program a way to ask a web index a question and get machine-readable results back. A consumer search engine returns a page for a person to scan and click. An API returns fields, such as a URL, a title and text, that code can store, rank, filter or pass to a model. The developer decides what the user sees.

Providers differ in where their results come from. Some run their own index, some license results from another index, and some mix several sources. The source changes coverage, freshness and how often a page is missing, so it is worth asking before you commit.

Many buyers also ask whether they may keep results. Read the storage, caching and attribution terms, because some providers limit how long results can be kept or how they can be shown. For a product that stores results, those terms can matter as much as the price per call.

The four jobs under one label

The label covers several jobs that often overlap in one product. Most apps need one or two of them, so name the job first.

• Link lookup: a list of URLs and titles for a pipeline to process later.
• Passage retrieval: short excerpts that ground a model's reply in the source text.
• Page extraction: the full readable content of a page, for deeper reading.
• Cited answers: a short answer plus the quotes it rests on, so a user can check it.

An app that only needs links can use a cheaper, simpler call. An app that must show evidence to a user needs quotes or citations, and an app that reads many pages needs a budget for the text it pulls in. Cited answers also hide some of their reasoning, so an app that needs an audit trail should keep the raw URLs and quotes it received, not only the final text.

Explore by topic

Continue this reading path

• Best Web Search API
• Resource directory

Cite this entry