Latency
The time between sending a request and receiving the full response.
What it means
Latency is the time between sending a request and receiving the full response. For search APIs, it depends on how much the provider does per call, such as searching, reranking, fetching pages and building excerpts. Providers often publish a median and a tail figure, and the tail matters more for a live chat feature, because a few slow calls are what users notice.
Latency in an agent includes more than the search call. Each model turn and each extra search adds time, so a task that searches ten times can take much longer than one search. Measure end-to-end time for the whole task, not only the API call. Compare providers at the concurrency your app will use, since queued requests can add delay under load.
Continue this reading path
Cite this entry
https://bestwebsearchapi.com/glossary/latency/