October 28 catch-up edition. First published November 2, 2026.
An assistant starts writing quickly, then spends a long time searching. Another stays silent before producing a complete answer. Which one is faster? The answer depends on what you measured and when the result became useful.
A single “response time” hides several different experiences. Before comparing models or changing a workflow, define the beginning, the useful milestones and the end of the task. This article proposes a measurement worksheet; it does not report a new performance benchmark.
Start with a user-visible timeline
For a document task, you might record when you submit the request, when the first useful answer content appears, when a tool starts and finishes, and when the final usable result arrives. An animated status indicator is not the same milestone as an answer you can read.
Record what you can actually observe. If you only have a browser stopwatch, you may know the total elapsed time without knowing the server’s queue time or model processing time. Leave those causes unknown instead of assigning an explanation to every pause.
Distinguish observed milestones from internal metrics
Hugging Face’s Text Generation Inference documentation lists separate metrics for request duration, queue duration, inference duration and mean time per generated token. That distinction is useful: a slow request does not, by itself, establish slow token generation.
Those named metrics belong to that system. A hosted chat product may expose different measurements or none at all. Do not infer its internal timings from a similarly named metric in another platform.
The W3C Server Timing working draft provides a way for servers to communicate performance metrics to a client. It does not mean every website reveals all processing stages. Instrumentation must exist, and its boundaries must be understood.
A fictional comparison with two clocks
Imagine two proposed test runs for the same task. In run A, useful text begins after 2 seconds and the final checked result arrives after 18 seconds. In run B, useful text begins after 6 seconds and the final result arrives after 11 seconds. These numbers are invented to illustrate the worksheet.
Run A begins usefully sooner; run B finishes sooner. Neither observation proves that its underlying model generates faster. The output lengths, tool activity and completion criteria also need recording. A quick partial answer may help a reader, but it is not interchangeable with a completed task.
Keep a measurement card for each run
| Field | What to record |
|---|---|
| Task and input version | The same bounded job and source material |
| Environment | Product or model version where known, relevant settings and date |
| First useful output | Your operational definition and observed elapsed time |
| Tool activity | Observed calls, waiting periods and failures |
| Usable completion | The moment the defined task is satisfied |
| Quality and repair | Errors, missing work and human correction time |
Compare like with like
Keep the requested result and source material consistent. If one run asks for a short extraction and another asks for an essay, their elapsed times describe different work. Save the outputs so a reviewer can see whether both met the same acceptance rule.
Repeat the task where appropriate and retain the slow or failed runs. A memorable best case is not a dependable service expectation. Note relevant changes, such as a different tool configuration, instead of silently combining them into one average.
For a workflow with simultaneous calls, do not simply add every call’s duration and label the sum “elapsed time.” Record overlapping activity on the timeline. The total wait experienced by a user and the cumulative work across components answer different questions.
Optimize the part you have evidence about
If a document lookup consumes most of the observed delay, investigate that lookup before replacing the model. If repeated repair dominates the task, faster initial output may have limited practical value. The whole-task productivity guide develops that broader accounting.
Keep content quality in view as well. The agent task scorecard can sit beside the timing worksheet. A useful comparison records both what finished and how long a person waited for a result they could trust.
AI-generated conceptual illustration. All numerical run examples are fictional; no latency experiment was performed.
