Validation, Errors, and Fallbacks

The response is not the end of the program

When an LLM API returns a response, the application still has work to do.

It may need to check that the HTTP request succeeded, verify that the response contains the expected fields, verify that the generated content is usable, and decide what to do if any of those checks fail.

That logic is ordinary software engineering.

In the introductory LLM course, we could just focus on the prompt and the response. In an actual application, the response is just a part of a larger program. The rest of the program needs to decide whether that response is acceptable.

Validate what matters

Suppose that the program expects a plain text reply from a Responses API call:

const messageItem = (data.output ?? []).find((item) => item.type === "message");
const textPart = messageItem?.content?.find((part) => part.type === "output_text");
const content = textPart?.text;

if (typeof content !== "string" || content.trim().length === 0) {
  throw new Error("Model response did not contain usable text.");
}

If the program expects structured output, the validation should be stricter.

const isValidSummary = (value) => {
  return typeof value === "object" &&
    value !== null &&
    typeof value.summary === "string" &&
    Array.isArray(value.risks);
};

The general rule is simple: validate the parts of the output that later code depends on. If downstream logic needs summary to be a string and risks to be an array, those are exactly the assumptions that the validation layer should encode.

Errors happen in different places

There are at least three broad kinds of failure. The request itself may fail. The provider may return an unexpected response. Or the model may return content that does not satisfy the application needs.

These are different situations and should not all be handled the same way.

A network error suggests one kind of retry or user feedback. A malformed provider response suggests another. A structurally valid but poor-quality model answer may require a fallback or a request for user clarification. Treating all three cases as the same “something went wrong” error throws away useful information.

Retries and fallbacks

Sometimes retrying is reasonable, especially for temporary request failures. Sometimes retrying only wastes time and tokens. The application needs a clear policy.

Figure 1 shows a simple flow for request handling.

Fig 1. — A small application should still distinguish successful responses, invalid responses, retries, and fallback behavior.

Possible fallback behaviors in a CLI application include printing a clear error message, returning a raw text answer instead of structured output, or asking the user to retry with a simpler request.

The important point is that fallback behavior should be designed in advance. If the application only decides what to do after a failure has already occurred, the resulting behavior is often inconsistent and hard to explain.

That also means the retry policy itself deserves review. A policy can sound helpful while still introducing silent failures or misleading defaults.

Good error messages help users and developers

An error message such as “Something went wrong” is rarely enough. A better CLI application should help the user understand whether the problem came from configuration, networking, or output validation.

This does not mean exposing every internal detail. It means being specific enough to guide the next step. If the API key is missing, say that the API key is missing. If the model returned malformed JSON, say that the response format was invalid. Specificity reduces debugging time and makes the application easier to trust.

The programming exercise for this chapter uses stubbed sequences of responses: an HTTP failure, a malformed model output, and a valid structured response. That setup makes the retry policy testable without contacting a live provider.

$ deno test