Choosing rules, generation or retrieval

Not every task involving text needs a generative model. Adding a quantity column or checking whether a field exists is a defined operation that a rule or function can perform. Rewriting long instructions for another audience while preserving their conditions may benefit from generation. Answering a question about changing internal documentation also requires finding the right evidence before writing. A specialist begins with task properties rather than a favorite tool and compares approaches that meet the requirements with manageable complexity. Complexity includes components, dependencies and places where meaning can be lost, not merely prompt length.

A rule-based approach specifies an explicit transformation from input to output. Examples include rejecting records without identifiers or calculating a charge using a fixed rate supplied in a training case. Its strength is an inspectable execution rule with directly testable boundaries. Its limitation is that a small rule may not capture varied natural language. Searching for “urgent” can miss a message describing stopped operations without that word. Rules are therefore not universally better. Ask whether important distinctions fit a small, stable set of conditions or would require an increasingly fragile list that is difficult to maintain.

Direct generation gives the model instructions and input and requests an output. It suits a bounded trial when the required material is already present and the result can be checked. Summarizing three paragraphs does not automatically require retrieval or a multi-step agent. First define what must be preserved and what may be shortened; then test cases containing negation, numbers and conditions. Human review may make the output useful, but review must not conceal a large correction burden. Record correction time and the kinds of changes needed when comparing alternative approaches. A polished first draft can still be an inefficient workflow.

Retrieval-augmented generation fetches relevant material from a source collection and supplies it as context for writing an answer. Lewis and colleagues described models combining learned memory with external retrieval. For practical work here, use the general idea of separating evidence selection from answer composition without claiming that every retrieval application implements that paper's architecture. Retrieval can help when knowledge sits outside the original request or requires updates managed through documents. It also adds questions: may this user access the document, is its version effective, was the exception-bearing passage retrieved, and does that passage actually support the answer?

Retrieval does not make an unreliable source reliable merely by fetching it. If a collection contains two conflicting procedure versions, a system may retrieve one or both without determining which governs. Establish version and access rules before evaluating answer quality. Conversely, no search result does not prove that a rule is absent from every document. The query may use wording poorly matched to the index. Distinguish “we found no evidence” from “we have evidence that the rule is absent.” Give a bounded answer when the task does not permit broader verification, and make the missing evidence visible to the next reviewer.

Some tasks need a fixed sequence: extract items, validate fields, and summarize accepted items. This is a defined workflow when its sequence and transition rules are specified beforehand. An agent that chooses tools and next steps dynamically adds flexibility as well as additional failure paths. Anthropic's agent-pattern article recommends starting with simple solutions and adding complexity when needed. Use that principle to justify a staged experiment, not to declare a particular vendor superior. Choosing something called an agent does not establish that a task needs dynamic decisions. State which decision cannot reasonably be fixed in advance before introducing that flexibility.

For practical comparison, ask five questions. Can the answer be computed or matched through rules? Is the necessary evidence in the input? Does the source change frequently? Does the output require linguistic variation or interpretation of diverse expressions? What happens if the result is wrong? Do not combine the answers into an unexplained magic score. Use them to justify a choice. Extracting an invoice number from a structured field may be direct. Extracting it from irregular messages may need another method. Sending a financial demand using that number introduces authority and checks that successful extraction alone does not establish.

Create a baseline: an existing or simple method performing the same task on the same data. Comparing a model that writes a paragraph with a program that returns a number says little about quality without a shared output definition. Agree on acceptance criteria, the unit of work and how exceptions count. A small trial can record correct records, records requiring correction, review time and total time. Few formatting errors do not imply few meaning errors. Keep quality, time and cost measures separate enough to show a tradeoff rather than hiding deterioration in an ambiguous average. A comparison should help someone decide, not merely produce an attractive score.

Define a stopping or fallback condition. If the required source is unavailable, request clarification or route to an appropriate reviewer. If output structure fails, do not pass the result to a consequential action. If a new solution shows no advantage over the baseline within the trial's scope, retaining the simpler method may be sensible. Choosing not to use generation for part of a process is a valid specialist decision when task evidence supports it. Explain that decision in terms of the result: what will be produced, how it will be checked and what happens when the evidence is insufficient. A clear fallback is part of the design, not an admission that design failed.