Open models just reached frontier code review (Sponsored)
PR-AF is an open-source code review agent that ranks #2 of 42 on Martian's Code-Review-Bench, ahead of CodeRabbit, Copilot, and Devin, running a single open model. The trick is the harness: it plans a review strategy per PR, spawns parallel reviewer agents, verifies every finding against your source, and drops anything it cannot prove. That makes it about 10x cheaper per review than closed-source tools. One API call, and a drop-in GitHub Action. If open models writing frontier-level reviews sounds useful, star the repo. A chatbot answers a question, and an agent completes a task. This seems like a huge difference. However, the gap between the two is narrower than it appears. An agent is an LLM placed inside a loop where the model itself decides when the loop should stop. Everything interesting about agents follows from this shift. The autonomy that makes them useful, the cost that makes them expensive, and the design challenges that come with building one all trace back to it. It helps to see where that shift sits in the broader picture. Software built around language models has moved through a recognizable progression. It started with a single LLM call that took an input and produced an output. Then, the function calling arrived, which let the model reach out to a tool when it needed information or wanted to take an action. After that, developers began chaining calls together in code, with each step’s output feeding into the next, to handle problems too large for a single call. The most recent of these is the agent, where the developer hands control of the loop itself to the model and lets it iterate until it decides the work is done. In this article, we will walk through that progression. We will also look at how an agent is structured, what choices the model makes on every turn, what scaffolding holds it together, and when an agent is actually the right pattern to reach for. Disclaimer: This post is based on publicly shared details from various sources. Please comment if you notice any inaccuracies. FoundationsBefore the loop can be interesting, the unit that sits inside it has to be clear. A bare large language model call is a stateless function. Text goes in, text comes out, and each call typically stands alone in the model’s view. If we want it to fetch today’s weather or update a record in a database, the bare model lacks the means to do either. It can describe weather in general terms and talk about how a database record would change, but reaching the actual forecast or touching the actual database requires something more. What makes the model useful in real systems is augmentation. In other words, we give it the ability to call functions we have defined, often called tool use or function calling. We give it access to a retrieval layer that can pull in relevant documents at runtime. We also give it a way to write down information that it should carry between calls, which acts as memory. Anthropic calls the combination of these capabilities the augmented LLM, and treats it as the foundational unit of every agentic system. The augmented LLM is the building block from which everything else gets composed. This is the unit most developers have already worked with, even if they have called it something else. Examples include a chat assistant that runs Python code, a function-calling API wired to your service, and a RAG application that pulls from your documents. These systems are useful, and they are also still one call. The model produces an output, the system returns it, and the interaction ends until the next request arrives.
WorkflowsWhat happens when a problem reaches beyond what a single call can handle? The natural response is to string calls together in a sequence we design, a pattern called prompt chaining. Each step in the chain is its own LLM call, and the output of one step becomes the input to the next. We might use one call to draft an outline, a second call to expand the outline into paragraphs, and a third call to translate the paragraphs into another language. The chain is fixed in advance, and the developer writes which steps run, in what order, and what each step’s prompt looks like. This is the family of patterns Anthropic groups together as workflows. A workflow is a system where the LLM and its tools are orchestrated through a code path that the developer designed. Beyond prompt chaining, there are other workflow patterns:
The details differ, but every workflow shares the same property. The number of steps and the path through them are decided by the developer at design time, before the model sees the input. Most production systems built on LLMs today are workflows. They are predictable, debuggable, and usually cheaper than full-blown agents.
The LoopAn agent is what we get when we wrap an LLM in a loop and let the model decide when the loop should exit. The loop itself is plain code. The runtime calls the LLM, reads the output, dispatches whatever action that output specified, feeds the result back into the model’s view, and calls the model again. This continues until the model produces a response that signals a final answer. There are four steps inside each iteration, and we can call them perceive, reason, act, and observe:
The most important detail is who decides when the loop should stop. In a workflow, the developer decides at design time how many steps run. In an agent, the model decides at runtime. The model exits the loop by producing an output that the runtime interprets as a final answer. The developer typically sets a hard ceiling on the number of iterations, often called a max-turns parameter, but that ceiling is mainly a safety net. The primary stop signal comes from the model. Observation matters as a first-class step for the same reason. The model needs to see the result of its actions before deciding the next move. Removing observation would collapse the loop into a chain, with the model running on prior expectations rather than fresh results from the world. The closed loop, where every action is followed by an observation, is what lets the model adjust as it goes. With the loop established, the next question is what actually happens on each turn through it. DecisionsOn every turn through the loop, the model’s output picks one of four branches, and the runtime acts on that pick. This branching makes the loop feel intelligent because the LLM is choosing what kind of move to make. Here are the four branches:
OpenAI’s Agents SDK documents the first three branches as first-class behaviors of its loop. The fourth is more a property of how a particular prompt is written than a separate code path ReActReAct is the prompting pattern that stands for Reasoning plus Acting. The pattern asks the model to interleave reasoning steps with action steps inside the same response, so the model can think about what to do, do it, see what happened, and adjust. A ReAct trace reads like a structured journal. The model produces a thought, then an action, then receives an observation from the runtime, then produces another thought, and so on, until it reaches a final answer. Imagine an agent handling customer support:
See the diagram below:
Two things to consider from this flow are as follows:
ReAct is one way to fill the loop, and it is by far the most common pattern in the agent frameworks we work with today. GuardrailsGuardrails belong at the points where the loop crosses into the outside world. They are part of the architecture itself, designed in alongside the loop. For reference, OpenAI’s Agents SDK documents three families of guardrails, all defined by where in the loop they run.
The structural point is that guardrails sit at every interface where the loop meets the world. TradeoffsHanding control of the loop to a model is powerful, and it comes with three costs that every developer should understand before using the pattern:
ConclusionThe agent loop sits at the end of a recognizable progression. We started with a single LLM call that takes text in and returns text. We added tools, retrieval, and memory to get the augmented LLM. We strung calls together into workflows when one call was outgrown. The agent is the next rung, the point where the developer hands the control flow itself to the model. Inside the loop, four steps repeat. The model perceives the current state, reasons about it, takes an action, and observes the result. On every turn, the model’s output picks one of four branches. These are a final answer, a tool call, a handoff, and a continued thought. ReAct is the most common prompting pattern for filling the loop, with reasoning and action interleaved. Guardrails live at every place where the loop crosses into the outside world. The design carries three real costs. These are compounding reliability across steps, the harness scaffolding that production loops require, and the question of whether a workflow would solve the problem more cheaply. References:
|
Comments
Post a Comment