RefineAct
A runtime verification framework for checking an LLM agent's proposed actions before execution.
RefineAct studies LLM-agent reliability at the boundary between proposed action and executed action. It inserts a runtime verification layer that checks whether a candidate action is consistent with the task intent before the action is allowed to affect the environment.
Standard agent frameworks can execute a tool action as soon as the model produces it. If the action is unsafe or inconsistent with the task, the error may become visible only after the environment has already changed.
RefineAct inserts a verification layer between action generation and execution. Candidate actions are checked against a task-intent specification, and rejected actions are regenerated before they can run.
Accepted at ASE 2026. Across 144 agent tasks spanning five ToolEmu domains, RefineAct reduces failure incidence from 77% to 39% while improving task completion quality from 1.0 to 1.9 on a 0–3 scale, and agents self-correct in 68% of blocked actions.
Checking actions at runtime provides an auditable control point for agent behavior, complementing prompt design and model training with an execution-time reliability mechanism.
The code from the paper is available at RefineAct.
The formal approach and complete evaluation are in the paper (Batole et al., 2026).