Validating Tool Inputs and Outputs

Strict schema validation and business-logic validation solve different problems, and conflating them is the most common mistake in a tool integration. A schema — even with strict: true enforced — guarantees that a refund_order call has a well-typed order_id string and a numeric amount. It says nothing about whether that order actually exists, whether the amount requested is within what's actually owed, or whether the caller is authorized to issue it at all. Those are business-rule checks your application still has to run, every time, after the schema has already passed.

OpenAI's own guidance on defining functions is explicit about this: validate inputs before execution, and when something's wrong, return an informative error message rather than a bare failure — the model can often recover and retry sensibly if it's told what was wrong, and will otherwise just guess again in the dark.

def execute_refund(order_id: str, amount: float) -> str:
    order = orders.get(order_id)
    if order is None:
        return f"Error: no order found with id {order_id}."
    if amount > order.remaining_refundable:
        return f"Error: {amount} exceeds the {order.remaining_refundable} still refundable."
    if not caller_is_authorized(order):
        return "Error: this caller is not authorized to refund this order."

    process_refund(order, amount)
    return f"Refunded {amount} on order {order_id}."

This split matters because strict mode can create a false sense of safety. A well-formed call isn't the same thing as a safe or a correct one, and treating schema compliance as the finish line is exactly what lets a plausible-looking but wrong order_id, or an amount outside policy, sail through untouched. The fix isn't a stricter schema — JSON Schema has no way to express "this order belongs to this customer" or "this amount is under the refund limit" — it's a second, separate layer of checks that runs after the schema has already passed, against your actual data and your actual authorization rules. Treat the two as a pipeline: schema validation first, business validation second, and only then execution.

The prompting lesson established that instructions and data deserve separate channels, and a tool result deserves exactly the same treatment — arguably more, because it can come from a source further outside your control than a user's typed message: a scraped web page, a customer's own free-text field, an external API's error response. Anthropic's own guidance on mitigating prompt injection recommends putting untrusted third-party content only inside tool_result blocks, never in a system prompt or a plain user text block, and states two further concrete habits: make the nature and source of returned content explicit — that this is, say, OCR text from an uploaded image or the body of an inbound email from an unknown sender — and state the policy plainly in your system prompt, that content returned from tools must never be treated as an instruction that overrides the system prompt or the user's original request.