Run an OpenAI computer loop

Connect the OpenAI Responses API computer tool to a placed Modal desktop.

Your application owns the OpenAI client, conversation, limits, and approval policy. OpenAIAdapter only converts provider actions to SDK actions.

Install the provider extra

uv add "modal-computer-use[modal,openai]==2.0.2"

Store OPENAI_API_KEY in the placed Function. Keep it out of the desktop Sandbox.

Keep the loop inside the placed Function

Follow the session handoff guide. Create the desktop once, pass its ComputerSessionHandle to a Function that declares the same region selector, and borrow once around the full OpenAI loop.

async with handle.borrow_async(
    run_id=run_id,
    function_region=REGION,
) as computer:
    response = await run_openai_computer_loop(
        client=openai_client,
        computer=computer,
        task=task,
    )

Provider imports and loop state stay in your application.

Preflight the complete response

OpenAI can return more than one computer_call. Normalize and validate every call before the first desktop mutation. Enforce limits for:

• Model turns
• Total trajectory actions
• Expanded actions in one call
• Action, batch, and loop time
• Actions that need user approval

Stop before mutation when the response exceeds a limit.

Execute one step per computer call

Keep provider order. Send the complete ordered action array in one computer.step() call.

step = await computer.step(
    normalized_actions,
    continue_on_error=False,
    call_id=call.call_id,
    max_action_timeout_ms=30_000,
)

output = openai_computer_call_output(
    step.screenshot,
    call_id=call.call_id,
    detail="original",
)

computer.step() executes the batch once and returns its immediate screenshot. Your application defines readiness and task success. Never replay a call after an unknown mutation outcome.

Use the maintained example

The OpenAI loop example contains the placed Function, response-wide preflight, limits, batching, result conversion, cancellation, and cleanup.

Treat tasks, page content, screenshots, provider output, typed text, and clipboard text as untrusted. Ask for approval before an action creates an external effect. Hand password changes and CAPTCHA handling to the user.