Orgo - Cloud computers for AI agents

Wire Anthropic’s computer use toolset to an Orgo computer. Claude sees a screenshot, asks for a click or a keystroke, and your loop runs it on the computer.

Coordinate space

Orgo computers boot at 1280x720x24. The current toolset, computer_toolset_20260801, takes no display size. Anthropic’s documentation states the rule directly: coordinates are always in the pixel space of the screenshots you return. Send an Orgo screenshot unresized and the coordinates Claude returns are already the computer’s pixels. Pass them straight to left_click. A 1280x720 screenshot is inside Claude’s image size limit, so there is no reason to shrink one. If you resize anyway, keep the scale factor and divide the coordinates back before you click:

# Only needed if you resized the screenshot before sending it.
def to_computer_pixels(x, y, scale):
    return round(x / scale), round(y / scale)

Send a downscaled screenshot and click the raw coordinates and every click lands short of where Claude aimed. Nothing errors. The agent looks like it is confidently clicking the wrong thing.

The older tool versions work differently. computer_20251124 and computer_20250124 still take display_width_px and display_height_px, and those must match the size of the screenshots you send. On a default Orgo computer that is 1280 and 720. If you create a computer at a non-default resolution, or resize its screen later, use that resolution instead. GET /computers/{id}/screens reports the current width and height.

Setup

Install the required packages:

pip install orgo anthropic

npm install orgo@1 @anthropic-ai/sdk

yarn add orgo@1 @anthropic-ai/sdk

pnpm add orgo@1 @anthropic-ai/sdk

The TypeScript SDK ships in version 1 of the orgo npm package. From version 2, orgo on npm is the CLI and exports no library, so pin orgo@1 to import { Computer } from 'orgo'.

Set up your API keys:

# Export as environment variables
export ORGO_API_KEY=your_orgo_api_key
export ANTHROPIC_API_KEY=your_anthropic_api_key

import os
os.environ["ORGO_API_KEY"] = "your_orgo_api_key"
os.environ["ANTHROPIC_API_KEY"] = "your_anthropic_api_key"

process.env.ORGO_API_KEY = "your_orgo_api_key";
process.env.ANTHROPIC_API_KEY = "your_anthropic_api_key";

Let Orgo run the loop

To skip writing the loop, call the Computer Use API. POST /v1/chat/completions runs the agent on Orgo’s side against the computer you name in computer_id, then returns the final assistant message in OpenAI’s chat completions format.

Run the agent loop yourself

For full control, drive the Anthropic API yourself. Three things about the toolset shape the loop:

One entry in tools declares every member. The member name arrives in the tool_use block’s name, and there is no action field in input.
Claude can return several member calls in one turn. Run them in order and stop at the first failure.
Return one tool_result per tool_use block, in a single user message. Member results echo toolset_name.

The examples pass ram and cpu, and take screenshots and scroll over HTTP, so they also run on older SDKs. SDK versions before 0.0.47 (Python) and 1.0.22 (TypeScript) default to 2 GB, which POST /computers rejects with 400. Their screenshot() and screenshot_base64() cannot read the stored-image path the API returns by default, and their scroll() takes no coordinates, so the computer scrolls at the top-left corner.

import os

import anthropic
import requests
from orgo import Computer

API = "https://www.orgo.ai/api"
HEADERS = {"Authorization": f"Bearer {os.environ['ORGO_API_KEY']}"}

TOOLS = [
    {"type": "computer_toolset_20260801"},
    {"type": "bash_20250124", "name": "bash"},
]


def create_agent_loop(instruction, model="claude-sonnet-5", max_iterations=20):
    # Pass a size: SDKs before 0.0.47 default to 2 GB, which the API rejects.
    computer = Computer(ram=4, cpu=1)
    client = anthropic.Anthropic()

    try:
        messages = [{"role": "user", "content": instruction}]

        for _ in range(max_iterations):
            response = client.messages.create(
                model=model,
                messages=messages,
                tools=TOOLS,
                max_tokens=8192,
            )
            messages.append({"role": "assistant", "content": response.content})

            # No tool calls means Claude is done.
            tool_results = run_tool_calls(computer, response)
            if not tool_results:
                break

            messages.append({"role": "user", "content": tool_results})

        return messages

    finally:
        computer.destroy()


def run_tool_calls(computer, response):
    """Run one turn's tool calls in order, stopping at the first failure."""
    results = []
    failed = False

    for block in response.content:
        if block.type != "tool_use":
            continue

        if block.name == "bash":
            results.append({
                "type": "tool_result",
                "tool_use_id": block.id,
                "content": computer.bash(block.input["command"]),
            })
            continue

        # Dispatch on the pair, because a custom tool may reuse a member name.
        if getattr(block, "toolset_name", None) != "computer":
            continue

        result = {
            "type": "tool_result",
            "tool_use_id": block.id,
            "toolset_name": "computer",
        }

        if failed:
            result["content"] = "Not executed: an earlier computer action in this turn failed."
            result["is_error"] = True
        else:
            try:
                result["content"] = run_member(computer, block.name, block.input)
            except Exception as error:
                result["content"] = f"Error: {error}"
                result["is_error"] = True
                failed = True

        results.append(result)

    return results


def screenshot_base64(computer):
    """The screen as base64 PNG, straight from the API."""
    response = requests.get(
        f"{API}/computers/{computer.computer_id}/screenshot",
        params={"response_format": "base64"},
        headers=HEADERS,
    )
    response.raise_for_status()
    return response.json()["image"]


def scroll(computer, x, y, direction, amount):
    """Scroll at (x, y). Orgo scrolls up or down only."""
    if direction not in ("up", "down"):
        raise ValueError(f"Unsupported scroll direction: {direction}")
    response = requests.post(
        f"{API}/computers/{computer.computer_id}/scroll",
        json={"x": x, "y": y, "direction": direction, "amount": amount},
        headers=HEADERS,
    )
    response.raise_for_status()


def run_member(computer, name, params):
    """Map one member tool to an Orgo call. Only screenshot returns an image."""
    if name == "screenshot":
        return [{
            "type": "image",
            "source": {
                "type": "base64",
                "media_type": "image/png",
                "data": screenshot_base64(computer),
            },
        }]

    if name == "left_click":
        computer.left_click(*params["coordinate"])
    elif name == "right_click":
        computer.right_click(*params["coordinate"])
    elif name == "double_click":
        computer.double_click(*params["coordinate"])
    elif name == "type":
        computer.type(params["text"])
    elif name == "key":
        computer.key(params["text"])
    elif name == "scroll":
        scroll(computer, *params["coordinate"], params["scroll_direction"], params["scroll_amount"])
    elif name == "wait":
        computer.wait(params["duration"])
    else:
        raise ValueError(f"Unsupported member: {name}")

    return "OK"

import { Computer } from 'orgo';
import Anthropic from '@anthropic-ai/sdk';

const API = "https://www.orgo.ai/api";
const HEADERS = { Authorization: `Bearer ${process.env.ORGO_API_KEY}` };

const TOOLS: any[] = [
    { type: "computer_toolset_20260801" },
    { type: "bash_20250124", name: "bash" },
];

async function createAgentLoop(
    instruction: string,
    model = "claude-sonnet-5",
    maxIterations = 20,
) {
    // Pass a size: SDKs before 1.0.22 default to 2 GB, which the API rejects.
    const computer = await Computer.create({ ram: 4, cpu: 1 });
    const client = new Anthropic();

    try {
        const messages: any[] = [{ role: "user", content: instruction }];

        for (let i = 0; i < maxIterations; i++) {
            const response = await client.messages.create({
                model,
                messages,
                tools: TOOLS,
                max_tokens: 8192,
            });
            messages.push({ role: "assistant", content: response.content });

            // No tool calls means Claude is done.
            const toolResults = await runToolCalls(computer, response);
            if (toolResults.length === 0) break;

            messages.push({ role: "user", content: toolResults });
        }

        return messages;

    } finally {
        await computer.destroy();
    }
}

async function runToolCalls(computer: Computer, response: any) {
    const results: any[] = [];
    let failed = false;

    for (const block of response.content) {
        if (block.type !== "tool_use") continue;

        if (block.name === "bash") {
            results.push({
                type: "tool_result",
                tool_use_id: block.id,
                content: await computer.bash(block.input.command),
            });
            continue;
        }

        // Dispatch on the pair, because a custom tool may reuse a member name.
        if (block.toolset_name !== "computer") continue;

        const result: any = {
            type: "tool_result",
            tool_use_id: block.id,
            toolset_name: "computer",
        };

        if (failed) {
            result.content = "Not executed: an earlier computer action in this turn failed.";
            result.is_error = true;
        } else {
            try {
                result.content = await runMember(computer, block.name, block.input);
            } catch (error) {
                result.content = `Error: ${error}`;
                result.is_error = true;
                failed = true;
            }
        }

        results.push(result);
    }

    return results;
}

// The screen as base64 PNG, straight from the API.
async function screenshotBase64(computer: Computer): Promise<string> {
    const response = await fetch(
        `${API}/computers/${computer.computerId}/screenshot?response_format=base64`,
        { headers: HEADERS },
    );
    if (!response.ok) throw new Error(`Screenshot failed: ${response.status}`);
    return (await response.json()).image;
}

// Scroll at (x, y). Orgo scrolls up or down only.
async function scroll(computer: Computer, x: number, y: number, direction: string, amount: number) {
    if (direction !== "up" && direction !== "down") {
        throw new Error(`Unsupported scroll direction: ${direction}`);
    }
    const response = await fetch(`${API}/computers/${computer.computerId}/scroll`, {
        method: "POST",
        headers: { ...HEADERS, "Content-Type": "application/json" },
        body: JSON.stringify({ x, y, direction, amount }),
    });
    if (!response.ok) throw new Error(`Scroll failed: ${response.status}`);
}

async function runMember(computer: Computer, name: string, params: any) {
    if (name === "screenshot") {
        return [{
            type: "image",
            source: {
                type: "base64",
                media_type: "image/png",
                data: await screenshotBase64(computer),
            },
        }];
    }

    const [x, y] = params.coordinate ?? [];

    switch (name) {
        case "left_click":   await computer.leftClick(x, y); break;
        case "right_click":  await computer.rightClick(x, y); break;
        case "double_click": await computer.doubleClick(x, y); break;
        case "type":         await computer.type(params.text); break;
        case "key":          await computer.key(params.text); break;
        case "scroll":       await scroll(computer, x, y, params.scroll_direction, params.scroll_amount); break;
        case "wait":         await computer.wait(params.duration); break;
        default: throw new Error(`Unsupported member: ${name}`);
    }

    return "OK";
}

The toolset has 17 members, and the loop above maps 8 of them. The rest come back as an error tool_result, which Claude recovers from but wastes a turn on. Disable them with configs so Claude never sees them:

{
    "type": "computer_toolset_20260801",
    "configs": {
        name: {"enabled": False}
        for name in [
            "zoom", "middle_click", "triple_click", "left_click_drag",
            "mouse_move", "left_mouse_down", "left_mouse_up",
            "cursor_position", "hold_key",
        ]
    },
}

A configs that disables every member is rejected, so drop the whole entry instead of emptying it.

Using Claude’s thinking capability

Claude can stream its reasoning through the thinking parameter:

import anthropic
from orgo import Computer

# Initialize components
computer = Computer(ram=4, cpu=1)
client = anthropic.Anthropic()

try:
    response = client.messages.create(
        model="claude-sonnet-5",
        max_tokens=8192,
        messages=[{"role": "user", "content": "Find an image of a cat on the web"}],
        tools=[
            {"type": "computer_toolset_20260801"},
            {"type": "bash_20250124", "name": "bash"},
        ],
        thinking={"type": "adaptive", "display": "summarized"},
    )

    # Access the thinking content
    for block in response.content:
        if block.type == "thinking":
            print("Claude's reasoning:")
            print(block.thinking)
finally:
    # Clean up
    computer.destroy()

import { Computer } from 'orgo';
import Anthropic from '@anthropic-ai/sdk';

// Initialize components
const computer = await Computer.create({ ram: 4, cpu: 1 });
const client = new Anthropic();

try {
    const response = await client.messages.create({
        model: "claude-sonnet-5",
        max_tokens: 8192,
        messages: [{ role: "user", content: "Find an image of a cat on the web" }],
        tools: [
            { type: "computer_toolset_20260801" },
            { type: "bash_20250124", name: "bash" },
        ] as any,
        thinking: { type: "adaptive", display: "summarized" },
    });

    // Access the thinking content
    for (const block of response.content) {
        if (block.type === "thinking") {
            console.log("Claude's reasoning:");
            console.log((block as any).thinking);
        }
    }
} finally {
    // Clean up
    await computer.destroy();
}

Set display: "summarized" to read the reasoning. The current models default to "omitted", which streams thinking blocks with empty text.

Tool compatibility

Each member the loop above handles maps to one Orgo call:

Toolset member	Orgo call (Python)	Orgo call (TypeScript)	Description
`screenshot`	`GET /computers/{id}/screenshot?response_format=base64`	`GET /computers/{id}/screenshot?response_format=base64`	Capture the screen (returns base64 PNG)
`left_click`	`computer.left_click(x, y)`	`await computer.leftClick(x, y)`	Left click at coordinates
`right_click`	`computer.right_click(x, y)`	`await computer.rightClick(x, y)`	Right click at coordinates
`double_click`	`computer.double_click(x, y)`	`await computer.doubleClick(x, y)`	Double click at coordinates
`type`	`computer.type(text)`	`await computer.type(text)`	Type text
`key`	`computer.key(key_sequence)`	`await computer.key(keySequence)`	Press keys (e.g. “Return”, “ctrl+c”)
`scroll`	`POST /computers/{id}/scroll` with `x`, `y`, `direction`, `amount`	`POST /computers/{id}/scroll` with `x`, `y`, `direction`, `amount`	Scroll up or down at coordinates
`wait`	`computer.wait(seconds)`	`await computer.wait(seconds)`	Wait for a number of seconds

Only screenshot and zoom return images. Every other member returns text, so "OK" is a valid result.

Picking a model

Model	When to use
`claude-opus-5-5`	Hardest, multi-step desktop tasks where accuracy and judgment matter most.
`claude-sonnet-5-5`	The default for most computer-use agents: fast, capable, and cheaper than Opus.
`claude-opus-5`, `claude-sonnet-5`, `claude-opus-4-8`	Previous generations, still supported on the current toolset.

claude-fable-5 and claude-fable-5-1 also support the toolset. claude-haiku-4-5 supports only the older computer_20250124 tool.

claude-opus-5-5 and claude-sonnet-5-5 accept only computer_toolset_20260801 on the Claude API. Sending them computer_20251124 returns a 400.

Tool versions

Match the tool type to the model you are calling:

Tool `type`	Models	Beta header
`computer_toolset_20260801`	Opus 5.5, Sonnet 5.5, Opus 5, Sonnet 5, Opus 4.8, Fable 5, Fable 5.1	None
`computer_20251124`	Opus 5, Sonnet 5, Opus 4.7, Opus 4.6, Sonnet 4.6, Opus 4.5	`computer-use-2025-11-24`
`computer_20250124`	Haiku 4.5, Sonnet 4.5, and earlier Claude models	`computer-use-2025-01-24`

The two dated tool versions remain available for existing integrations. Use the toolset for anything new: it needs no beta header, it batches several actions into one turn, and it drops the display size fields that were the most common source of misplaced clicks. The bash_20250124 tool needs no beta header on any of them.

TypeScript users: All methods are async and must be awaited. The TypeScript SDK uses camelCase for method names (e.g. leftClick instead of left_click).

Video tutorial

The video covers setup. The tool shapes above are the current ones.

Get Started

Guides

Migrations

Templates

Model integrations

Claude Computer Use

Coordinate space

Setup

Let Orgo run the loop

Run the agent loop yourself

Using Claude’s thinking capability

Tool compatibility

Picking a model

Tool versions

Video tutorial

​Coordinate space

​Setup

​Let Orgo run the loop

​Run the agent loop yourself

​Using Claude’s thinking capability

​Tool compatibility

​Picking a model

​Tool versions

​Video tutorial

Coordinate space

Setup

Let Orgo run the loop

Run the agent loop yourself

Using Claude’s thinking capability

Tool compatibility

Picking a model

Tool versions

Video tutorial