> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orgo.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Claude Computer Use

> Let Claude control an Orgo computer

Wire Anthropic's computer use toolset to an Orgo computer. Claude sees a screenshot, asks for a click or a keystroke, and your loop runs it on the computer.

## Coordinate space

Orgo computers boot at `1280x720x24`.

The current toolset, `computer_toolset_20260801`, takes no display size. Anthropic's documentation states the rule directly: coordinates are always in the pixel space of the screenshots you return. Send an Orgo screenshot unresized and the coordinates Claude returns are already the computer's pixels. Pass them straight to `left_click`.

A 1280x720 screenshot is inside Claude's image size limit, so there is no reason to shrink one. If you resize anyway, keep the scale factor and divide the coordinates back before you click:

```python theme={null}
# Only needed if you resized the screenshot before sending it.
def to_computer_pixels(x, y, scale):
    return round(x / scale), round(y / scale)
```

<Warning>
  Send a downscaled screenshot and click the raw coordinates and every click lands short of where Claude aimed. Nothing errors. The agent looks like it is confidently clicking the wrong thing.
</Warning>

The older tool versions work differently. `computer_20251124` and `computer_20250124` still take `display_width_px` and `display_height_px`, and those must match the size of the screenshots you send. On a default Orgo computer that is `1280` and `720`.

If you create a computer at a non-default resolution, or resize its screen later, use that resolution instead. `GET /computers/{id}/screens` reports the current `width` and `height`.

## Setup

Install the required packages:

<CodeGroup>
  ```bash pip theme={null}
  pip install orgo anthropic
  ```

  ```bash npm theme={null}
  npm install orgo@1 @anthropic-ai/sdk
  ```

  ```bash yarn theme={null}
  yarn add orgo@1 @anthropic-ai/sdk
  ```

  ```bash pnpm theme={null}
  pnpm add orgo@1 @anthropic-ai/sdk
  ```
</CodeGroup>

<Note>
  The TypeScript SDK ships in version 1 of the `orgo` npm package. From version 2, `orgo` on npm is the [CLI](/guides/cli) and exports no library, so pin `orgo@1` to `import { Computer } from 'orgo'`.
</Note>

Set up your API keys:

<CodeGroup>
  ```bash terminal icon="terminal" theme={null}
  # Export as environment variables
  export ORGO_API_KEY=your_orgo_api_key
  export ANTHROPIC_API_KEY=your_anthropic_api_key
  ```

  ```python setup.py icon="python" theme={null}
  import os
  os.environ["ORGO_API_KEY"] = "your_orgo_api_key"
  os.environ["ANTHROPIC_API_KEY"] = "your_anthropic_api_key"
  ```

  ```typescript setup.ts icon="square-js" theme={null}
  process.env.ORGO_API_KEY = "your_orgo_api_key";
  process.env.ANTHROPIC_API_KEY = "your_anthropic_api_key";
  ```
</CodeGroup>

## Let Orgo run the loop

To skip writing the loop, call the [Computer Use API](/api-reference/chat/completions). `POST /v1/chat/completions` runs the agent on Orgo's side against the computer you name in `computer_id`, then returns the final assistant message in OpenAI's chat completions format.

## Run the agent loop yourself

For full control, drive the Anthropic API yourself.

Three things about the toolset shape the loop:

* One entry in `tools` declares every member. The member name arrives in the `tool_use` block's `name`, and there is no `action` field in `input`.
* Claude can return several member calls in one turn. Run them in order and stop at the first failure.
* Return one `tool_result` per `tool_use` block, in a single user message. Member results echo `toolset_name`.

<Warning>
  The examples pass `ram` and `cpu`, and take screenshots and scroll over HTTP, so they also run on older SDKs. SDK versions before 0.0.47 (Python) and 1.0.22 (TypeScript) default to 2 GB, which `POST /computers` rejects with `400`. Their `screenshot()` and `screenshot_base64()` cannot read the stored-image path the API returns by default, and their `scroll()` takes no coordinates, so the computer scrolls at the top-left corner.
</Warning>

<CodeGroup>
  ```python advanced.py expandable icon="python" theme={null}
  import os

  import anthropic
  import requests
  from orgo import Computer

  API = "https://www.orgo.ai/api"
  HEADERS = {"Authorization": f"Bearer {os.environ['ORGO_API_KEY']}"}

  TOOLS = [
      {"type": "computer_toolset_20260801"},
      {"type": "bash_20250124", "name": "bash"},
  ]


  def create_agent_loop(instruction, model="claude-sonnet-5", max_iterations=20):
      # Pass a size: SDKs before 0.0.47 default to 2 GB, which the API rejects.
      computer = Computer(ram=4, cpu=1)
      client = anthropic.Anthropic()

      try:
          messages = [{"role": "user", "content": instruction}]

          for _ in range(max_iterations):
              response = client.messages.create(
                  model=model,
                  messages=messages,
                  tools=TOOLS,
                  max_tokens=8192,
              )
              messages.append({"role": "assistant", "content": response.content})

              # No tool calls means Claude is done.
              tool_results = run_tool_calls(computer, response)
              if not tool_results:
                  break

              messages.append({"role": "user", "content": tool_results})

          return messages

      finally:
          computer.destroy()


  def run_tool_calls(computer, response):
      """Run one turn's tool calls in order, stopping at the first failure."""
      results = []
      failed = False

      for block in response.content:
          if block.type != "tool_use":
              continue

          if block.name == "bash":
              results.append({
                  "type": "tool_result",
                  "tool_use_id": block.id,
                  "content": computer.bash(block.input["command"]),
              })
              continue

          # Dispatch on the pair, because a custom tool may reuse a member name.
          if getattr(block, "toolset_name", None) != "computer":
              continue

          result = {
              "type": "tool_result",
              "tool_use_id": block.id,
              "toolset_name": "computer",
          }

          if failed:
              result["content"] = "Not executed: an earlier computer action in this turn failed."
              result["is_error"] = True
          else:
              try:
                  result["content"] = run_member(computer, block.name, block.input)
              except Exception as error:
                  result["content"] = f"Error: {error}"
                  result["is_error"] = True
                  failed = True

          results.append(result)

      return results


  def screenshot_base64(computer):
      """The screen as base64 PNG, straight from the API."""
      response = requests.get(
          f"{API}/computers/{computer.computer_id}/screenshot",
          params={"response_format": "base64"},
          headers=HEADERS,
      )
      response.raise_for_status()
      return response.json()["image"]


  def scroll(computer, x, y, direction, amount):
      """Scroll at (x, y). Orgo scrolls up or down only."""
      if direction not in ("up", "down"):
          raise ValueError(f"Unsupported scroll direction: {direction}")
      response = requests.post(
          f"{API}/computers/{computer.computer_id}/scroll",
          json={"x": x, "y": y, "direction": direction, "amount": amount},
          headers=HEADERS,
      )
      response.raise_for_status()


  def run_member(computer, name, params):
      """Map one member tool to an Orgo call. Only screenshot returns an image."""
      if name == "screenshot":
          return [{
              "type": "image",
              "source": {
                  "type": "base64",
                  "media_type": "image/png",
                  "data": screenshot_base64(computer),
              },
          }]

      if name == "left_click":
          computer.left_click(*params["coordinate"])
      elif name == "right_click":
          computer.right_click(*params["coordinate"])
      elif name == "double_click":
          computer.double_click(*params["coordinate"])
      elif name == "type":
          computer.type(params["text"])
      elif name == "key":
          computer.key(params["text"])
      elif name == "scroll":
          scroll(computer, *params["coordinate"], params["scroll_direction"], params["scroll_amount"])
      elif name == "wait":
          computer.wait(params["duration"])
      else:
          raise ValueError(f"Unsupported member: {name}")

      return "OK"
  ```

  ```typescript advanced.ts expandable icon="square-js" theme={null}
  import { Computer } from 'orgo';
  import Anthropic from '@anthropic-ai/sdk';

  const API = "https://www.orgo.ai/api";
  const HEADERS = { Authorization: `Bearer ${process.env.ORGO_API_KEY}` };

  const TOOLS: any[] = [
      { type: "computer_toolset_20260801" },
      { type: "bash_20250124", name: "bash" },
  ];

  async function createAgentLoop(
      instruction: string,
      model = "claude-sonnet-5",
      maxIterations = 20,
  ) {
      // Pass a size: SDKs before 1.0.22 default to 2 GB, which the API rejects.
      const computer = await Computer.create({ ram: 4, cpu: 1 });
      const client = new Anthropic();

      try {
          const messages: any[] = [{ role: "user", content: instruction }];

          for (let i = 0; i < maxIterations; i++) {
              const response = await client.messages.create({
                  model,
                  messages,
                  tools: TOOLS,
                  max_tokens: 8192,
              });
              messages.push({ role: "assistant", content: response.content });

              // No tool calls means Claude is done.
              const toolResults = await runToolCalls(computer, response);
              if (toolResults.length === 0) break;

              messages.push({ role: "user", content: toolResults });
          }

          return messages;

      } finally {
          await computer.destroy();
      }
  }

  async function runToolCalls(computer: Computer, response: any) {
      const results: any[] = [];
      let failed = false;

      for (const block of response.content) {
          if (block.type !== "tool_use") continue;

          if (block.name === "bash") {
              results.push({
                  type: "tool_result",
                  tool_use_id: block.id,
                  content: await computer.bash(block.input.command),
              });
              continue;
          }

          // Dispatch on the pair, because a custom tool may reuse a member name.
          if (block.toolset_name !== "computer") continue;

          const result: any = {
              type: "tool_result",
              tool_use_id: block.id,
              toolset_name: "computer",
          };

          if (failed) {
              result.content = "Not executed: an earlier computer action in this turn failed.";
              result.is_error = true;
          } else {
              try {
                  result.content = await runMember(computer, block.name, block.input);
              } catch (error) {
                  result.content = `Error: ${error}`;
                  result.is_error = true;
                  failed = true;
              }
          }

          results.push(result);
      }

      return results;
  }

  // The screen as base64 PNG, straight from the API.
  async function screenshotBase64(computer: Computer): Promise<string> {
      const response = await fetch(
          `${API}/computers/${computer.computerId}/screenshot?response_format=base64`,
          { headers: HEADERS },
      );
      if (!response.ok) throw new Error(`Screenshot failed: ${response.status}`);
      return (await response.json()).image;
  }

  // Scroll at (x, y). Orgo scrolls up or down only.
  async function scroll(computer: Computer, x: number, y: number, direction: string, amount: number) {
      if (direction !== "up" && direction !== "down") {
          throw new Error(`Unsupported scroll direction: ${direction}`);
      }
      const response = await fetch(`${API}/computers/${computer.computerId}/scroll`, {
          method: "POST",
          headers: { ...HEADERS, "Content-Type": "application/json" },
          body: JSON.stringify({ x, y, direction, amount }),
      });
      if (!response.ok) throw new Error(`Scroll failed: ${response.status}`);
  }

  async function runMember(computer: Computer, name: string, params: any) {
      if (name === "screenshot") {
          return [{
              type: "image",
              source: {
                  type: "base64",
                  media_type: "image/png",
                  data: await screenshotBase64(computer),
              },
          }];
      }

      const [x, y] = params.coordinate ?? [];

      switch (name) {
          case "left_click":   await computer.leftClick(x, y); break;
          case "right_click":  await computer.rightClick(x, y); break;
          case "double_click": await computer.doubleClick(x, y); break;
          case "type":         await computer.type(params.text); break;
          case "key":          await computer.key(params.text); break;
          case "scroll":       await scroll(computer, x, y, params.scroll_direction, params.scroll_amount); break;
          case "wait":         await computer.wait(params.duration); break;
          default: throw new Error(`Unsupported member: ${name}`);
      }

      return "OK";
  }
  ```
</CodeGroup>

<Note>
  The toolset has 17 members, and the loop above maps 8 of them. The rest come back as an error `tool_result`, which Claude recovers from but wastes a turn on. Disable them with `configs` so Claude never sees them:

  ```python theme={null}
  {
      "type": "computer_toolset_20260801",
      "configs": {
          name: {"enabled": False}
          for name in [
              "zoom", "middle_click", "triple_click", "left_click_drag",
              "mouse_move", "left_mouse_down", "left_mouse_up",
              "cursor_position", "hold_key",
          ]
      },
  }
  ```

  A `configs` that disables every member is rejected, so drop the whole entry instead of emptying it.
</Note>

## Using Claude's thinking capability

Claude can stream its reasoning through the `thinking` parameter:

<CodeGroup>
  ```python thinking.py icon="python" theme={null}
  import anthropic
  from orgo import Computer

  # Initialize components
  computer = Computer(ram=4, cpu=1)
  client = anthropic.Anthropic()

  try:
      response = client.messages.create(
          model="claude-sonnet-5",
          max_tokens=8192,
          messages=[{"role": "user", "content": "Find an image of a cat on the web"}],
          tools=[
              {"type": "computer_toolset_20260801"},
              {"type": "bash_20250124", "name": "bash"},
          ],
          thinking={"type": "adaptive", "display": "summarized"},
      )

      # Access the thinking content
      for block in response.content:
          if block.type == "thinking":
              print("Claude's reasoning:")
              print(block.thinking)
  finally:
      # Clean up
      computer.destroy()
  ```

  ```typescript thinking.ts icon="square-js" theme={null}
  import { Computer } from 'orgo';
  import Anthropic from '@anthropic-ai/sdk';

  // Initialize components
  const computer = await Computer.create({ ram: 4, cpu: 1 });
  const client = new Anthropic();

  try {
      const response = await client.messages.create({
          model: "claude-sonnet-5",
          max_tokens: 8192,
          messages: [{ role: "user", content: "Find an image of a cat on the web" }],
          tools: [
              { type: "computer_toolset_20260801" },
              { type: "bash_20250124", name: "bash" },
          ] as any,
          thinking: { type: "adaptive", display: "summarized" },
      });

      // Access the thinking content
      for (const block of response.content) {
          if (block.type === "thinking") {
              console.log("Claude's reasoning:");
              console.log((block as any).thinking);
          }
      }
  } finally {
      // Clean up
      await computer.destroy();
  }
  ```
</CodeGroup>

<Note>
  Set `display: "summarized"` to read the reasoning. The current models default to `"omitted"`, which streams `thinking` blocks with empty text.
</Note>

## Tool compatibility

Each member the loop above handles maps to one Orgo call:

| Toolset member | Orgo call (Python) | Orgo call (TypeScript) | Description |
| - | - | - | - |
| `screenshot` | `GET /computers/{id}/screenshot?response_format=base64` | `GET /computers/{id}/screenshot?response_format=base64` | Capture the screen (returns base64 PNG) |
| `left_click` | `computer.left_click(x, y)` | `await computer.leftClick(x, y)` | Left click at coordinates |
| `right_click` | `computer.right_click(x, y)` | `await computer.rightClick(x, y)` | Right click at coordinates |
| `double_click` | `computer.double_click(x, y)` | `await computer.doubleClick(x, y)` | Double click at coordinates |
| `type` | `computer.type(text)` | `await computer.type(text)` | Type text |
| `key` | `computer.key(key_sequence)` | `await computer.key(keySequence)` | Press keys (e.g. "Return", "ctrl+c") |
| `scroll` | `POST /computers/{id}/scroll` with `x`, `y`, `direction`, `amount` | `POST /computers/{id}/scroll` with `x`, `y`, `direction`, `amount` | Scroll up or down at coordinates |
| `wait` | `computer.wait(seconds)` | `await computer.wait(seconds)` | Wait for a number of seconds |

Only `screenshot` and `zoom` return images. Every other member returns text, so `"OK"` is a valid result.

## Picking a model

| Model | When to use |
| - | - |
| `claude-opus-5-5` | Hardest, multi-step desktop tasks where accuracy and judgment matter most. |
| `claude-sonnet-5-5` | The default for most computer-use agents: fast, capable, and cheaper than Opus. |
| `claude-opus-5`, `claude-sonnet-5`, `claude-opus-4-8` | Previous generations, still supported on the current toolset. |

`claude-fable-5` and `claude-fable-5-1` also support the toolset. `claude-haiku-4-5` supports only the older `computer_20250124` tool.

<Warning>
  `claude-opus-5-5` and `claude-sonnet-5-5` accept only `computer_toolset_20260801` on the Claude API. Sending them `computer_20251124` returns a 400.
</Warning>

## Tool versions

Match the tool `type` to the model you are calling:

| Tool `type` | Models | Beta header |
| - | - | - |
| `computer_toolset_20260801` | Opus 5.5, Sonnet 5.5, Opus 5, Sonnet 5, Opus 4.8, Fable 5, Fable 5.1 | None |
| `computer_20251124` | Opus 5, Sonnet 5, Opus 4.7, Opus 4.6, Sonnet 4.6, Opus 4.5 | `computer-use-2025-11-24` |
| `computer_20250124` | Haiku 4.5, Sonnet 4.5, and earlier Claude models | `computer-use-2025-01-24` |

The two dated tool versions remain available for existing integrations. Use the toolset for anything new: it needs no beta header, it batches several actions into one turn, and it drops the display size fields that were the most common source of misplaced clicks.

The `bash_20250124` tool needs no beta header on any of them.

<Note>
  TypeScript users: All methods are async and must be awaited. The TypeScript SDK uses camelCase for method names (e.g. `leftClick` instead of `left_click`).
</Note>

## Video tutorial

<iframe width="100%" height="400" src="https://www.youtube.com/embed/JTbgxry--Fk" title="Anthropic Computer Use Setup in 30 Seconds (With Orgo)" frameBorder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowFullScreen />

The video covers setup. The tool shapes above are the current ones.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.