Chains and capture

Both features work on every direct-control action and save you round trips.

Capture the screen after an action

Add capture to any action to get the screen as it is right after the action, so you skip a separate /state call:

{ "index": 12, "capture": true }

capture: true returns the element tree with default options. Pass an object to change them:

Option	Effect
`ui`	Include the element tree. Default `true`.
`screenshot`	Include `screenshot_url`, a link to `GET …/screenshot`. See the note below.
`verbosity`	`compact` (drops indentation and non-interactive elements, for far fewer tokens), `default` (the tree Lusor’s own agent reads) or `full` (every flag).
`only`	Keep only `clickable`, `focused` or `editable` elements. `all` keeps everything.
`min_size_px`	Drop elements smaller than this many pixels.
`include_keyboard`	Include the on-screen keyboard’s elements. Default `false`.
`max_depth`	Limit the depth of the tree, 1 to 50.

{ "index": 12, "capture": { "verbosity": "compact", "only": "clickable" } }

The response then carries a capture object:

Field	Present when	Meaning
`elements`	`ui` is on	The element tree as text, with `[N]` indices you can tap.
`element_count`	`ui` is on	How many indexed elements the tree has.
`screen_dimensions`	`ui` is on	`width` and `height`.
`screenshot_url`	`screenshot` is on	A link to `GET …/screenshot`.

screenshot_url takes a new screenshot each time you fetch it. It shows the screen at fetch time, not the moment after the action, so fetch it straight away, before the next action. It needs the same API key, and on a Lusor Phone it is metered like any other GET …/screenshot.

When the capture fails

The action has already run when the capture is taken. If the action succeeds but the capture fails, the response is still 200, with:

{ "ok": true, "capture": { "failed": true, "error": "…" } }

Do not retry the action, because it ran. Read the screen with GET …/state instead. On your own iPhone, a capture that asks for the element tree always fails this way. Inside a chain, a capture failure aborts the chain instead (see below).

Chains

POST …/chain runs 1 to 20 actions in order in one request.

curl https://api.lusor.ai/v1/workspaces/$WS/devices/$DEVICE/chain \
  -H "X-Api-Key: $LUSOR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "capture": "final",
    "steps": [
      { "verb": "start_app", "params": { "package": "com.android.settings" } },
      { "verb": "wait", "params": { "duration_ms": 1500 } },
      { "verb": "tap", "params": { "index": 4 } },
      { "verb": "type", "params": { "text": "battery" } },
      { "verb": "key", "params": { "keyCode": 66 } }
    ]
  }'

import os, requests

BASE = f"https://api.lusor.ai/v1/workspaces/{os.environ['WS']}/devices/{os.environ['DEVICE']}"
H = {"X-Api-Key": os.environ["LUSOR_API_KEY"]}

r = requests.post(f"{BASE}/chain", headers=H, json={
    "capture": "final",
    "steps": [
        {"verb": "start_app", "params": {"package": "com.android.settings"}},
        {"verb": "wait", "params": {"duration_ms": 1500}},
        {"verb": "tap", "params": {"index": 4}},
        {"verb": "type", "params": {"text": "battery"}},
        {"verb": "key", "params": {"keyCode": 66}},
    ],
})
body = r.json()
if r.ok and not body["ok"]:
    failed = [s for s in body["steps"] if not s["ok"]]
    print("failed steps:", failed)

const BASE = `https://api.lusor.ai/v1/workspaces/${process.env.WS}/devices/${process.env.DEVICE}`;

const r = await fetch(`${BASE}/chain`, {
  method: "POST",
  headers: { "X-Api-Key": process.env.LUSOR_API_KEY, "Content-Type": "application/json" },
  body: JSON.stringify({
    capture: "final",
    steps: [
      { verb: "start_app", params: { package: "com.android.settings" } },
      { verb: "wait", params: { duration_ms: 1500 } },
      { verb: "tap", params: { index: 4 } },
      { verb: "type", params: { text: "battery" } },
      { verb: "key", params: { keyCode: 66 } },
    ],
  }),
});
const body = await r.json();
if (r.ok && !body.ok) console.log(body.steps.filter((s) => !s.ok));

Steps

Each step is { "verb": …, "params": { … } }, with an optional capture next to params. The params are the same as the body of the matching action endpoint.

Verb	Same as
`tap`, `swipe`, `type`, `key`, `clear`, `back`, `home`, `recent`, `screenshot`	The action of the same name.
`start_app`, `stop_app`	`apps/start`, `apps/stop`.
`wait`	Pause for `duration_ms`, 0 to 60000.

Steps run back to back, with no pause between them. Add wait steps where the screen needs time to settle, for example after a tap that opens a new screen. All wait steps in one chain add up to at most 120 seconds.
capture at the top level is none (default), each or final. A step’s own capture overrides it, for example { "verb": "tap", "params": { "index": 4 }, "capture": true }.

Results

The response has ok and one entry in steps per step, in order. Each entry has the step’s verb, ok, exit_code and status, plus resolved (for a tap), cleared and capture when they apply.

When a step fails

There are two kinds of failure, and they behave differently.

A step completes with ok: false. The chain keeps going. The top-level ok is true only if every step succeeded. Check steps[n].ok to find the failure.
A step errors. The chain stops. This happens when the phone cannot be reached, when a step’s capture fails, or when billing refuses a later step (402 or 503 on a Lusor Phone). The response is that error alone, without per-step results, and the steps before it have already run on the phone.

After an error, you cannot tell from the response how far the chain got. Read the screen before you retry a chain.

On a Lusor Phone, a chain is metered before each step, so a 402 or 503 stops it before that step reaches the phone. See Billing.

Get started

Agents & sessions

Phones

Billing and limits

Chains and capture

Capture the screen after an action

When the capture fails

Chains

Steps

Results

When a step fails

​Capture the screen after an action

​When the capture fails

​Chains

​Steps

​Results

​When a step fails

Capture the screen after an action

When the capture fails

Chains

Steps

Results

When a step fails