Skip to content

Marking up what you send

“Make this bigger and the blue one darker” is a sentence an agent cannot act on. So you write a paragraph describing where things are, or you send a screenshot and hope the agent finds what you meant.

Offsite puts a small editor in front of every image you send. Draw a shape, Offsite numbers it, you write one line per number. The agent gets the marked picture and the notes as text (“1: make this bigger, 2: dark blue”), with the numbers on the image matching the numbers in the list.

The annotation editor on the Mac over a checkout page: a circle around the Pay button, an arrow to the total, and a line for each number 0:54
Three files dropped on an agent's card, the screenshot circled and numbered in the annotation editor, the batch uploaded to a Linux host in the cloud, and the paths and notes in the prompt; then the same from the iPhone. A 54-second film, with sound.

The editor sits in front of the upload, on every path that sends an image:

You do thisOn
Drop images onto an agent cardmacOS
Upload Files from the card’s action pillmacOS
Upload from the command bar: Photo Library, Camera, FilesiPhone, iPad
Mark an area in a Browser or Agent Browser cardmacOS, iPhone, iPad

Non-images skip it entirely and upload as before.

Pick a tool (circle, rectangle, arrow or freehand) and drag it over the part you mean. A click without dragging drops the shape at default size. Every shape gets a numbered dot in its centre, in the order you drew them; delete one and the rest renumber, so the agent never reads a list with gaps. Each shape takes one of five colours, so a marker stays visible on any picture.

Blur hides what nobody should read, a token or a name: it pixelates the region in the image. A blurred region gets no number and no comment, and the text block leaves it out.

On the Mac, the comments sit in a panel beside the picture, one field per marker. On the iPhone, tap the speech bubble in a marker’s centre to write its line. A marker with a comment turns green in the editor, so you can see at a glance what you have already described.

The first field is marker 0: the whole image (on the iPhone, the bubble on the image’s thumbnail). Use it for the sentence that applies to everything, like “home screen after login, dark mode”, with or without any shapes at all.

Sending several images at once puts a strip of thumbnails in the editor. Each image keeps its own shapes and its own comments; switching between them saves as you go. Then Insert (on the Mac Insert 3 for three images, or ⌘Return) puts the batch into the session.

On macOS: V for select, 1 to 5 for circle, rectangle, arrow, freehand and blur, ⌘Z / ⇧⌘Z, ⌫ to delete the selected shape, ← / → to walk through the images, ⌘Return to insert.

The shapes are burned into the image at full resolution, and the comments travel as a plain block of text that is pasted into the session next to the file paths:

===== Attached images =====
A detail from a page in my browser, plus the page around it.
2 images; the numbered markers in each picture match the notes below it.
Image 1: browser-mark-….png
/…/uploads/s-…/3f9c2a1b-browser-mark-….png
Overall: home screen after login, dark mode
Marker 1: make this bigger
Marker 2: dark blue
Marker 3: (marked — see the picture)
Image 2: …
/…/uploads/s-…/…png
(for reference, no notes)
Where this comes from:
Page: Bloom — http://localhost:5173/
Image 1 is the part I marked, 420×260 at (64,180) in the viewport. Image 2 shows the whole viewport so you can place it.
Selector: main > section.featured
To see it yourself: web_open http://localhost:5173/, then web_click main > section.featured … in your Agent Browser.
===== end of attached images =====

The block is fenced so an agent reads it as an attachment rather than as a truncated prompt. And when the image came from a browser card, it carries the page it came from plus a line telling the agent how to open that page in its own browser.

Nothing is submitted. The text lands in the session’s prompt, and you press Enter, as with every other upload.

If you add no comment anywhere in the batch, no text block is generated at all: you get the plain paths, the way an upload has always worked. Your drawings are still burned into the image.

Settings → Annotate images before sending, on macOS and on iPhone. Off, every entry point uploads directly again. The editor has the same switch, Use annotation, so you can turn it off in the moment it gets in your way.

  • Images are PNG, JPEG, HEIC, WebP or TIFF. Anything else never sees the editor.
  • The marked image is the one that travels: what you see in the editor is the file the agent gets.
  • Images larger than 2048 px on the long edge are scaled down after the shapes are burned in, so the numbers stay sharp while a 10 MB retina screenshot does not have to travel.
  • Shapes are stored in normalized coordinates and rendered by the same code that draws the preview and the final file, so what you see is what the agent gets.
  • In the file the agent gets, marker numbers are white dots with a black border, black once commented. Those colours are fixed so they stay readable on any screenshot; the green of a commented marker belongs to the editor only.