# Neko Plugin

One-click install and control of a [Neko](https://github.com/m1k1o/neko) shared browser so **you and the agent can use the same browser**. Uses **Colima** + Mugi Homebrew (same as SearXNG). Default image is **Chromium** with CDP enabled; the agent drives it via the raw Chrome DevTools Protocol WebSocket (no Playwright runtime required for the Neko plugin itself — see [Agent control (CDP)](#agent-control-cdp)). All setup lives under `~/.mugi/neko/`. After install, the dashboard shows **Open in browser**, **Copy link**, **Audio**, and **Remove Neko**.

## Install (into Mugi)

1. Copy this folder to your plugins directory (re-run to get updates like the Remove button):
   ```bash
   cp -r templates/plugin_examples/neko ~/.mugi/plugins/neko
   ```
2. In Mugi: **Settings → Plugins**. Approve the "Neko" plugin (Trust & Enable).
3. (Optional) If Colima is not set up yet: **Settings → Network** → **Set up SearXNG (Docker)**. That installs Colima and Docker via Mugi Homebrew so Neko can use the same runtime.
4. Say "open Neko dashboard" or run the `open_neko_dashboard` tool. In the plugin dashboard, click **Install Neko**.
5. After install, use **Open in browser** or **Copy link** in the dashboard, or use the tools `get_neko_room_link` and `open_url_in_neko` from the agent.

## Requirements

- **Colima** (container runtime). Same as SearXNG. Run **Settings → Network → Set up SearXNG (Docker)** first if needed.
- All Neko state is under **`~/.mugi/neko/`** (compose file, etc.).
- **Default image: Chromium** (`ghcr.io/m1k1o/neko/chromium:latest`) so the agent can control the browser via CDP when enabled. Optional **Firefox** in plugin settings (Remove + Install to apply).

## Settings (Preferences → Plugins → Neko)

- **Neko room URL**: After install, default `http://localhost:8080`.
- **Use Firefox instead of Chromium**: Off by default. Enable and re-install if you prefer Firefox (no CDP agent control).
- **Chrome DevTools Protocol URL**: Default `http://localhost:9222`. Used by `neko_navigate`, `neko_click`, `neko_type`. The default install uses a CDP-enabled image; see **Agent control (CDP)** below.
- **Default room name**: Optional; for neko-rooms.

## Tools

- **open_neko_dashboard** — Open the Neko dashboard (Install, Open in browser, Copy link, Audio, Remove Neko).
- **install_neko** — Install and start the Neko container (Chromium by default).
- **get_neko_room_link** — Get the Neko room URL.
- **open_url_in_neko** — Tell the user to open a URL in Neko (or use `neko_navigate` if CDP is enabled).
- **neko_browser_snapshot** — Read-only list of the tabs Chromium has open (URL + title). Use this instead of `neko_navigate("about:blank")` to inspect state.
- **neko_page_visible_text** *(max_chars)* — Read visible text from the active tab (CDP). Capped at 120000 chars.
- **neko_navigate** *(url)* — Replace the active tab with `url`. Requires CDP.
- **neko_click_selector** *(css_selector)* — Click the first element matching a CSS selector via CDP.
- **neko_fill_field** *(css_selector, text, clear_first)* — Focus an input/textarea and type `text` via `Input.insertText`.
- **neko_insert_text** *(text)* — Insert Unicode text into whatever currently has focus.
- **neko_press_key** *(key)* — Send Enter, Tab, Escape, or Backspace to the focused element.
- **neko_wait_for** *(url_equals | url_prefix | selector | text, timeout_seconds)* — Wait for the page to reach a target state. Exactly one condition per call; timeout clamped to 1–60s; returns `ok:<kind>` or a stable `wait_timeout:<kind>` code.
- **neko_click** / **neko_type** — Legacy `xdotool` fallbacks (require shell access). Prefer the CDP variants.

### Golden path (copy/paste)

```
1. open_neko_dashboard              # install if needed; share the room link with the user
2. neko_navigate("https://example.com/")
3. neko_wait_for(url_prefix="https://example.com/", timeout_seconds=20)
4. neko_page_visible_text(max_chars=2000)   # bounded read for the assistant to quote
```

This is the "tight loop" the new `neko_wait_for` primitive is designed to replace: one navigate, one wait, one bounded read — instead of five `neko_click` + `neko_browser_snapshot` alternations while the page loads.

### Decision table: what to do when Neko misbehaves

| Symptom | Most likely cause | Next step |
|---------|-------------------|-----------|
| `Failed: cdp_unreachable (Could not reach CDP at …)` | Container not running or CDP port not published | `docker ps \| grep mugi-neko`; re-run install; check **neko_cdp_url** in plugin settings |
| `wait_timeout:selector=…` | Page rendered but the selector never appeared (shadow root / iframe / wrong selector) | Use `neko_page_visible_text` to confirm the text you expect is present; selectors do not cross shadow roots or iframes |
| `wait_timeout:url_prefix` | Redirect through SSO / consent / CAPTCHA — human login needed | Ask the user to complete the flow in the Neko room, then retry |
| `wait_timeout:text` | Dynamic content still loading | Give the page longer (up to 60s), or wait on the known container selector instead |
| `[Policy blocked] … unattended external` on `neko_*` | Webhook/cron path default-denies shared-browser automation (Phase B) | Set `MUGI_ALLOW_UNATTENDED_BROWSER_AUTOMATION=1` on the Mugi backend process if this is intentional |
| `[Policy blocked] … plan mode` on `neko_*` | Plan mode blocks mutations; browser driving counts as a mutation | Either turn off plan mode or describe what you would do instead |
| Neko dashboard stuck on teal circle | Colima/WebRTC UDP issue (see Troubleshooting) | Set **WebRTC NAT 1:1 IP** to your LAN IP and reinstall Neko |
| Chromium closed every tab | User closed the last tab | Call `neko_navigate` with an https URL before further automation |

### Upgrading the plugin after a Mugi update

Mugi ships the plugin as a template. After pulling repo changes, copy the updates into your installed plugin and re-approve:

```bash
cp -r templates/plugin_examples/neko ~/.mugi/plugins/neko
```

Then: **Settings → Plugins → Reload Plugins**, re-approve the Neko plugin if Mugi prompts (the hash changes on edit). New tools (e.g. `neko_wait_for`) only show up after the reload.

## One-click install flow

1. Enable the Neko plugin.
2. (If needed) Set up Colima once: **Settings → Network → Set up SearXNG (Docker)**. Neko uses the same Colima VM.
3. Open the Neko dashboard (e.g. "open Neko dashboard" or run `open_neko_dashboard`).
4. Click **Install Neko** in the dashboard. The plugin will:
   - Ensure Colima is running (same as SearXNG)
   - Write `~/.mugi/neko/docker-compose.yml` (Chromium by default; WebRTC single-port mux for Colima)
   - Run `docker compose pull` then `docker compose up -d` using Mugi Homebrew's docker
   - Mark the plugin as installed and show the full controls (Open in browser, Copy link, Audio, Remove Neko)
5. Use the dashboard or tools to get the room link and share it.

## Troubleshooting

- **Stuck on the n.eko logo / teal circle (looks like a frozen loading screen)**  
  That screen is Neko’s WebRTC “connect” UI. If it never becomes the desktop, ICE/WebRTC failed — not a Mugi bug in the UI. Common on **Colima** because UDP port forwarding to `127.0.0.1` is unreliable.

  1. **Preferences → Plugins → Neko → WebRTC NAT 1:1 IP**  
     Leave **empty** to auto-detect your Mac’s primary IPv4 (recommended). Or set it explicitly to the same IP shown under **System Settings → Network** for your active interface (e.g. `192.168.x.x`). Neko’s [troubleshooting](https://neko.m1k1o.net/docs/v3/troubleshooting) recommends a **local LAN IP** for local-only use; hard-coding `127.0.0.1` often breaks WebRTC through Docker/Colima.
  2. Regenerate compose: **Remove Neko** in the dashboard, then **Install Neko** again (or re-run `install_neko`) so `NEKO_WEBRTC_NAT1TO1` updates.
  3. Check what Neko thinks your IP is: `docker logs mugi-neko 2>&1 | grep nat_ips` — if it’s wrong, set **WebRTC NAT 1:1 IP** manually.
  4. **Colima UDP issues**: try starting Colima with `LIMA_SSH_PORT_FORWARDER=true colima start`, or `colima start --network-address` and use the shown address if docs recommend it. See [Colima port forwarding issues](https://github.com/abiosoft/colima/issues).

- **Stuck on splash or "Reconnecting"** after you were already connected: same WebRTC path — check logs, NAT IP, and Colima as above.

- **General checks**:
  1. `docker logs mugi-neko` for WebRTC / `invalid 1:1 NAT` / UDP errors.
  2. Confirm Neko works in your environment: run the [official compose](https://neko.m1k1o.net/docs/v3/installation/examples) in a separate folder. If that also fails, the issue is likely Colima/UDP or macOS firewall, not the plugin.
  3. After changing the plugin, copy updates from `templates/plugin_examples/neko` to `~/.mugi/plugins/neko` and re-approve if needed.

## Agent control (CDP)

Every `neko_*` interaction tool (`neko_navigate`, `neko_click_selector`, `neko_fill_field`, `neko_insert_text`, `neko_press_key`, `neko_wait_for`, `neko_browser_snapshot`, `neko_page_visible_text`) connects to the Neko Chromium browser over the raw Chrome DevTools Protocol WebSocket. **Playwright is not required** for these tools — the Neko plugin talks CDP directly. The separate `tools/playwright_screenshot.py` helper (Settings → Advanced → Install Playwright) is only used for webpage screenshots by other tools, not by this plugin.

When you **Install Neko** with the default (Chromium), the plugin builds a CDP-enabled image (`mugi-neko-chromium-cdp:latest`) from the stock Neko Chromium image by running Chromium with `--remote-debugging-port=9223` and adding a `socat` forwarder so the host can reach CDP at `0.0.0.0:9222`. Port 9222 is exposed on the container and **neko_cdp_url** defaults to `http://localhost:9222`.

**When CDP connection fails** (`Failed: cdp_unreachable …`): the default install already wires CDP — do not suggest "use a CDP-enabled image" or "start Neko with `--remote-debugging-port=9222`" (the plugin already does that). Instead, check:

1. **Container running** — `docker ps | grep mugi-neko` or open the Neko dashboard.
2. **`neko_cdp_url`** setting — Preferences → Plugins → Neko. Default `http://localhost:9222`.
3. **Firefox mode** — if the user switched to Firefox in plugin settings, CDP is not available (the Firefox image has no CDP); switch back to Chromium and re-install.

### Unattended automation (webhooks / cron)

Phase B default-denies shared-browser automation from unattended contexts. See `docs/harness_threat_model.md` §14a for the full rationale. If a webhook or cron-driven flow should be allowed to drive Neko, set the env var on the Mugi backend process:

```bash
export MUGI_ALLOW_UNATTENDED_BROWSER_AUTOMATION=1
```

The exemption covers only the `BROWSER_UI_AUTOMATION` capability; xdotool-based fallbacks (`neko_click`, `neko_type`) still require shell access on their own.
