Skip to content

Commit 4e4c57f

Browse files
author
VicarDa
authored
Merge pull request #118 from Core-Mate/codex/workbuddy-task-home-events
feat(workbuddy): editable task home and event-driven progress
2 parents 36028ef + c7b5196 commit 4e4c57f

38 files changed

Lines changed: 2009 additions & 267 deletions

‎README.md‎

Lines changed: 26 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -81,19 +81,39 @@ Model availability, pricing, and policy behavior vary by version and region. Whi
8181

8282
## Use OpenGUI in WorkBuddy
8383

84-
WorkBuddy has a separate [MCP + Skill + Hooks connector](./workbuddy-plugin/README.md#install-on-macos). It uses WorkBuddy's current visual model to operate Android phones and opens read-only scrcpy windows by default. No DSH installation, full OpenGUI backend, or extra model API key is required.
84+
On **macOS**, open WorkBuddy and send this in a new chat:
8585

86-
Version `0.3.1` is a macOS public-testing prerelease, not marketplace approval or complete real-device acceptance. It installs the MCP, `opengui` Skill and lifecycle Hooks together, preserves other plugins, and backs up the affected WorkBuddy configuration. WorkBuddy 5.5.6+ supports the live installation flow; older compatible versions use the Command-Q fallback reported by preflight.
86+
```text
87+
Install https://github.com/Core-Mate/OpenGUI into WorkBuddy
88+
```
8789

88-
After installation, enable/trust the `opengui` MCP if prompted, review the external Hook change in `/hooks`, confirm `opengui` in `/skills`, connect a USB-debugging-authorized Android phone, select `/opengui`, and send:
90+
You can also request installation in Chinese:
8991

9092
```text
91-
Open Settings and report the Android version on my phone.
93+
安装 https://github.com/Core-Mate/OpenGUI
9294
```
9395

94-
Phone tasks send screenshots to the selected model. For local viewing only, ask “Show my phone screens without taking screenshots for the model or operating the phones.” Task completion leaves the mirrors open. Closing a mirror does not cancel a running task; use WorkBuddy's stop control to stop execution.
96+
**For the installing assistant:** when this request comes from WorkBuddy, follow the [WorkBuddy installation guide](./workbuddy-plugin/INSTALL.md). Install the WorkBuddy MCP, `opengui` Skill and lifecycle Hooks together. Use the verified WorkBuddy release assets; the DSH installer and full backend bootstrap are separate options.
97+
98+
**WorkBuddy 5.6.2 compatibility:** the public 0.3.1 installer currently reports `HOST_HOOKS` because it checks an older CLI filename. The source installer has the compatibility fix, pending publication. Until then, the short prompt is an installation request, not a verified one-step install on 5.6.2; see the [known blocker and local candidate procedure](./workbuddy-plugin/INSTALL.md).
99+
100+
WorkBuddy must also authorize configuration writes when its file policy requests approval. With the tested default permissions, that step was refused and rolled back; a completed package download alone is not a completed installation.
101+
102+
The installer detects WorkBuddy, downloads and checks the package, prepares its private runtime, backs up affected settings, and preserves other plugins. You do not need to clone the repository, install Node.js yourself, deploy the OpenGUI backend, or install DSH. Use WorkBuddy **5.5.6 or newer** for installation while the app stays open; the installer reports when an older compatible host needs to quit first. Finish existing phone tasks and close their viewers before upgrading.
103+
104+
**After installation:**
105+
106+
1. Enable/trust the `opengui` MCP if WorkBuddy prompts you. Review the external Hook change in `/hooks` and confirm `opengui` in `/skills`. Open a new chat if the current one has not refreshed.
107+
2. Connect an Android phone by USB, enable USB debugging, and accept the authorization prompt on the phone. An Android emulator also works. Ask `@opengui List connected devices without operating them` to check the connection.
108+
3. Select the `opengui` Skill in WorkBuddy and try:
109+
110+
```text
111+
@opengui Open Settings, then return to the home screen. Stop after confirming the home screen is visible.
112+
```
113+
114+
4. View the phone and progress in WorkBuddy's right-hand panel. Follow any device/model selection or start confirmation shown by your installed version. Use the task's stop control to end execution; closing the preview alone does not stop a task.
95115

96-
If `/opengui` is missing or tasks do not continue automatically, see [installation checks and troubleshooting](./workbuddy-plugin/README.md#verify-the-installation). MCP alone does not install the Skill or Hooks.
116+
The downloadable [WorkBuddy 0.3.1 release](https://github.com/Core-Mate/OpenGUI/releases/tag/opengui-workbuddy-v0.3.1) is a public-testing prerelease. Features described in the [current development guide](./workbuddy-plugin/README.md), including the editable task home opened by a bare `@opengui`, may be newer than that immutable release. Phone tasks send screenshots to the selected execution model; local video preview does not send every frame to a model. See [installation checks and recovery](./workbuddy-plugin/INSTALL.md#verify-the-installation) if the Skill or tools are missing.
97117

98118
## Run the Full OpenGUI Stack
99119

‎README.zh-CN.md‎

Lines changed: 26 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -82,19 +82,39 @@ OpenGUI 正式支持 DSH `0.1.0-rc.7`、`0.1.0-rc.8`、`0.1.1-rc.1` 和 `0.1.1-r
8282

8383
## WorkBuddy 安装
8484

85-
WorkBuddy 使用独立的 [MCP + Skill + Hooks 插件](./workbuddy-plugin/README.zh-CN.md#macos-安装),由当前视觉模型看图操作 Android 手机,默认打开只读 scrcpy 投屏窗口。不需要安装 DSH、部署完整 OpenGUI 后端或额外配置模型 API Key。
85+
在 **macOS** 上打开 WorkBuddy,新建聊天,直接发送:
8686

87-
当前 `0.3.1` 是 macOS 公测预发布版,不代表已经上架市场或完成全部真机验收。安装会把 MCP、`opengui` Skill 和生命周期 Hooks 一起配置,备份相关 WorkBuddy 配置并保留其他插件。WorkBuddy 5.5.6 及以上支持运行中安装;旧版兼容宿主会由预检明确要求 Command-Q。
87+
```text
88+
安装 https://github.com/Core-Mate/OpenGUI
89+
```
8890

89-
安装后按宿主提示启用并信任 `opengui` MCP,在 `/hooks` 中审查外部 Hook 变更,并在 `/skills` 中确认 `opengui`。连接已授权 USB 调试的 Android 手机,在输入框选择 `/opengui`,发送:
91+
也可以明确指定安装目标:
9092

9193
```text
92-
打开手机设置,查看并告诉我 Android 版本。
94+
把 https://github.com/Core-Mate/OpenGUI 的 OpenGUI 插件安装到 WorkBuddy
9395
```
9496

95-
手机任务会把截图发送给当前模型。只想看投屏时,可以说“展示手机投屏,不截图给模型,也不要操作手机”。任务结束后投屏保留;关闭投屏窗口不会取消正在执行的任务,停止操作请使用 WorkBuddy 的停止按钮。
97+
**给执行安装的 AI:**如果当前宿主是 WorkBuddy,请按 [WorkBuddy 安装指南](./workbuddy-plugin/INSTALL.md)操作,使用经过校验的 WorkBuddy Release 安装器,同时安装 MCP、`opengui` Skill 和生命周期 Hooks。DSH 安装器和完整后端部署属于其他使用方式。
98+
99+
**WorkBuddy 5.6.2 兼容性:**公开 0.3.1 安装器仍检查旧版 CLI 文件名,会报 `HOST_HOOKS`;源码安装器已修复,但尚未发布。因此目前不能承诺 5.6.2 上仅凭这条短提示词就能安装完成。具体阻塞与本地候选版验证方式见[安装指南](./workbuddy-plugin/INSTALL.md)。
100+
101+
WorkBuddy 文件策略要求授权时,还需允许安装器写入配置。本次默认权限实测在该步被拒并回滚;仅下载或解压了安装包,不代表安装完成。
102+
103+
安装器会识别 WorkBuddy、下载并校验插件、准备独立运行环境、备份受影响的设置并保留其他插件。用户无需手动克隆仓库、安装 Node.js、部署 OpenGUI 后端或安装 DSH。建议使用 **WorkBuddy 5.5.6 及以上**,可保持应用打开完成安装;旧版兼容宿主会提示先退出。升级前请结束现有手机任务并关闭其预览页面。
104+
105+
**安装后这样用:**
106+
107+
1. 如果 WorkBuddy 弹出提示,启用并信任 `opengui` MCP;在 `/hooks` 中审查外部 Hook 变更,在 `/skills` 中确认 `opengui`。当前聊天未刷新时,新建聊天。
108+
2. 用 USB 连接 Android 手机,开启 USB 调试,并在手机上允许这台电脑调试;也支持 Android 模拟器。先发送 `@opengui 列出已连接设备,不要操作手机` 检查连接。
109+
3. 在 WorkBuddy 中选择 `opengui` 技能,试一个简单任务:
110+
111+
```text
112+
@opengui 打开手机设置,再返回桌面,确认回到桌面后结束。
113+
```
114+
115+
4. 在右侧面板查看手机画面和进度;如果所装版本展示设备、模型选择或开始确认,按页面提示完成。停止任务请使用任务的停止按钮,单纯关闭预览不会停止操作。
96116

97-
找不到 `/opengui` 或任务不能自动续跑时,查看 [安装验证与排查](./workbuddy-plugin/README.zh-CN.md#安装验证与排查)。只添加 MCP 不会自动安装技能和 Hooks。
117+
目前可下载的 [WorkBuddy 0.3.1](https://github.com/Core-Mate/OpenGUI/releases/tag/opengui-workbuddy-v0.3.1) 是公测预发布版。[当前开发版说明](./workbuddy-plugin/README.zh-CN.md)中的部分功能(例如只发 `@opengui` 打开可编辑任务首页)可能晚于该固定发布包。任务截图会发送给所选执行模型;本地视频预览不会把每帧发送给模型。找不到技能或工具时,参见[安装验证与恢复](./workbuddy-plugin/INSTALL.md#verify-the-installation)。
98118

99119
## 运行完整 OpenGUI 技术栈
100120

‎docs/preview.html‎

Lines changed: 287 additions & 0 deletions
Large diffs are not rendered by default.

‎workbuddy-plugin/INSTALL.md‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
# Install OpenGUI into WorkBuddy
2+
3+
Use this guide when a user asks WorkBuddy to install `https://github.com/Core-Mate/OpenGUI`. The target is the WorkBuddy connector: MCP + `opengui` Skill + lifecycle Hooks. Do not select the DSH installer or deploy `server/` and `client/` for this request.
4+
5+
## Requirements
6+
7+
- macOS on Apple Silicon or Intel, with WorkBuddy installed. Version 5.5.6+ supports live configuration; the installer checks compatibility and reports any restart requirement.
8+
- Network access to GitHub release assets and the runtime downloads used by the installer.
9+
- No phone is needed to install. USB debugging authorization is needed later to operate an Android phone.
10+
11+
Use the public **`opengui-workbuddy-v0.3.1`** prerelease below. Do not use GitHub's generic `/releases/latest`: this repository also publishes DSH and Codex plugins. Do not substitute a local development archive or claim unreleased features are included in the public package.
12+
13+
**Known compatibility blocker (verified 2026-10-06):** the published 0.3.1 installer checks only the older `dist/codebuddy.js` layout and fails with `HOST_HOOKS` on WorkBuddy 5.6.2. The current repository installer also checks `codebuddy-headless.js` and `codebuddy-lite-wb.mjs`, but that fix is not in the immutable release installer. Until a compatible installer is published, do not claim the short GitHub prompt completes installation on 5.6.2. Maintainers can validate a local candidate using the development-build procedure below; that is not a public-release installation test.
14+
15+
The source installer also avoids process enumeration for live-compatible WorkBuddy 5.5.6+ because the host command sandbox can deny `/bin/ps`. Older hosts still require the process check; the active OpenGUI broker check remains mandatory for every version. Do not disable sandbox protections or skip upgrade checks to install.
16+
17+
The development Hook now starts a broker only for an explicit skill invocation, rather than any text containing “OpenGUI”. Earlier local candidates can start the old broker merely from an installation prompt containing the repository URL. If that happens during an upgrade, stop the old tasks/viewers and disable the old MCP, allow the broker to exit, then continue the existing installation chat without invoking the phone Skill. If the host refuses configuration writes or cleanup, report the exact denied operation and use its normal authorization flow; do not weaken host security settings.
18+
19+
**Actual host authorization remains required:** on WorkBuddy 5.6.2 with the tested default permissions, the candidate reached `install-local.mjs` but the host refused the atomic configuration rename with `Brokered host rename source refused by file policy: prompt`. The transaction rolled back. Package download/extraction is not installation success. The user must complete the required WorkBuddy file authorization before configuration can be written; never describe this as an unattended install. See the [recorded prompt tests](docs/install-prompt-verification.zh-CN.md).
20+
21+
## Install from the public release
22+
23+
Download the installer and its adjacent SHA-256 file from the same release, verify before executing, then let the installer download and verify the package. A source checkout, npm install, sudo, and a system Node installation are unnecessary.
24+
25+
```bash
26+
(
27+
set -euo pipefail
28+
install_dir=$(mktemp -d "${TMPDIR:-/tmp}/opengui-workbuddy.XXXXXXXX")
29+
release_url='https://github.com/Core-Mate/OpenGUI/releases/download/opengui-workbuddy-v0.3.1'
30+
installer='opengui-workbuddy-0.3.1-install.command'
31+
cd "$install_dir"
32+
for asset in "$installer" "$installer.sha256"; do
33+
curl --proto '=https' --proto-redir '=https' --tlsv1.2 --fail --location \
34+
--connect-timeout 15 --max-time 240 --retry 2 "$release_url/$asset" -o "$asset"
35+
done
36+
shasum -a 256 -c "$installer.sha256"
37+
bash "$installer" --check
38+
bash "$installer"
39+
)
40+
```
41+
42+
The installer identifies the application and configuration directory, prepares private dependencies, and merges MCP/Skill/Hooks while preserving unrelated settings and backups. If multiple WorkBuddy applications are present, pass the intended absolute `.app` path with `--app` to both invocations. Do not guess another host's configuration directory.
43+
44+
For upgrades, finish existing OpenGUI tasks and close their viewers first. If the old broker is still active, disable the old OpenGUI MCP in WorkBuddy and wait for its normal idle exit before retrying. Do not force-kill host or phone processes or delete runtime locks to bypass this check. If preflight requests Command-Q, let the user finish other work and quit before continuing. A download or checksum failure is an installation failure, not permission to skip verification.
45+
46+
## Verify the installation
47+
48+
Report each stage separately:
49+
50+
1. **Configuration written:** the installer exits successfully and reports `CONFIG_WRITTEN`, `LIVE_CONFIG_WRITTEN`, or an already-configured result. This alone does not prove WorkBuddy loaded the plugin.
51+
2. **Host loaded:** enable/trust OpenGUI if prompted, review external Hook changes in `/hooks`, confirm `opengui` in `/skills`, and check that the OpenGUI MCP tools are available. A new chat may be needed to refresh discovery.
52+
3. **Device discovery:** ask `@opengui List connected devices without operating them`. An empty list means installation may be valid but no authorized device is connected; explain phone-side setup instead of declaring a device test passed.
53+
4. **First task:** after the user connects and authorizes a phone, try `@opengui Open Settings, then return to the home screen. Stop after confirming the home screen is visible.` Verify the right-hand preview and the actual result. Do not launch a phone task merely to complete installation unless the user requested it.
54+
55+
Keep trust, device authorization, and any account/model prompts visible to the user. Do not claim automatic phone authorization or complete business acceptance from an installer success message.
56+
57+
## Development builds
58+
59+
Maintainers may deliberately install a locally built archive using `scripts/install-macos.command --archive /absolute/path/opengui-mcp-0.3.1.tgz`; its adjacent `.sha256` is required. This is a separate test from installing the public GitHub release. Record the archive digest because local candidates currently share the `0.3.1` version label with the published release. Never overwrite immutable release assets to make these packages appear identical.
60+
61+
Current development behavior and data flow: [English](README.md) · [简体中文](README.zh-CN.md).

0 commit comments

Comments
 (0)