Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a4b27f991c | ||
|
|
809a7fb017 |
@@ -1,6 +1,3 @@
|
|||||||
__pycache__/
|
__pycache__/
|
||||||
*.pyc
|
*.pyc
|
||||||
*.log
|
*.log
|
||||||
shot_*.png
|
|
||||||
lc_mcp/job.ps1
|
|
||||||
lc_mcp/run.vbs
|
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
[submodule "vendor/aalc"]
|
||||||
|
path = vendor/aalc
|
||||||
|
url = https://github.com/KIYI671/AhabAssistantLimbusCompany
|
||||||
@@ -1,68 +1,92 @@
|
|||||||
# lc-mcp — Limbus Company 语义化操控 MCP server (跨平台)
|
# lc-mcp — 基于 AALC 的 Limbus Company MCP server
|
||||||
|
|
||||||
给 AI agent (codex / deepseek / 任意 MCP 客户端) 提供**语义化**游戏操控,
|
把 [AhabAssistantLimbusCompany (AALC)](https://github.com/KIYI671/AhabAssistantLimbusCompany)
|
||||||
解决"盲看屏幕猜坐标→思考慢、点错率高"。
|
的识别与操控能力包装成 MCP 工具,给 AI agent(codex / 任意 MCP 客户端)用。
|
||||||
|
|
||||||
## 核心设计
|
- **低层**:截图、RapidOCR、AALC 模板图匹配(几百张按钮/图标模板)、后台点击/按键,全部直接调 AALC 的 `module.automation`
|
||||||
|
- **高层**:`run_task` 直接跑 AALC 现成的完整流程(日常经验本/纽本、领奖励、狂气换体力、镜牢),自带战斗处理、掉线重试、游戏重启
|
||||||
|
|
||||||
AI 不用猜坐标。工具层做"文本化 + 语义点击":
|
AALC 以 git submodule 引入(`vendor/aalc`,固定到某个 commit),不改它的代码,升级时只需更新 submodule 指针。
|
||||||
|
|
||||||
|
## 架构
|
||||||
|
|
||||||
```
|
```
|
||||||
get_state() -> 截图+远端OCR:全屏文字元素 + 坐标
|
MCP 客户端 ──ssh──> python lc_mcp/bridge.py (stdio, 只用标准库)
|
||||||
click_text("开始") -> 内部 OCR 定位"开始"并点击中心
|
│ 127.0.0.1:8765, 首行 token 认证
|
||||||
wait_for_text("结算") -> 轮询直到文字出现/消失
|
▼
|
||||||
|
python -m lc_mcp.server ← 登录时由计划任务在「交互桌面会话」启动
|
||||||
|
├─ AalcEngine → AALC auto / ocr / screen (找窗口、后台截图、PostMessage 输入)
|
||||||
|
└─ TaskRunner → 子进程 python -m lc_mcp.aalc_task <task> (停止 = 杀进程树)
|
||||||
```
|
```
|
||||||
|
|
||||||
任务循环: `get_state -> 决策 -> click_text -> wait_for_text -> 确认 -> 下一步`
|
为什么要常驻服务 + bridge:SSH 登录的进程跑在 session 0,看不到游戏窗口,也发不了输入。
|
||||||
|
旧版靠"每次截图注册一个计划任务"绕过,慢且不能输入;现在整个服务常驻在桌面会话里。
|
||||||
|
|
||||||
## 平台矩阵
|
## 部署(Windows,跑游戏的那台机器)
|
||||||
|
|
||||||
| 平台 | 截图 | 输入后端 | 备注 |
|
```powershell
|
||||||
|------|------|----------|------|
|
git clone --recursive https://git.lingll.icu/yws/lc-mcp
|
||||||
| win | 交互计划任务 | `sendinput`(默认) / `interception`(物理HID级) | interception 需装驱动+重启 |
|
cd lc-mcp\vendor\aalc
|
||||||
| linux | X11 PIL / scrot | `xdotool` / `uinput`(内核虚拟设备,物理级) | uinput 需 root 或 uinput 组 |
|
uv sync # 按 AALC 的 pyproject 装依赖 (Python >= 3.12),生成 .venv
|
||||||
| mac | screencapture | `cg`(CGEvent, PyObjC) | 需辅助功能权限 |
|
# 先用 AALC 自己的界面 (uv run main.py) 配好队伍、镜牢设置等,lc-mcp 读同一个 config.yaml
|
||||||
|
cd ..\..
|
||||||
|
powershell -ExecutionPolicy Bypass -File scripts\install_logon_task.ps1
|
||||||
|
```
|
||||||
|
|
||||||
`LCP_PLATFORM=win|linux|mac` 强制指定,默认按系统自动检测。
|
`install_logon_task.ps1` 会注册 `LcMCPServer` 计划任务(登录时启动、普通权限、失败自动重启)。
|
||||||
|
游戏以管理员身份运行时才需要加 `-Elevated`。卸载:`-Uninstall`。
|
||||||
|
|
||||||
OCR 全本地:`ocr_local.py` 调 Windows 自带 OCR(Windows.Media.Ocr,需 `pip install --user winsdk`)。
|
## MCP 客户端配置
|
||||||
无任何云端/远端服务依赖——MCP 是纯本地"外骨骼"。
|
|
||||||
|
|
||||||
## 部署
|
|
||||||
|
|
||||||
1. 拷贝 `lc-mcp-dev/` 到目标机器
|
|
||||||
2. 装依赖(只用标准库;linux xdotool 后端需 `apt install xdotool`;mac cg 需 `pip install pyobjc-framework-Quartz`)
|
|
||||||
3. 启动:`python lc_mcp.py`(stdio),MCP 客户端 spawn
|
|
||||||
|
|
||||||
```jsonc
|
```jsonc
|
||||||
// codex MCP 配置 (codex/mcp.json 或项目 .mcp.json)
|
|
||||||
{
|
{
|
||||||
"mcpServers": {
|
"mcpServers": {
|
||||||
"limbus": {
|
"limbus": {
|
||||||
"type": "stdio",
|
"type": "stdio",
|
||||||
"command": "ssh",
|
"command": "ssh",
|
||||||
"args": ["win", "python", "C:/Users/yws-hhh/lc-mcp-dev/lc_mcp.py"],
|
"args": ["win", "python", "C:/Users/<you>/lc-mcp/lc_mcp/bridge.py"]
|
||||||
"env": { "INPUT_BACKEND": "interception" }
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
|
bridge 读取 `~/lc_mcp/token` 认证(服务首次启动时生成),所以 SSH 用户必须和登录桌面的用户是同一个账号。
|
||||||
|
服务没起来时 bridge 仍会完成握手,并对每个调用返回“服务未运行”的原因。
|
||||||
|
|
||||||
|
本机直接用(客户端本身就在桌面会话里):`python -m lc_mcp.server --stdio`。
|
||||||
|
|
||||||
|
## 工具
|
||||||
|
|
||||||
|
| 工具 | 说明 |
|
||||||
|
|------|------|
|
||||||
|
| `get_state` | 截图 + OCR,返回窗口内全部文字 `{text,x,y,w,h,score}` |
|
||||||
|
| `click_text(text, index)` | 按文字点击;忽略空格/大小写,完全匹配优先 |
|
||||||
|
| `wait_for_text` / `wait_for_text_gone` | 轮询 OCR 等文字出现/消失(上限 300s) |
|
||||||
|
| `find_image` / `click_image(name, threshold, model)` | AALC 模板图匹配,适合没有文字的按钮 |
|
||||||
|
| `list_images(prefix)` | 列出模板图,如 `battle/`、`mirror/` |
|
||||||
|
| `screenshot` | 窗口截图(`~/lc_mcp/shots`,只保留最近 30 张) |
|
||||||
|
| `click(x,y,times)` / `keypress` / `type_text` | 低层输入(窗口客户区坐标) |
|
||||||
|
| `run_task(task, …)` | `all` / `daily(exp,thread)` / `reward` / `enkephalin(times)` / `mirror(count)` |
|
||||||
|
| `task_status(lines)` / `stop_task` | 任务状态、日志尾部(`~/lc_mcp/runs`)、强制停止 |
|
||||||
|
|
||||||
|
任务运行期间,点击/按键类工具会被拒绝(防止打乱 AALC 的流程),只读工具仍可用。
|
||||||
|
`run_task` 传入的次数只对本次运行生效,结束后恢复 `config.yaml` 原值。
|
||||||
|
|
||||||
## 环境变量
|
## 环境变量
|
||||||
|
|
||||||
| 变量 | 默认 | 说明 |
|
| 变量 | 默认 | 说明 |
|
||||||
|------|------|------|
|
|------|------|------|
|
||||||
| `LCP_PLATFORM` | 自动 | win/linux/mac |
|
| `AALC_DIR` | `vendor/aalc` | 使用其他位置的 AALC 源码 |
|
||||||
| `INPUT_BACKEND` | 平台默认 | mock/sendinput/interception/xdotool/uinput/cg |
|
| `LCP_PORT` | 8765 | 服务/bridge 端口 |
|
||||||
| `OCR_BACKEND` | local | 本地 OCR(仅 local;Win 需 winsdk) |
|
| `LCP_KEEP_SHOTS` | 30 | 截图保留张数 |
|
||||||
| `OCR_MIN_SCORE` | 0.5 | 过滤低置信度 OCR 结果 |
|
|
||||||
|
|
||||||
## 工具
|
## 开发 / 测试
|
||||||
|
|
||||||
get_state · click_text · wait_for_text · wait_for_text_gone · screenshot · click · move · keypress · type_text
|
```bash
|
||||||
|
python -m unittest discover -s tests # 不需要 Windows/AALC (FakeEngine)
|
||||||
|
python -m lc_mcp.server --stdio --fake # 假引擎手动调试协议
|
||||||
|
```
|
||||||
|
|
||||||
## 安全提醒
|
## 许可证
|
||||||
|
|
||||||
- 输入会真实操作你已登录的桌面,跑任务时别动鼠标
|
AALC 使用 AGPL-3.0。本项目会导入并运行 AALC 的代码,分发时需遵守 AGPL-3.0。
|
||||||
- win 的截图走交互计划任务(Registry + wscript),杀软可能告警
|
|
||||||
- interception/uinput 是"物理化"输入,用于降低虚拟输入痕迹;真·硬件级可外接 Arduino/RPi Pico HID
|
|
||||||
|
|||||||
@@ -1,35 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
"""输入后端接口。每种后端实现同一套动作:click/move/keypress/type_text。"""
|
|
||||||
from abc import ABC, abstractmethod
|
|
||||||
|
|
||||||
|
|
||||||
class InputBackend(ABC):
|
|
||||||
|
|
||||||
@abstractmethod
|
|
||||||
def click(self, x, y, button='left', clicks=1):
|
|
||||||
...
|
|
||||||
|
|
||||||
@abstractmethod
|
|
||||||
def move(self, x, y):
|
|
||||||
...
|
|
||||||
|
|
||||||
@abstractmethod
|
|
||||||
def keypress(self, key):
|
|
||||||
...
|
|
||||||
|
|
||||||
@abstractmethod
|
|
||||||
def type_text(self, text):
|
|
||||||
...
|
|
||||||
|
|
||||||
|
|
||||||
def backend_factory(name, **kwargs):
|
|
||||||
if name == 'mock':
|
|
||||||
from .mock import MockBackend
|
|
||||||
return MockBackend(**kwargs)
|
|
||||||
if name == 'sendinput':
|
|
||||||
from .sendinput_win import SendInputBackend
|
|
||||||
return SendInputBackend(**kwargs)
|
|
||||||
if name == 'interception':
|
|
||||||
from .interception_win import InterceptionBackend
|
|
||||||
return InterceptionBackend(**kwargs)
|
|
||||||
raise ValueError('unknown backend: %s' % name)
|
|
||||||
@@ -1,64 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
"""macOS 输入后端:Quartz CGEvent (需 PyObjC: pip install pyobjc-framework-Quartz)。"""
|
|
||||||
import time
|
|
||||||
|
|
||||||
|
|
||||||
class CGBackend:
|
|
||||||
|
|
||||||
def __init__(self):
|
|
||||||
try:
|
|
||||||
import Quartz
|
|
||||||
self.q = Quartz
|
|
||||||
except ImportError:
|
|
||||||
raise RuntimeError('PyObjC Quartz not installed (pip install pyobjc-framework-Quartz)')
|
|
||||||
|
|
||||||
def _post(self, event):
|
|
||||||
self.q.CGEventPost(self.q.kCGHIDEventTap, event)
|
|
||||||
time.sleep(0.02)
|
|
||||||
|
|
||||||
def move(self, x, y):
|
|
||||||
e = self.q.CGEventCreateMouseEvent(None, self.q.kCGEventMouseMoved, (float(x), float(y)), self.q.kCGMouseButtonLeft)
|
|
||||||
self._post(e)
|
|
||||||
return {'ok': True}
|
|
||||||
|
|
||||||
def click(self, x, y, button='left', times=1):
|
|
||||||
btn = self.q.kCGMouseButtonLeft if button == 'left' else self.q.kCGMouseButtonRight
|
|
||||||
self.move(x, y)
|
|
||||||
for _ in range(times):
|
|
||||||
down = self.q.CGEventCreateMouseEvent(None, self.q.kCGEventLeftMouseDown if button == 'left' else self.q.kCGEventRightMouseDown, (float(x), float(y)), btn)
|
|
||||||
up = self.q.CGEventCreateMouseEvent(None, self.q.kCGEventLeftMouseUp if button == 'left' else self.q.kCGEventRightMouseUp, (float(x), float(y)), btn)
|
|
||||||
self._post(down)
|
|
||||||
self._post(up)
|
|
||||||
return {'ok': True}
|
|
||||||
|
|
||||||
def keypress(self, key):
|
|
||||||
mapping = {'enter': 36, 'esc': 53, 'space': 49, 'tab': 48, 'f4': 96,
|
|
||||||
'ctrl': 59, 'shift': 56, 'alt': 58, 'backspace': 51}
|
|
||||||
k = key.lower()
|
|
||||||
if k in mapping:
|
|
||||||
self._key(mapping[k])
|
|
||||||
return {'ok': True}
|
|
||||||
if len(k) == 1 and 'a' <= k <= 'z':
|
|
||||||
# mac 虚拟键码: a=0, b=11, ... 非连续, 用简单表
|
|
||||||
table = {'a': 0, 'b': 11, 'c': 8, 'd': 2, 'e': 14, 'f': 3, 'g': 5, 'h': 4,
|
|
||||||
'i': 34, 'j': 38, 'k': 40, 'l': 37, 'm': 46, 'n': 45, 'o': 31,
|
|
||||||
'p': 35, 'q': 12, 'r': 15, 's': 1, 't': 17, 'u': 32, 'v': 9,
|
|
||||||
'w': 13, 'x': 7, 'y': 16, 'z': 6}
|
|
||||||
if k in table:
|
|
||||||
self._key(table[k])
|
|
||||||
return {'ok': True}
|
|
||||||
return {'ok': False, 'error': 'unsupported key %s' % key}
|
|
||||||
|
|
||||||
def _key(self, vk):
|
|
||||||
down = self.q.CGEventCreateKeyboardEvent(None, vk, True)
|
|
||||||
up = self.q.CGEventCreateKeyboardEvent(None, vk, False)
|
|
||||||
self._post(down)
|
|
||||||
self._post(up)
|
|
||||||
|
|
||||||
def type_text(self, text):
|
|
||||||
for ch in text:
|
|
||||||
r = self.keypress(ch)
|
|
||||||
if not r.get('ok'):
|
|
||||||
return r
|
|
||||||
time.sleep(0.03)
|
|
||||||
return {'ok': True}
|
|
||||||
@@ -1,195 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
"""Interception 物理输入后端(Windows + Interception 驱动)。
|
|
||||||
|
|
||||||
输入在驱动层转为真实 HID 事件(对应用透明,非 SendInput 虚拟输入)。
|
|
||||||
需要: install-interception.exe /install 后重启系统,且本 dll 与驱动共存。
|
|
||||||
"""
|
|
||||||
import ctypes
|
|
||||||
import time
|
|
||||||
|
|
||||||
from ctypes import wintypes
|
|
||||||
|
|
||||||
# ---- Interception 常量 ----
|
|
||||||
MOUSE_LEFTBUTTON_DOWN = 0x001
|
|
||||||
MOUSE_LEFTBUTTON_UP = 0x002
|
|
||||||
MOUSE_RIGHTBUTTON_DOWN = 0x004
|
|
||||||
MOUSE_RIGHTBUTTON_UP = 0x008
|
|
||||||
MOUSE_MIDDLEBUTTON_DOWN = 0x010
|
|
||||||
MOUSE_MIDDLEBUTTON_UP = 0x020
|
|
||||||
|
|
||||||
MOUSE_MOVE_RELATIVE = 0x000
|
|
||||||
MOUSE_MOVE_ABSOLUTE = 0x001
|
|
||||||
MOUSE_VIRTUAL_DESKTOP = 0x002
|
|
||||||
|
|
||||||
KEY_DOWN = 0x00
|
|
||||||
KEY_UP = 0x01
|
|
||||||
KEY_E0 = 0x02
|
|
||||||
|
|
||||||
# ---- Set 1 扫描码 ----
|
|
||||||
SCAN_ALPHA = {chr(c): 0x1E + i for i, c in enumerate(range(ord('A'), ord('Z') + 1))}
|
|
||||||
# 上面的映射只在 A-P 连续,Q-Z 不连续,改为显式表
|
|
||||||
SCAN = {
|
|
||||||
'a': 0x1E, 'b': 0x30, 'c': 0x2E, 'd': 0x20, 'e': 0x12, 'f': 0x21,
|
|
||||||
'g': 0x22, 'h': 0x23, 'i': 0x17, 'j': 0x24, 'k': 0x25, 'l': 0x26,
|
|
||||||
'm': 0x32, 'n': 0x31, 'o': 0x18, 'p': 0x19, 'q': 0x10, 'r': 0x13,
|
|
||||||
's': 0x1F, 't': 0x14, 'u': 0x16, 'v': 0x2F, 'w': 0x11, 'x': 0x2D,
|
|
||||||
'y': 0x15, 'z': 0x2C,
|
|
||||||
'1': 0x02, '2': 0x03, '3': 0x04, '4': 0x05, '5': 0x06,
|
|
||||||
'6': 0x07, '7': 0x08, '8': 0x09, '9': 0x0A, '0': 0x0B,
|
|
||||||
'enter': 0x1C, 'return': 0x1C, 'esc': 0x01, 'space': 0x39, 'tab': 0x0F,
|
|
||||||
'ctrl': 0x1D, 'alt': 0x38, 'lshift': 0x2A, 'rshift': 0x36, 'shift': 0x2A,
|
|
||||||
'f1': 0x3B, 'f2': 0x3C, 'f3': 0x3D, 'f4': 0x3E, 'f5': 0x3F, 'f6': 0x40,
|
|
||||||
'f7': 0x41, 'f8': 0x42, 'f9': 0x43, 'f10': 0x44, 'f11': 0x57, 'f12': 0x58,
|
|
||||||
',': 0x33, '.': 0x34, '/': 0x35, ';': 0x27, "'": 0x28, '[': 0x1A,
|
|
||||||
']': 0x1B, '-': 0x0C, '=': 0x0D, '`': 0x29, '\\': 0x2B,
|
|
||||||
}
|
|
||||||
UPPER_SCAN = { # 大写所需符号的扫描码(按住 Shift 时)
|
|
||||||
'!': 0x02, '@': 0x03, '#': 0x04, '$': 0x05, '%': 0x06, '^': 0x07,
|
|
||||||
'&': 0x08, '*': 0x09, '(': 0x0A, ')': 0x0B, '_': 0x0C, '+': 0x0D,
|
|
||||||
'{': 0x1A, '}': 0x1B, ':': 0x27, '"': 0x28, '<': 0x33, '>': 0x34,
|
|
||||||
'?': 0x35, '~': 0x29, '|': 0x2B,
|
|
||||||
}
|
|
||||||
|
|
||||||
MAX_DEVICES = 64 # 键盘下游设备数量通常 <= 16,鼠标 <= 16;放宽到 64 覆盖异常
|
|
||||||
|
|
||||||
|
|
||||||
class MouseStroke(ctypes.Structure):
|
|
||||||
_fields_ = [("state", ctypes.c_ushort), ("flags", ctypes.c_ushort),
|
|
||||||
("rolling", ctypes.c_short), ("x", ctypes.c_ulong),
|
|
||||||
("y", ctypes.c_ulong), ("information", ctypes.c_ulong)]
|
|
||||||
|
|
||||||
|
|
||||||
class KeyStroke(ctypes.Structure):
|
|
||||||
_fields_ = [("code", ctypes.c_ushort), ("state", ctypes.c_ushort),
|
|
||||||
("information", ctypes.c_ulong)]
|
|
||||||
|
|
||||||
|
|
||||||
class InterceptionBackend:
|
|
||||||
|
|
||||||
def __init__(self, dll_path=None):
|
|
||||||
if dll_path is None:
|
|
||||||
dll_path = r'C:\Program Files\Interception\interception.dll'
|
|
||||||
try:
|
|
||||||
self.dll = ctypes.CDLL(dll_path)
|
|
||||||
except OSError:
|
|
||||||
# 兜底:尝试系统 PATH / 常见目录
|
|
||||||
self.dll = ctypes.CDLL('interception.dll')
|
|
||||||
self._api()
|
|
||||||
self._user32 = ctypes.WinDLL('user32', use_last_error=True)
|
|
||||||
self.ctx = self.dll.interception_create_context()
|
|
||||||
if not self.ctx:
|
|
||||||
raise RuntimeError('interception_create_context failed (驱动未安装/未重启?)')
|
|
||||||
self._kbd_devs, self._mou_devs = self._discover()
|
|
||||||
self._scr_w = self._user32.GetSystemMetrics(0)
|
|
||||||
self._scr_h = self._user32.GetSystemMetrics(1)
|
|
||||||
|
|
||||||
def _api(self):
|
|
||||||
d = self.dll
|
|
||||||
d.interception_create_context.restype = ctypes.c_void_p
|
|
||||||
d.interception_create_context.argtypes = []
|
|
||||||
d.interception_destroy_context.argtypes = [ctypes.c_void_p]
|
|
||||||
d.interception_open_device.restype = ctypes.c_bool
|
|
||||||
d.interception_open_device.argtypes = [ctypes.c_void_p, ctypes.c_int]
|
|
||||||
d.interception_close_device.argtypes = [ctypes.c_void_p, ctypes.c_int]
|
|
||||||
d.interception_get_hardware_id.restype = ctypes.c_int
|
|
||||||
d.interception_get_hardware_id.argtypes = [ctypes.c_void_p, ctypes.c_int,
|
|
||||||
ctypes.c_char_p, ctypes.c_int]
|
|
||||||
d.interception_send.restype = ctypes.c_int
|
|
||||||
d.interception_send.argtypes = [ctypes.c_void_p, ctypes.c_int,
|
|
||||||
ctypes.c_void_p, ctypes.c_size_t]
|
|
||||||
|
|
||||||
def _discover(self):
|
|
||||||
kbd, mou = [], []
|
|
||||||
buf = ctypes.create_string_buffer(256)
|
|
||||||
for dev in range(1, MAX_DEVICES + 1):
|
|
||||||
if not self.dll.interception_open_device(self.ctx, dev):
|
|
||||||
continue
|
|
||||||
n = self.dll.interception_get_hardware_id(self.ctx, dev, buf, 256)
|
|
||||||
hid = buf.value.decode('ascii', 'replace').lower() if n > 0 else ''
|
|
||||||
if 'keyboard' in hid:
|
|
||||||
kbd.append(dev)
|
|
||||||
elif 'mouse' in hid:
|
|
||||||
mou.append(dev)
|
|
||||||
return kbd, mou
|
|
||||||
|
|
||||||
# ---- 底层发送 ----
|
|
||||||
def _kbd_stroke(self, code, state):
|
|
||||||
ks = KeyStroke()
|
|
||||||
ks.code = ctypes.c_ushort(code)
|
|
||||||
ks.state = ctypes.c_ushort(state)
|
|
||||||
for dev in self._kbd_devs or [1]:
|
|
||||||
try:
|
|
||||||
self.dll.interception_send(self.ctx, dev, ctypes.byref(ks), ctypes.sizeof(ks))
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
def _mouse_stroke(self, state=0, flags=0, x=0, y=0):
|
|
||||||
ms = MouseStroke()
|
|
||||||
ms.state = ctypes.c_ushort(state)
|
|
||||||
ms.flags = ctypes.c_ushort(flags)
|
|
||||||
ms.x = ctypes.c_ulong(int(x))
|
|
||||||
ms.y = ctypes.c_ulong(int(y))
|
|
||||||
for dev in self._mou_devs or [17]:
|
|
||||||
try:
|
|
||||||
self.dll.interception_send(self.ctx, dev, ctypes.byref(ms), ctypes.sizeof(ms))
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
# ---- 公开动作 ----
|
|
||||||
def move(self, x, y):
|
|
||||||
nx = int(int(x) * 65535 / self._scr_w) if self._scr_w else 0
|
|
||||||
ny = int(int(y) * 65535 / self._scr_h) if self._scr_h else 0
|
|
||||||
self._mouse_stroke(state=0, flags=MOUSE_MOVE_ABSOLUTE, x=nx, y=ny)
|
|
||||||
time.sleep(0.03)
|
|
||||||
return {'ok': True, 'backend': 'interception'}
|
|
||||||
|
|
||||||
def click(self, x, y, button='left', clicks=1):
|
|
||||||
self.move(x, y)
|
|
||||||
time.sleep(0.05)
|
|
||||||
if button == 'left':
|
|
||||||
down, up = MOUSE_LEFTBUTTON_DOWN, MOUSE_LEFTBUTTON_UP
|
|
||||||
else:
|
|
||||||
down, up = MOUSE_RIGHTBUTTON_DOWN, MOUSE_RIGHTBUTTON_UP
|
|
||||||
for _ in range(max(1, int(clicks))):
|
|
||||||
self._mouse_stroke(state=down)
|
|
||||||
time.sleep(0.04)
|
|
||||||
self._mouse_stroke(state=up)
|
|
||||||
time.sleep(0.05)
|
|
||||||
return {'ok': True, 'clicked': [int(x), int(y)], 'backend': 'interception'}
|
|
||||||
|
|
||||||
def _scancode(self, key):
|
|
||||||
k = str(key).lower()
|
|
||||||
return SCAN.get(k)
|
|
||||||
|
|
||||||
def keypress(self, key):
|
|
||||||
k = str(key)
|
|
||||||
code = SCAN.get(k.lower())
|
|
||||||
if code is None:
|
|
||||||
return {'ok': False, 'error': 'unknown key %s' % key}
|
|
||||||
self._kbd_stroke(code, KEY_DOWN)
|
|
||||||
time.sleep(0.04)
|
|
||||||
self._kbd_stroke(code, KEY_UP)
|
|
||||||
time.sleep(0.02)
|
|
||||||
return {'ok': True, 'key': k, 'backend': 'interception'}
|
|
||||||
|
|
||||||
def type_text(self, text):
|
|
||||||
for ch in str(text):
|
|
||||||
low = ch.lower()
|
|
||||||
if ch.isupper() and low in SCAN:
|
|
||||||
self._kbd_stroke(SCAN['shift'], KEY_DOWN)
|
|
||||||
self._kbd_stroke(SCAN[low], KEY_DOWN)
|
|
||||||
self._kbd_stroke(SCAN[low], KEY_UP)
|
|
||||||
self._kbd_stroke(SCAN['shift'], KEY_UP)
|
|
||||||
elif ch.lower() in SCAN:
|
|
||||||
code = SCAN[low]
|
|
||||||
self._kbd_stroke(code, KEY_DOWN)
|
|
||||||
self._kbd_stroke(code, KEY_UP)
|
|
||||||
elif ch in UPPER_SCAN:
|
|
||||||
self._kbd_stroke(SCAN['shift'], KEY_DOWN)
|
|
||||||
self._kbd_stroke(UPPER_SCAN[ch], KEY_DOWN)
|
|
||||||
self._kbd_stroke(UPPER_SCAN[ch], KEY_UP)
|
|
||||||
self._kbd_stroke(SCAN['shift'], KEY_UP)
|
|
||||||
else:
|
|
||||||
return {'ok': False, 'error': 'untypable char %r' % ch}
|
|
||||||
time.sleep(0.015)
|
|
||||||
return {'ok': True, 'typed': str(text), 'backend': 'interception'}
|
|
||||||
@@ -1,31 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
"""Mock 后端:记录动作,无真实输入。开发/测试用。"""
|
|
||||||
import time
|
|
||||||
|
|
||||||
|
|
||||||
class MockBackend:
|
|
||||||
|
|
||||||
def __init__(self, log=None):
|
|
||||||
self.log = log if log is not None else []
|
|
||||||
self._screen = (1920, 1080)
|
|
||||||
|
|
||||||
def click(self, x, y, button='left', clicks=1):
|
|
||||||
self.log.append({'act': 'click', 'x': int(x), 'y': int(y),
|
|
||||||
'button': button, 'clicks': int(clicks)})
|
|
||||||
time.sleep(0.02)
|
|
||||||
return {'ok': True, 'clicked': [int(x), int(y)], 'backend': 'mock'}
|
|
||||||
|
|
||||||
def move(self, x, y):
|
|
||||||
self.log.append({'act': 'move', 'x': int(x), 'y': int(y)})
|
|
||||||
time.sleep(0.01)
|
|
||||||
return {'ok': True, 'backend': 'mock'}
|
|
||||||
|
|
||||||
def keypress(self, key):
|
|
||||||
self.log.append({'act': 'keypress', 'key': str(key)})
|
|
||||||
time.sleep(0.01)
|
|
||||||
return {'ok': True, 'key': str(key), 'backend': 'mock'}
|
|
||||||
|
|
||||||
def type_text(self, text):
|
|
||||||
self.log.append({'act': 'type_text', 'text': str(text)})
|
|
||||||
time.sleep(0.01 * len(text))
|
|
||||||
return {'ok': True, 'typed': str(text), 'backend': 'mock'}
|
|
||||||
@@ -1,100 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
"""SendInput 后端(Windows,ctypes user32)。
|
|
||||||
当前线上 lc_mcp.py 用的机制;作为 interception 不可用时的兜底。
|
|
||||||
"""
|
|
||||||
import ctypes
|
|
||||||
import time
|
|
||||||
|
|
||||||
from ctypes import wintypes
|
|
||||||
|
|
||||||
INPUT_MOUSE = 0
|
|
||||||
INPUT_KEYBOARD = 1
|
|
||||||
KEYEVENTF_KEYUP = 0x0002
|
|
||||||
KEYEVENTF_UNICODE = 0x0004
|
|
||||||
MOUSEEVENTF_LEFTDOWN = 0x0002
|
|
||||||
MOUSEEVENTF_LEFTUP = 0x0004
|
|
||||||
MOUSEEVENTF_RIGHTDOWN = 0x0008
|
|
||||||
MOUSEEVENTF_RIGHTUP = 0x0010
|
|
||||||
|
|
||||||
|
|
||||||
class MOUSEINPUT(ctypes.Structure):
|
|
||||||
_fields_ = [("dx", wintypes.LONG), ("dy", wintypes.LONG),
|
|
||||||
("mouseData", wintypes.DWORD), ("dwFlags", wintypes.DWORD),
|
|
||||||
("time", wintypes.DWORD), ("dwExtraInfo", ctypes.POINTER(ctypes.c_ulong))]
|
|
||||||
|
|
||||||
|
|
||||||
class KEYBDINPUT(ctypes.Structure):
|
|
||||||
_fields_ = [("wVk", wintypes.WORD), ("wScan", wintypes.WORD),
|
|
||||||
("dwFlags", wintypes.DWORD), ("time", wintypes.DWORD),
|
|
||||||
("dwExtraInfo", ctypes.POINTER(ctypes.c_ulong))]
|
|
||||||
|
|
||||||
|
|
||||||
class INPUT(ctypes.Structure):
|
|
||||||
class _I(ctypes.Union):
|
|
||||||
_fields_ = [("mi", MOUSEINPUT), ("ki", KEYBDINPUT)]
|
|
||||||
_anonymous_ = ("i",)
|
|
||||||
_fields_ = [("type", wintypes.DWORD), ("i", _I)]
|
|
||||||
|
|
||||||
|
|
||||||
KEYMAP = {'enter': 0x0D, 'return': 0x0D, 'esc': 0x1B, 'space': 0x20,
|
|
||||||
'tab': 0x09, 'ctrl': 0x11, 'alt': 0x12, 'shift': 0x10,
|
|
||||||
'p': 0x50, 'f4': 0x73, 'f5': 0x74, 'f8': 0x77, 'f12': 0x7B,
|
|
||||||
'1': 0x31, '2': 0x32, '3': 0x33, '4': 0x34, '5': 0x35,
|
|
||||||
'6': 0x36, '7': 0x37, '8': 0x38, '9': 0x39, '0': 0x30}
|
|
||||||
|
|
||||||
|
|
||||||
class SendInputBackend:
|
|
||||||
|
|
||||||
def __init__(self):
|
|
||||||
self._user32 = ctypes.WinDLL('user32', use_last_error=True)
|
|
||||||
|
|
||||||
def _send_input(self, *inputs):
|
|
||||||
n = len(inputs)
|
|
||||||
arr = (INPUT * n)(*inputs)
|
|
||||||
return self._user32.SendInput(n, arr, ctypes.sizeof(INPUT))
|
|
||||||
|
|
||||||
def click(self, x, y, button='left', clicks=1):
|
|
||||||
self._user32.SetCursorPos(int(x), int(y))
|
|
||||||
time.sleep(0.03)
|
|
||||||
down_flag = MOUSEEVENTF_LEFTDOWN if button == 'left' else MOUSEEVENTF_RIGHTDOWN
|
|
||||||
up_flag = MOUSEEVENTF_LEFTUP if button == 'left' else MOUSEEVENTF_RIGHTUP
|
|
||||||
for _ in range(max(1, int(clicks))):
|
|
||||||
down = INPUT(); down.type = INPUT_MOUSE; down.mi.dwFlags = down_flag
|
|
||||||
up = INPUT(); up.type = INPUT_MOUSE; up.mi.dwFlags = up_flag
|
|
||||||
self._send_input(down, up)
|
|
||||||
time.sleep(0.05)
|
|
||||||
return {'ok': True, 'clicked': [int(x), int(y)], 'backend': 'sendinput'}
|
|
||||||
|
|
||||||
def move(self, x, y):
|
|
||||||
self._user32.SetCursorPos(int(x), int(y))
|
|
||||||
time.sleep(0.01)
|
|
||||||
return {'ok': True, 'backend': 'sendinput'}
|
|
||||||
|
|
||||||
def keypress(self, key):
|
|
||||||
name = str(key).lower()
|
|
||||||
vk = KEYMAP.get(name)
|
|
||||||
if vk is None and len(name) == 1:
|
|
||||||
vk = ord(name.upper()) if name.isalpha() else ord(name)
|
|
||||||
if vk is None:
|
|
||||||
return {'ok': False, 'error': 'unknown key %s' % key}
|
|
||||||
self._press(vk)
|
|
||||||
return {'ok': True, 'key': str(key), 'backend': 'sendinput'}
|
|
||||||
|
|
||||||
def _press(self, vk):
|
|
||||||
ki = INPUT(); ki.type = INPUT_KEYBOARD; ki.ki.wVk = vk
|
|
||||||
self._send_input(ki)
|
|
||||||
time.sleep(0.04)
|
|
||||||
ki2 = INPUT(); ki2.type = INPUT_KEYBOARD; ki2.ki.wVk = vk
|
|
||||||
ki2.ki.dwFlags = KEYEVENTF_KEYUP
|
|
||||||
self._send_input(ki2)
|
|
||||||
|
|
||||||
def type_text(self, text):
|
|
||||||
for ch in str(text):
|
|
||||||
ki = INPUT(); ki.type = INPUT_KEYBOARD
|
|
||||||
ki.ki.wScan = ord(ch); ki.ki.dwFlags = KEYEVENTF_UNICODE
|
|
||||||
ki2 = INPUT(); ki2.type = INPUT_KEYBOARD
|
|
||||||
ki2.ki.wScan = ord(ch)
|
|
||||||
ki2.ki.dwFlags = KEYEVENTF_UNICODE | KEYEVENTF_KEYUP
|
|
||||||
self._send_input(ki, ki2)
|
|
||||||
time.sleep(0.01)
|
|
||||||
return {'ok': True, 'typed': str(text), 'backend': 'sendinput'}
|
|
||||||
@@ -1,94 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
"""Linux uinput 物理输入后端(虚拟内核 evdev 设备,需 root 或 uinput 组权限)。
|
|
||||||
比 xdotool 更接近硬件:内核层面人为物理设备,无 X 层痕迹。
|
|
||||||
依赖: python3-uinput 或手工 ioctl。
|
|
||||||
"""
|
|
||||||
import os
|
|
||||||
import struct
|
|
||||||
|
|
||||||
try:
|
|
||||||
import uinput
|
|
||||||
HAVE_UINPUT = True
|
|
||||||
except ImportError:
|
|
||||||
HAVE_UINPUT = False
|
|
||||||
|
|
||||||
|
|
||||||
class UinputBackend:
|
|
||||||
|
|
||||||
def __init__(self):
|
|
||||||
if not HAVE_UINPUT:
|
|
||||||
raise RuntimeError('python3-uinput not installed')
|
|
||||||
self._ctx = None
|
|
||||||
|
|
||||||
def _ensure(self):
|
|
||||||
if self._ctx is None:
|
|
||||||
from uinput import Device, KEY_A, KEY_Z, KEY_ENTER, KEY_ESC, KEY_SPACE, KEY_TAB, KEY_F4, KEY_LEFTCTRL, KEY_LEFTSHIFT, KEY_LEFTALT, KEY_BACKSPACE, REL_X, REL_Y, BTN_LEFT, BTN_RIGHT
|
|
||||||
events = (
|
|
||||||
# 全键盘范围(够用): A-Z + 数字 + 常用
|
|
||||||
*[key for key in range(KEY_A, KEY_Z + 1)],
|
|
||||||
KEY_ENTER, KEY_ESC, KEY_SPACE, KEY_TAB, KEY_F4,
|
|
||||||
KEY_LEFTCTRL, KEY_LEFTSHIFT, KEY_LEFTALT, KEY_BACKSPACE,
|
|
||||||
REL_X, REL_Y, BTN_LEFT, BTN_RIGHT,
|
|
||||||
)
|
|
||||||
self._dev = Device(events)
|
|
||||||
return self._dev
|
|
||||||
|
|
||||||
def _key(self, code):
|
|
||||||
dev = self._ensure()
|
|
||||||
dev.emit(code, 1)
|
|
||||||
dev.emit(code, 0)
|
|
||||||
time.sleep(0.03)
|
|
||||||
|
|
||||||
def move(self, x, y):
|
|
||||||
# uinput 只有相对位移; 需要知道当前位置。先简单相对 0 通过 xdotool 定位, 或 https 轮询
|
|
||||||
# 简化: 用 python-xlib 查指针位置 -> 计算 delta
|
|
||||||
try:
|
|
||||||
from Xlib import display
|
|
||||||
d = display.Display()
|
|
||||||
root = d.screen().root
|
|
||||||
qp = root.query_pointer()
|
|
||||||
curx, cury = qp.root_x, qp.root_y
|
|
||||||
dx, dy = int(x) - curx, int(y) - cury
|
|
||||||
dev = self._ensure()
|
|
||||||
dev.emit(REL_X, dx)
|
|
||||||
dev.emit(REL_Y, dy)
|
|
||||||
d.sync()
|
|
||||||
except Exception:
|
|
||||||
return {'ok': False, 'error': 'uinput move needs python-xlib'}
|
|
||||||
return {'ok': True}
|
|
||||||
|
|
||||||
def click(self, x, y, button='left', times=1):
|
|
||||||
self.move(x, y)
|
|
||||||
dev = self._ensure()
|
|
||||||
btn = BTN_LEFT if button == 'left' else BTN_RIGHT
|
|
||||||
for _ in range(times):
|
|
||||||
dev.emit(btn, 1)
|
|
||||||
time.sleep(0.05)
|
|
||||||
dev.emit(btn, 0)
|
|
||||||
time.sleep(0.05)
|
|
||||||
return {'ok': True}
|
|
||||||
|
|
||||||
def keypress(self, key):
|
|
||||||
mapping = {'enter': KEY_ENTER, 'esc': KEY_ESC, 'space': KEY_SPACE,
|
|
||||||
'tab': KEY_TAB, 'f4': KEY_F4, 'ctrl': KEY_LEFTCTRL,
|
|
||||||
'shift': KEY_LEFTSHIFT, 'alt': KEY_LEFTALT, 'backspace': KEY_BACKSPACE}
|
|
||||||
k = key.lower()
|
|
||||||
if k in mapping:
|
|
||||||
self._key(mapping[k])
|
|
||||||
return {'ok': True}
|
|
||||||
if len(k) == 1 and 'a' <= k <= 'z':
|
|
||||||
from uinput import KEY_A
|
|
||||||
self._key(KEY_A + (ord(k) - ord('a')))
|
|
||||||
return {'ok': True}
|
|
||||||
return {'ok': False, 'error': 'unsupported key %s' % key}
|
|
||||||
|
|
||||||
def type_text(self, text):
|
|
||||||
for ch in text:
|
|
||||||
if 'a' <= ch.lower() <= 'z':
|
|
||||||
self.keypress(ch.lower())
|
|
||||||
elif ch == ' ':
|
|
||||||
self.keypress('space')
|
|
||||||
else:
|
|
||||||
return {'ok': False, 'error': 'unsupported char %r' % ch}
|
|
||||||
time.sleep(0.05)
|
|
||||||
return {'ok': True}
|
|
||||||
@@ -1,50 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
"""Linux X11 输入后端:xdotool(X 事件注入,xdotool 需已安装)。
|
|
||||||
物理化升级路径: uinput 后端(虚拟内核 evdev 设备)。
|
|
||||||
"""
|
|
||||||
import shutil
|
|
||||||
import subprocess
|
|
||||||
import time
|
|
||||||
|
|
||||||
|
|
||||||
def _run(args):
|
|
||||||
return subprocess.run(args, capture_output=True, text=True, timeout=30)
|
|
||||||
|
|
||||||
|
|
||||||
class XdotoolBackend:
|
|
||||||
|
|
||||||
def __init__(self):
|
|
||||||
if not shutil.which('xdotool'):
|
|
||||||
raise RuntimeError('xdotool not found (apt install xdotool)')
|
|
||||||
|
|
||||||
def _screen(self):
|
|
||||||
try:
|
|
||||||
r = _run(['xdotool', 'getdisplaygeometry'])
|
|
||||||
w, h = r.stdout.strip().split()
|
|
||||||
return int(w), int(h)
|
|
||||||
except Exception:
|
|
||||||
return 1920, 1080
|
|
||||||
|
|
||||||
def move(self, x, y):
|
|
||||||
_run(['xdotool', 'mousemove', str(int(x)), str(int(y))])
|
|
||||||
return {'ok': True}
|
|
||||||
|
|
||||||
def click(self, x, y, button='left', times=1):
|
|
||||||
btn = {'left': '1', 'right': '3', 'middle': '2'}.get(button, '1')
|
|
||||||
_run(['xdotool', 'mousemove', str(int(x)), str(int(y))])
|
|
||||||
_run(['xdotool', 'click', '--repeat', str(times), '--delay', '60', btn])
|
|
||||||
return {'ok': True}
|
|
||||||
|
|
||||||
def keypress(self, key):
|
|
||||||
key = key.lower()
|
|
||||||
# xdotool 键名映射
|
|
||||||
mapping = {'enter': 'Return', 'esc': 'Escape', 'space': 'space',
|
|
||||||
'tab': 'Tab', 'alt': 'Alt_L', 'f4': 'F4', 'ctrl': 'Control_L',
|
|
||||||
'shift': 'Shift_L', 'backspace': 'BackSpace'}
|
|
||||||
k = mapping.get(key, key)
|
|
||||||
_run(['xdotool', 'key', k])
|
|
||||||
return {'ok': True}
|
|
||||||
|
|
||||||
def type_text(self, text):
|
|
||||||
_run(['xdotool', 'type', '--delay', '60', text])
|
|
||||||
return {'ok': True}
|
|
||||||
@@ -1,228 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
# -*- coding: utf-8 -*-
|
|
||||||
"""Limbus Company 语义化操控 MCP server v0.5 (跨平台)。
|
|
||||||
|
|
||||||
核心思路: 不让 AI 盲猜坐标。工具层做"文本化 + 语义点击"——
|
|
||||||
get_state : 截图 + OCR -> 全页文字元素(带坐标)
|
|
||||||
click_text : 说文字/按钮名, 内部 OCR 定位后点击
|
|
||||||
wait_for_text / wait_for_text_gone : 等条件出现/消失再回报
|
|
||||||
平台无关: 截图/输入按 LCP_PLATFORM (win|linux|mac, 默认自动检测) 加载实现。
|
|
||||||
win : 截图=交互计划任务; 输入=sendinput|interception
|
|
||||||
linux : 截图=X11 PIL/scrot; 输入=xdotool|uinput(物理)
|
|
||||||
mac : 截图=screencapture; 输入=cg
|
|
||||||
OCR 服务: 跑在远端(默认 http://35.212.193.150:8599/), 客户端机器零安装。
|
|
||||||
协议: MCP stdio (newline-delimited JSON-RPC)。
|
|
||||||
"""
|
|
||||||
import os
|
|
||||||
import sys
|
|
||||||
import json
|
|
||||||
import time
|
|
||||||
|
|
||||||
from platform import detect_platform, get_input_backend, get_screenshot_provider
|
|
||||||
|
|
||||||
BACKEND_NAME = os.environ.get('INPUT_BACKEND', '').strip().lower()
|
|
||||||
OCR_BACKEND = os.environ.get('OCR_BACKEND', 'local').strip().lower()
|
|
||||||
MIN_SCORE = float(os.environ.get('OCR_MIN_SCORE', '0.5'))
|
|
||||||
BASE = os.path.join(os.path.expanduser('~'), 'lc_mcp')
|
|
||||||
os.makedirs(BASE, exist_ok=True)
|
|
||||||
|
|
||||||
|
|
||||||
def log(msg):
|
|
||||||
try:
|
|
||||||
with open(os.path.join(BASE, 'lc_mcp.log'), 'a', encoding='utf-8') as f:
|
|
||||||
f.write('[%s] %s\n' % (time.strftime('%H:%M:%S'), msg))
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def ocr_image(img_path):
|
|
||||||
"""本地 OCR(Windows 自带 OCR,winsdk;无任何云端依赖)。"""
|
|
||||||
try:
|
|
||||||
import ocr_local
|
|
||||||
except Exception as e:
|
|
||||||
return None, 'ocr_local import fail: %s' % e
|
|
||||||
if not ocr_local.available():
|
|
||||||
return None, 'winsdk 不可用: pip install --user winsdk'
|
|
||||||
items, err = ocr_local.ocr_image(img_path)
|
|
||||||
if items is None:
|
|
||||||
return None, err
|
|
||||||
items = [it for it in items if float(it.get('score', 1)) >= MIN_SCORE]
|
|
||||||
return items, None
|
|
||||||
|
|
||||||
|
|
||||||
def calc_click(txt, items):
|
|
||||||
"""在 OCR items 里找包含 txt 的项,返回中心和候选列表。"""
|
|
||||||
hits = []
|
|
||||||
for it in items:
|
|
||||||
if txt.lower() in it['text'].lower():
|
|
||||||
cx = it['x'] + it['w'] // 2
|
|
||||||
cy = it['y'] + it['h'] // 2
|
|
||||||
hits.append({'text': it['text'], 'x': cx, 'y': cy})
|
|
||||||
return hits
|
|
||||||
|
|
||||||
|
|
||||||
class MCPServer:
|
|
||||||
|
|
||||||
def __init__(self, platform, provider, backend):
|
|
||||||
self.platform = platform
|
|
||||||
self.provider = provider
|
|
||||||
self.backend = backend
|
|
||||||
self.tools = [
|
|
||||||
{"name": "get_state", "description": "截图并OCR, 返回当前屏幕全部文字元素(含坐标)。AI 决策前先调这个, 不需要自己猜坐标",
|
|
||||||
"inputSchema": {"type": "object", "properties": {}}},
|
|
||||||
{"name": "click_text", "description": "语义点击: 给按钮文字(支持子串), 内部OCR定位后点击其中心。例: click_text('开始')",
|
|
||||||
"inputSchema": {"type": "object", "properties": {
|
|
||||||
"text": {"type": "string"}, "index": {"type": "number", "default": 0}},
|
|
||||||
"required": ["text"]}},
|
|
||||||
{"name": "wait_for_text", "description": "循环截图+OCR, 等到指定文字出现(默认最多30秒)", "inputSchema": {"type": "object", "properties": {
|
|
||||||
"text": {"type": "string"}, "timeout": {"type": "number", "default": 30}},
|
|
||||||
"required": ["text"]}},
|
|
||||||
{"name": "wait_for_text_gone", "description": "循环截图+OCR, 等到指定文字消失(默认最多30秒)", "inputSchema": {"type": "object", "properties": {
|
|
||||||
"text": {"type": "string"}, "timeout": {"type": "number", "default": 30}},
|
|
||||||
"required": ["text"]}},
|
|
||||||
{"name": "screenshot", "description": "截图并返回PNG路径(用于模板/人工查看)", "inputSchema": {"type": "object", "properties": {}}},
|
|
||||||
{"name": "click", "description": "直接按坐标点击(一般用 click_text 更稳)", "inputSchema": {"type": "object", "properties": {
|
|
||||||
"x": {"type": "number"}, "y": {"type": "number"},
|
|
||||||
"button": {"type": "string", "enum": ["left", "right"], "default": "left"}},
|
|
||||||
"required": ["x", "y"]}},
|
|
||||||
{"name": "move", "description": "移动鼠标到(x,y)", "inputSchema": {"type": "object", "properties": {
|
|
||||||
"x": {"type": "number"}, "y": {"type": "number"}}, "required": ["x", "y"]}},
|
|
||||||
{"name": "keypress", "description": "按一次键(p/enter/esc/space/tab/alt/f4/单字符等)", "inputSchema": {"type": "object", "properties": {
|
|
||||||
"key": {"type": "string"}}, "required": ["key"]}},
|
|
||||||
{"name": "type_text", "description": "输入一串文本", "inputSchema": {"type": "object", "properties": {
|
|
||||||
"text": {"type": "string"}}, "required": ["text"]}},
|
|
||||||
]
|
|
||||||
|
|
||||||
def call(self, name, args):
|
|
||||||
args = args or {}
|
|
||||||
if name == 'screenshot':
|
|
||||||
return self.provider.grab()
|
|
||||||
if name == 'get_state':
|
|
||||||
sh = self.provider.grab()
|
|
||||||
if not sh.get('ok'):
|
|
||||||
return sh
|
|
||||||
items, err = ocr_image(sh['path'])
|
|
||||||
if err:
|
|
||||||
return {'ok': False, 'error': err}
|
|
||||||
return {'ok': True, 'screen': sh['path'], 'items': items}
|
|
||||||
if name == 'click_text':
|
|
||||||
sh = self.provider.grab()
|
|
||||||
if not sh.get('ok'):
|
|
||||||
return sh
|
|
||||||
items, err = ocr_image(sh['path'])
|
|
||||||
if err:
|
|
||||||
return {'ok': False, 'error': err}
|
|
||||||
hits = calc_click(args.get('text', ''), items or [])
|
|
||||||
if not hits:
|
|
||||||
return {'ok': False, 'error': 'text not found: %s' % args.get('text', ''),
|
|
||||||
'screen': sh['path'], 'items': items}
|
|
||||||
idx = min(int(args.get('index', 0)), len(hits) - 1)
|
|
||||||
h = hits[idx]
|
|
||||||
r = self.backend.click(h['x'], h['y'])
|
|
||||||
r['on'] = h['text']
|
|
||||||
return r
|
|
||||||
if name == 'wait_for_text':
|
|
||||||
return self._wait(args.get('text', ''), float(args.get('timeout', 30)), True)
|
|
||||||
if name == 'wait_for_text_gone':
|
|
||||||
return self._wait(args.get('text', ''), float(args.get('timeout', 30)), False)
|
|
||||||
if name == 'click':
|
|
||||||
return self.backend.click(int(args.get('x', 0)), int(args.get('y', 0)), args.get('button', 'left'), 1)
|
|
||||||
if name == 'move':
|
|
||||||
return self.backend.move(int(args.get('x', 0)), int(args.get('y', 0)))
|
|
||||||
if name == 'keypress':
|
|
||||||
return self.backend.keypress(args.get('key', ''))
|
|
||||||
if name == 'type_text':
|
|
||||||
return self.backend.type_text(args.get('text', ''))
|
|
||||||
return {'ok': False, 'error': 'unknown tool %s' % name}
|
|
||||||
|
|
||||||
def _wait(self, txt, timeout, want_present):
|
|
||||||
deadline = time.time() + timeout
|
|
||||||
last_items = []
|
|
||||||
while time.time() < deadline:
|
|
||||||
sh = self.provider.grab()
|
|
||||||
if sh.get('ok'):
|
|
||||||
items, err = ocr_image(sh['path'])
|
|
||||||
if not err:
|
|
||||||
last_items = items or []
|
|
||||||
hits = calc_click(txt, last_items)
|
|
||||||
if bool(hits) == want_present:
|
|
||||||
return {'ok': True, 'present': bool(hits),
|
|
||||||
'hits': hits[:5], 'screen': sh['path']}
|
|
||||||
time.sleep(1.2)
|
|
||||||
return {'ok': False, 'error': 'timeout waiting %s (present=%s)' % (txt, want_present),
|
|
||||||
'last_items': last_items[:10]}
|
|
||||||
|
|
||||||
|
|
||||||
def read_msg():
|
|
||||||
line = sys.stdin.readline()
|
|
||||||
if not line:
|
|
||||||
return None
|
|
||||||
return json.loads(line.strip())
|
|
||||||
|
|
||||||
|
|
||||||
def send(obj):
|
|
||||||
sys.stdout.write(json.dumps(obj, ensure_ascii=False) + '\n')
|
|
||||||
sys.stdout.flush()
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
platform = detect_platform()
|
|
||||||
log('lc_mcp v0.5 platform=%s backend=%s ocr=%s' % (platform, BACKEND_NAME or 'auto', OCR_BACKEND))
|
|
||||||
backend = None
|
|
||||||
try:
|
|
||||||
provider = get_screenshot_provider(platform)
|
|
||||||
backend = get_input_backend(BACKEND_NAME, platform)
|
|
||||||
except Exception as e:
|
|
||||||
log('init fail: %s' % e)
|
|
||||||
provider = None
|
|
||||||
backend = None
|
|
||||||
if backend is None:
|
|
||||||
log('WARN: input backend unavailable, semantic/click tools disabled')
|
|
||||||
server = MCPServer(platform, provider, backend)
|
|
||||||
while True:
|
|
||||||
try:
|
|
||||||
msg = read_msg()
|
|
||||||
if msg is None:
|
|
||||||
break
|
|
||||||
except Exception as e:
|
|
||||||
log('read err %s' % e)
|
|
||||||
break
|
|
||||||
method = msg.get('method')
|
|
||||||
mid = msg.get('id')
|
|
||||||
if method == 'initialize':
|
|
||||||
send({"jsonrpc": "2.0", "id": mid, "result": {
|
|
||||||
"protocolVersion": "2024-11-05",
|
|
||||||
"capabilities": {"tools": {"listChanged": False}},
|
|
||||||
"serverInfo": {"name": "lc-mcp", "version": "0.5.0"}}})
|
|
||||||
elif method == 'notifications/initialized':
|
|
||||||
continue
|
|
||||||
elif method == 'ping':
|
|
||||||
send({"jsonrpc": "2.0", "id": mid, "result": {}})
|
|
||||||
elif method == 'tools/list':
|
|
||||||
send({"jsonrpc": "2.0", "id": mid, "result": {"tools": server.tools}})
|
|
||||||
elif method == 'tools/call':
|
|
||||||
params = msg.get('params', {})
|
|
||||||
name = params.get('name')
|
|
||||||
args = params.get('arguments', {})
|
|
||||||
log('call %s %s' % (name, args))
|
|
||||||
if server.backend is None and name not in ('screenshot', 'get_state'):
|
|
||||||
send({"jsonrpc": "2.0", "id": mid, "result": {"content": [
|
|
||||||
{"type": "text", "text": json.dumps({'ok': False, 'error': 'input backend unavailable'})}]}})
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
result = server.call(name, args)
|
|
||||||
send({"jsonrpc": "2.0", "id": mid, "result": {"content": [
|
|
||||||
{"type": "text", "text": json.dumps(result, ensure_ascii=False)}]}})
|
|
||||||
except Exception as e:
|
|
||||||
log('tool err %s' % e)
|
|
||||||
send({"jsonrpc": "2.0", "id": mid, "result": {"content": [
|
|
||||||
{"type": "text", "text": json.dumps({'ok': False, 'error': str(e)})}]}, "isError": True})
|
|
||||||
else:
|
|
||||||
if mid is not None:
|
|
||||||
send({"jsonrpc": "2.0", "id": mid,
|
|
||||||
"error": {"code": -32601, "message": "method not found"}})
|
|
||||||
log('lc_mcp exit')
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
main()
|
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
"""lc-mcp: 以 AALC (AhabAssistantLimbusCompany) 为底座的 Limbus Company MCP server。"""
|
||||||
|
|
||||||
|
VERSION = '0.7.0'
|
||||||
@@ -0,0 +1,143 @@
|
|||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
"""AALC 引擎: 截图/OCR/模板匹配/输入全部走 AALC 的 module.automation。
|
||||||
|
|
||||||
|
只能在 Windows 的交互桌面会话里运行(找窗口句柄、后台截图、PostMessage 都依赖它)。
|
||||||
|
AALC 代码按相对路径读 assets/ 和 config.yaml, 所以 bootstrap() 会 chdir 到 AALC 目录。
|
||||||
|
"""
|
||||||
|
import glob
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
|
||||||
|
from .engine import Engine
|
||||||
|
|
||||||
|
REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||||
|
DEFAULT_AALC_DIR = os.path.join(REPO, 'vendor', 'aalc')
|
||||||
|
_booted = None
|
||||||
|
|
||||||
|
|
||||||
|
def aalc_dir():
|
||||||
|
return os.path.abspath(os.environ.get('AALC_DIR') or DEFAULT_AALC_DIR)
|
||||||
|
|
||||||
|
|
||||||
|
def bootstrap():
|
||||||
|
"""进程内初始化 AALC (幂等)。必须在 import 任何 AALC 模块之前调用。"""
|
||||||
|
global _booted
|
||||||
|
if _booted:
|
||||||
|
return _booted
|
||||||
|
d = aalc_dir()
|
||||||
|
if not os.path.isfile(os.path.join(d, 'main.py')):
|
||||||
|
raise RuntimeError('AALC 目录无效: %s (git submodule update --init, 或设置 AALC_DIR)' % d)
|
||||||
|
os.chdir(d)
|
||||||
|
if d not in sys.path:
|
||||||
|
sys.path.insert(0, d)
|
||||||
|
# 与 AALC main.py 相同的 DPI 设置, 否则截图/点击坐标在缩放下错位
|
||||||
|
from ctypes import c_void_p, windll
|
||||||
|
try:
|
||||||
|
windll.user32.SetProcessDpiAwarenessContext(c_void_p(-4))
|
||||||
|
except (AttributeError, OSError):
|
||||||
|
try:
|
||||||
|
windll.shcore.SetProcessDpiAwareness(2)
|
||||||
|
except (AttributeError, OSError):
|
||||||
|
windll.user32.SetProcessDPIAware()
|
||||||
|
from module.logger.my_log import Logger
|
||||||
|
Logger()
|
||||||
|
_booted = d
|
||||||
|
return d
|
||||||
|
|
||||||
|
|
||||||
|
def prune(directory, pattern, keep):
|
||||||
|
files = sorted(glob.glob(os.path.join(directory, pattern)), key=os.path.getmtime)
|
||||||
|
for p in files[:-keep] if keep > 0 else []:
|
||||||
|
try:
|
||||||
|
os.remove(p)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class AalcEngine(Engine):
|
||||||
|
|
||||||
|
def __init__(self, shot_dir, keep_shots=30):
|
||||||
|
self.shot_dir = os.path.abspath(shot_dir)
|
||||||
|
os.makedirs(self.shot_dir, exist_ok=True)
|
||||||
|
self.keep_shots = keep_shots
|
||||||
|
root = bootstrap()
|
||||||
|
from module.automation import auto
|
||||||
|
from module.automation.input_handlers.input import key_list
|
||||||
|
from module.automation.screenshot import ScreenShot
|
||||||
|
from module.config import cfg
|
||||||
|
from module.game_and_screen import screen
|
||||||
|
from module.ocr import ocr
|
||||||
|
from utils.path_manager import path_manager
|
||||||
|
path_manager.initialize_paths()
|
||||||
|
self.auto, self.cfg, self.screen, self.ocr_engine = auto, cfg, screen, ocr
|
||||||
|
self.ScreenShot = ScreenShot
|
||||||
|
self.key_names = sorted(key_list)
|
||||||
|
self.images_root = os.path.join(root, 'assets', 'images')
|
||||||
|
|
||||||
|
# ---- 看 ----
|
||||||
|
def _grab(self, gray):
|
||||||
|
if not self.cfg.simulator and not self.screen.handle.hwnd:
|
||||||
|
raise RuntimeError('未找到游戏窗口 %r: 先启动游戏(run_task 会自动启动)' % self.cfg.game_title_name)
|
||||||
|
# 不用 auto.take_screenshot: 它连续失败 60 秒会直接 taskkill 游戏
|
||||||
|
with self.auto._screenshot_lock:
|
||||||
|
img = self.ScreenShot.take_screenshot(gray)
|
||||||
|
if img is None:
|
||||||
|
raise RuntimeError('AALC 截图失败(窗口最小化/被遮挡? 见 AALC logs)')
|
||||||
|
self.auto.screenshot = img # find_image_element 读这个
|
||||||
|
return img
|
||||||
|
|
||||||
|
def _save(self, img):
|
||||||
|
stamp = time.strftime('%Y%m%d_%H%M%S') + '_%03d' % (int(time.time() * 1000) % 1000)
|
||||||
|
path = os.path.join(self.shot_dir, 'shot_%s.png' % stamp)
|
||||||
|
img.save(path)
|
||||||
|
prune(self.shot_dir, 'shot_*.png', self.keep_shots)
|
||||||
|
return {'path': path.replace('\\', '/'), 'w': img.width, 'h': img.height}
|
||||||
|
|
||||||
|
def screenshot(self):
|
||||||
|
return self._save(self._grab(gray=False))
|
||||||
|
|
||||||
|
def ocr(self):
|
||||||
|
img = self._grab(gray=False)
|
||||||
|
shot = self._save(img)
|
||||||
|
res = self.ocr_engine.run(img)
|
||||||
|
items = []
|
||||||
|
boxes = res.boxes if res.boxes is not None else []
|
||||||
|
scores = res.scores if res.scores is not None else [1.0] * len(boxes)
|
||||||
|
for box, text, score in zip(boxes, res.txts or (), scores):
|
||||||
|
xs = [p[0] for p in box]
|
||||||
|
ys = [p[1] for p in box]
|
||||||
|
x0, y0 = int(min(xs)), int(min(ys))
|
||||||
|
items.append({'text': text, 'x': x0, 'y': y0,
|
||||||
|
'w': int(max(xs)) - x0, 'h': int(max(ys)) - y0,
|
||||||
|
'score': round(float(score), 3)})
|
||||||
|
return shot, items
|
||||||
|
|
||||||
|
def list_images(self, prefix=''):
|
||||||
|
names = set()
|
||||||
|
for theme in ('dark', 'default'):
|
||||||
|
for lang in ('zh_cn', 'en', 'share'):
|
||||||
|
base = os.path.join(self.images_root, theme, lang)
|
||||||
|
for p in glob.glob(os.path.join(base, '**', '*.png'), recursive=True):
|
||||||
|
names.add(os.path.relpath(p, base).replace('\\', '/'))
|
||||||
|
return sorted(n for n in names if n.startswith(prefix))
|
||||||
|
|
||||||
|
def find_image(self, name, threshold=0.8, model='normal'):
|
||||||
|
if name not in self.list_images(name):
|
||||||
|
raise ValueError('没有模板图 %r (用 list_images 查)' % name)
|
||||||
|
self._grab(gray=True)
|
||||||
|
pos = self.auto.find_image_element(name, threshold, model=model)
|
||||||
|
return (int(pos[0]), int(pos[1])) if pos else None
|
||||||
|
|
||||||
|
# ---- 动手 (AALC 输入后端, 由 config.yaml win_input_type 决定: 默认后台 PostMessage) ----
|
||||||
|
def click(self, x, y, times=1):
|
||||||
|
self.auto.mouse_click(int(x), int(y), times=int(times))
|
||||||
|
|
||||||
|
def keypress(self, key):
|
||||||
|
k = str(key).lower()
|
||||||
|
if k not in self.key_names:
|
||||||
|
raise ValueError('AALC 不支持按键 %r, 可用: %s' % (key, ' '.join(self.key_names)))
|
||||||
|
self.auto.key_press(k)
|
||||||
|
|
||||||
|
def type_text(self, text):
|
||||||
|
self.auto.input_text(str(text))
|
||||||
@@ -0,0 +1,81 @@
|
|||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
"""子进程入口: 跑 AALC 现成的任务流程。
|
||||||
|
|
||||||
|
用法: python -m lc_mcp.aalc_task <task> [json_params]
|
||||||
|
放在子进程里跑, 停止 = 杀进程树; 不会像 QThread.terminate 那样留下被占用的锁。
|
||||||
|
"""
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
|
||||||
|
from .aalc_engine import bootstrap
|
||||||
|
|
||||||
|
# 任务 -> {参数名: AALC 配置项}; 参数只在本次运行内生效
|
||||||
|
TASKS = {
|
||||||
|
'all': {}, # 完全按 config.yaml 里勾选的任务跑, 等价于 AALC 界面点"开始"
|
||||||
|
'daily': {'exp': 'set_EXP_count', 'thread': 'set_thread_count'},
|
||||||
|
'reward': {},
|
||||||
|
'enkephalin': {'times': 'set_lunacy_to_enkephalin'},
|
||||||
|
'mirror': {'count': 'set_mirror_count'},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def run(task, params):
|
||||||
|
bootstrap()
|
||||||
|
from module.automation import auto
|
||||||
|
from module.config import cfg
|
||||||
|
from tasks.base import script_task_scheme as scheme
|
||||||
|
from tasks.base.retry_monitor import retry_monitor
|
||||||
|
from tasks.battle import battle
|
||||||
|
from utils.path_manager import path_manager
|
||||||
|
|
||||||
|
originals = {}
|
||||||
|
for name, value in params.items():
|
||||||
|
key = TASKS[task][name]
|
||||||
|
originals[key] = cfg.get_value(key)
|
||||||
|
cfg.unsaved_set_value(key, int(value))
|
||||||
|
try:
|
||||||
|
if task == 'all':
|
||||||
|
scheme.script_task()
|
||||||
|
return
|
||||||
|
# 与 script_task() 相同的前置步骤
|
||||||
|
scheme.init_game()
|
||||||
|
path_manager.initialize_paths()
|
||||||
|
auto.clear_img_cache()
|
||||||
|
retry_monitor.start()
|
||||||
|
get_reward = None
|
||||||
|
if auto.click_element('battle/turn_assets.png', take_screenshot=True):
|
||||||
|
print('[lc_mcp] 检测到正在战斗, 先打完', flush=True)
|
||||||
|
get_reward = battle.fight()
|
||||||
|
{
|
||||||
|
'daily': scheme.Daily_task_wrapper(get_reward=get_reward),
|
||||||
|
'reward': scheme.to_get_reward,
|
||||||
|
'enkephalin': scheme.Buy_enkephalin,
|
||||||
|
'mirror': scheme.Mirror_task,
|
||||||
|
}[task]()
|
||||||
|
finally:
|
||||||
|
retry_monitor.stop()
|
||||||
|
# AALC 运行中会自行保存配置(换队伍队列等), 把临时覆盖的次数写回原值
|
||||||
|
for key, value in originals.items():
|
||||||
|
cfg.set_value(key, value)
|
||||||
|
if originals:
|
||||||
|
cfg.flush()
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv):
|
||||||
|
if len(argv) < 2 or argv[1] not in TASKS:
|
||||||
|
print('usage: python -m lc_mcp.aalc_task <%s> [json]' % '|'.join(TASKS), file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
task = argv[1]
|
||||||
|
params = json.loads(argv[2]) if len(argv) > 2 else {}
|
||||||
|
unknown = set(params) - set(TASKS[task])
|
||||||
|
if unknown:
|
||||||
|
print('unknown params for %s: %s' % (task, ', '.join(sorted(unknown))), file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
print('[lc_mcp] task %s start %s' % (task, json.dumps(params)), flush=True)
|
||||||
|
run(task, params)
|
||||||
|
print('[lc_mcp] task %s done' % task, flush=True)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
sys.exit(main(sys.argv))
|
||||||
@@ -0,0 +1,84 @@
|
|||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
"""stdio <-> 常驻服务 的中继。MCP 客户端(经 ssh)启动的是这个, 只用标准库。
|
||||||
|
|
||||||
|
ssh win python C:/path/lc-mcp/lc_mcp/bridge.py
|
||||||
|
|
||||||
|
为什么不直接 ssh 跑 server: SSH 会话在 session 0, 看不到游戏窗口、也发不了输入。
|
||||||
|
服务没起来时, 对每个请求回明确的错误, 而不是让客户端干等。
|
||||||
|
"""
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import socket
|
||||||
|
import sys
|
||||||
|
import threading
|
||||||
|
|
||||||
|
HOME = os.path.join(os.path.expanduser('~'), 'lc_mcp')
|
||||||
|
TOKEN_FILE = os.path.join(HOME, 'token')
|
||||||
|
PORT = int(os.environ.get('LCP_PORT', '8765'))
|
||||||
|
|
||||||
|
|
||||||
|
def connect(port=PORT, token_file=TOKEN_FILE):
|
||||||
|
with open(token_file, encoding='ascii') as f:
|
||||||
|
token = f.read().strip()
|
||||||
|
sock = socket.create_connection(('127.0.0.1', port), timeout=5)
|
||||||
|
sock.settimeout(None)
|
||||||
|
sock.sendall(('AUTH %s\n' % token).encode('ascii'))
|
||||||
|
rfile = sock.makefile('rb')
|
||||||
|
if rfile.readline().strip() != b'OK':
|
||||||
|
raise ConnectionError('token 被拒绝 (%s 与服务端不一致?)' % token_file)
|
||||||
|
return sock, rfile
|
||||||
|
|
||||||
|
|
||||||
|
def relay(sock, rfile, stdin, stdout):
|
||||||
|
def up():
|
||||||
|
try:
|
||||||
|
for line in iter(stdin.readline, b''):
|
||||||
|
sock.sendall(line)
|
||||||
|
finally:
|
||||||
|
try:
|
||||||
|
sock.shutdown(socket.SHUT_WR)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
threading.Thread(target=up, daemon=True).start()
|
||||||
|
for line in iter(rfile.readline, b''):
|
||||||
|
stdout.write(line)
|
||||||
|
stdout.flush()
|
||||||
|
|
||||||
|
|
||||||
|
def answer_offline(reason, stdin, stdout):
|
||||||
|
"""服务不可达: 仍完成握手, 但每个工具调用都返回原因。"""
|
||||||
|
for raw in iter(stdin.readline, b''):
|
||||||
|
try:
|
||||||
|
msg = json.loads(raw)
|
||||||
|
except ValueError:
|
||||||
|
continue
|
||||||
|
if not isinstance(msg, dict) or msg.get('id') is None:
|
||||||
|
continue
|
||||||
|
if msg.get('method') == 'initialize':
|
||||||
|
resp = {'protocolVersion': '2024-11-05', 'capabilities': {'tools': {}},
|
||||||
|
'serverInfo': {'name': 'lc-mcp (offline)', 'version': '0'}}
|
||||||
|
out = {'jsonrpc': '2.0', 'id': msg['id'], 'result': resp}
|
||||||
|
elif msg.get('method') == 'tools/list':
|
||||||
|
out = {'jsonrpc': '2.0', 'id': msg['id'], 'result': {'tools': []}}
|
||||||
|
else:
|
||||||
|
out = {'jsonrpc': '2.0', 'id': msg['id'], 'error': {'code': -32000, 'message': reason}}
|
||||||
|
stdout.write((json.dumps(out, ensure_ascii=False) + '\n').encode('utf-8'))
|
||||||
|
stdout.flush()
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
stdin, stdout = sys.stdin.buffer, sys.stdout.buffer
|
||||||
|
try:
|
||||||
|
sock, rfile = connect()
|
||||||
|
except (OSError, ConnectionError) as e:
|
||||||
|
reason = ('lc-mcp 服务未运行或不可达 (%s)。在 Windows 桌面会话里启动: '
|
||||||
|
'scripts/install_logon_task.ps1 或 python -m lc_mcp.server' % e)
|
||||||
|
sys.stderr.write(reason + '\n')
|
||||||
|
answer_offline(reason, stdin, stdout)
|
||||||
|
return 1
|
||||||
|
relay(sock, rfile, stdin, stdout)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
sys.exit(main())
|
||||||
@@ -0,0 +1,58 @@
|
|||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
"""引擎接口 + 平台无关的文字匹配。
|
||||||
|
|
||||||
|
引擎 = "看屏幕 + 动手" 的最小集合, 坐标一律是游戏窗口客户区截图里的像素坐标。
|
||||||
|
实现: AalcEngine(Windows, 调 AALC) / FakeEngine(测试)。
|
||||||
|
"""
|
||||||
|
import re
|
||||||
|
|
||||||
|
_WS = re.compile(r'\s+')
|
||||||
|
|
||||||
|
|
||||||
|
def norm(s):
|
||||||
|
"""匹配用归一化: 去全部空白 + 小写 (OCR 常把 "Hong Lu" 识别成 "HongLu" 或反之)。"""
|
||||||
|
return _WS.sub('', str(s)).lower()
|
||||||
|
|
||||||
|
|
||||||
|
def find_hits(text, items):
|
||||||
|
"""在 OCR items 里找包含 text 的项, 返回中心点列表。完全相等的排前面。"""
|
||||||
|
key = norm(text)
|
||||||
|
if not key:
|
||||||
|
return []
|
||||||
|
exact, partial = [], []
|
||||||
|
for it in items:
|
||||||
|
t = norm(it['text'])
|
||||||
|
if key not in t:
|
||||||
|
continue
|
||||||
|
hit = {'text': it['text'], 'x': it['x'] + it['w'] // 2, 'y': it['y'] + it['h'] // 2}
|
||||||
|
(exact if t == key else partial).append(hit)
|
||||||
|
return exact + partial
|
||||||
|
|
||||||
|
|
||||||
|
class Engine:
|
||||||
|
"""所有方法失败时抛异常, 由协议层转成 isError 结果。"""
|
||||||
|
|
||||||
|
def screenshot(self):
|
||||||
|
"""截图并落盘。返回 {'path': str, 'w': int, 'h': int}。"""
|
||||||
|
raise NotImplementedError
|
||||||
|
|
||||||
|
def ocr(self):
|
||||||
|
"""截图 + OCR。返回 (shot, items); items = [{text, x, y, w, h, score}]。"""
|
||||||
|
raise NotImplementedError
|
||||||
|
|
||||||
|
def find_image(self, name, threshold=0.8, model='normal'):
|
||||||
|
"""在当前画面找 AALC 模板图 (如 'battle/turn_assets.png')。返回 (x, y) 或 None。"""
|
||||||
|
raise NotImplementedError
|
||||||
|
|
||||||
|
def list_images(self, prefix=''):
|
||||||
|
"""列出可用模板图的相对路径。"""
|
||||||
|
raise NotImplementedError
|
||||||
|
|
||||||
|
def click(self, x, y, times=1):
|
||||||
|
raise NotImplementedError
|
||||||
|
|
||||||
|
def keypress(self, key):
|
||||||
|
raise NotImplementedError
|
||||||
|
|
||||||
|
def type_text(self, text):
|
||||||
|
raise NotImplementedError
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
"""测试用引擎: 屏幕内容由测试脚本预设, 记录所有输入动作。"""
|
||||||
|
from .engine import Engine
|
||||||
|
|
||||||
|
|
||||||
|
class FakeEngine(Engine):
|
||||||
|
|
||||||
|
def __init__(self, frames=None, images=None):
|
||||||
|
# frames: 每次 ocr() 依次返回的 items 列表; 用完后停在最后一帧
|
||||||
|
self.frames = list(frames or [[]])
|
||||||
|
self.images = dict(images or {}) # name -> (x, y)
|
||||||
|
self.actions = []
|
||||||
|
self.shots = 0
|
||||||
|
|
||||||
|
def screenshot(self):
|
||||||
|
self.shots += 1
|
||||||
|
return {'path': 'fake/shot_%d.png' % self.shots, 'w': 1920, 'h': 1080}
|
||||||
|
|
||||||
|
def ocr(self):
|
||||||
|
shot = self.screenshot()
|
||||||
|
items = self.frames.pop(0) if len(self.frames) > 1 else self.frames[0]
|
||||||
|
return shot, [dict(it, score=it.get('score', 0.99)) for it in items]
|
||||||
|
|
||||||
|
def find_image(self, name, threshold=0.8, model='normal'):
|
||||||
|
return self.images.get(name)
|
||||||
|
|
||||||
|
def list_images(self, prefix=''):
|
||||||
|
return sorted(n for n in self.images if n.startswith(prefix))
|
||||||
|
|
||||||
|
def click(self, x, y, times=1):
|
||||||
|
self.actions.append(('click', int(x), int(y), int(times)))
|
||||||
|
|
||||||
|
def keypress(self, key):
|
||||||
|
self.actions.append(('key', key))
|
||||||
|
|
||||||
|
def type_text(self, text):
|
||||||
|
self.actions.append(('type', text))
|
||||||
@@ -0,0 +1,209 @@
|
|||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
"""MCP (JSON-RPC 2.0) 处理: 与传输方式无关, handle(msg) -> 响应 dict 或 None(通知)。"""
|
||||||
|
import json
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
|
||||||
|
from . import VERSION
|
||||||
|
from .aalc_task import TASKS
|
||||||
|
from .engine import find_hits
|
||||||
|
|
||||||
|
MAX_WAIT = 300.0
|
||||||
|
WAIT_INTERVAL = 1.0
|
||||||
|
INPUT_TOOLS = {'click', 'click_text', 'click_image', 'keypress', 'type_text'}
|
||||||
|
TASK_TOOLS = {'run_task', 'task_status', 'stop_task'}
|
||||||
|
|
||||||
|
|
||||||
|
def _obj(props=None, required=()):
|
||||||
|
s = {'type': 'object', 'properties': props or {}}
|
||||||
|
if required:
|
||||||
|
s['required'] = list(required)
|
||||||
|
return s
|
||||||
|
|
||||||
|
|
||||||
|
_STR = {'type': 'string'}
|
||||||
|
_NUM = {'type': 'number'}
|
||||||
|
_TIMEOUT = {'type': 'number', 'default': 30, 'description': '秒, 上限 300'}
|
||||||
|
_IMG = {'name': {'type': 'string', 'description': 'AALC 模板图相对路径, 如 battle/turn_assets.png'},
|
||||||
|
'threshold': {'type': 'number', 'default': 0.8},
|
||||||
|
'model': {'type': 'string', 'enum': ['clam', 'normal', 'aggressive'], 'default': 'normal',
|
||||||
|
'description': 'clam=只在模板原位置找, normal=扩大范围, aggressive=全屏'}}
|
||||||
|
|
||||||
|
TOOLS = [
|
||||||
|
{'name': 'get_state', 'description': '截图 + OCR, 返回游戏窗口内全部文字(含坐标/置信度)。决策前先调这个, 不要猜坐标',
|
||||||
|
'inputSchema': _obj()},
|
||||||
|
{'name': 'click_text', 'description': '按文字点击: OCR 定位后点中心。子串匹配、忽略空格, 完全相同的优先; index 选第几个命中',
|
||||||
|
'inputSchema': _obj({'text': _STR, 'index': {'type': 'number', 'default': 0}}, ['text'])},
|
||||||
|
{'name': 'wait_for_text', 'description': '轮询 OCR 直到文字出现', 'inputSchema': _obj({'text': _STR, 'timeout': _TIMEOUT}, ['text'])},
|
||||||
|
{'name': 'wait_for_text_gone', 'description': '轮询 OCR 直到文字消失', 'inputSchema': _obj({'text': _STR, 'timeout': _TIMEOUT}, ['text'])},
|
||||||
|
{'name': 'find_image', 'description': '用 AALC 模板图在当前画面里找图标/按钮(比 OCR 稳, 适合没有文字的按钮)',
|
||||||
|
'inputSchema': _obj(_IMG, ['name'])},
|
||||||
|
{'name': 'click_image', 'description': '找到 AALC 模板图并点击它', 'inputSchema': _obj(_IMG, ['name'])},
|
||||||
|
{'name': 'list_images', 'description': '列出 AALC 模板图(按前缀过滤, 如 mirror/ battle/ home/)',
|
||||||
|
'inputSchema': _obj({'prefix': {'type': 'string', 'default': ''}})},
|
||||||
|
{'name': 'screenshot', 'description': '截游戏窗口, 返回 PNG 路径和尺寸', 'inputSchema': _obj()},
|
||||||
|
{'name': 'click', 'description': '按窗口截图像素坐标点击(优先用 click_text / click_image)',
|
||||||
|
'inputSchema': _obj({'x': _NUM, 'y': _NUM, 'times': {'type': 'number', 'default': 1}}, ['x', 'y'])},
|
||||||
|
{'name': 'keypress', 'description': '按一次键: a-z 0-9 enter esc space tab shift ctrl alt up down left right',
|
||||||
|
'inputSchema': _obj({'key': _STR}, ['key'])},
|
||||||
|
{'name': 'type_text', 'description': '向游戏窗口输入文本', 'inputSchema': _obj({'text': _STR}, ['text'])},
|
||||||
|
{'name': 'run_task', 'description': (
|
||||||
|
'后台运行 AALC 现成的完整任务流程(会自动启动游戏、处理战斗/掉线重试)。立即返回, 用 task_status 看进度。'
|
||||||
|
'all=按 AALC config.yaml 勾选的全部任务; daily=经验本+纽本(exp/thread 次数); reward=领奖励; '
|
||||||
|
'enkephalin=狂气换体力(times); mirror=镜牢(count 次数)。任务运行期间禁止手动点击类工具'),
|
||||||
|
'inputSchema': _obj({'task': {'type': 'string', 'enum': sorted(TASKS)},
|
||||||
|
'exp': _NUM, 'thread': _NUM, 'times': _NUM, 'count': _NUM}, ['task'])},
|
||||||
|
{'name': 'task_status', 'description': '查看任务状态(idle/running/succeeded/failed/stopped)和日志尾部',
|
||||||
|
'inputSchema': _obj({'lines': {'type': 'number', 'default': 30}})},
|
||||||
|
{'name': 'stop_task', 'description': '强制停止当前任务(杀掉任务进程)', 'inputSchema': _obj()},
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
class Server:
|
||||||
|
|
||||||
|
def __init__(self, engine, tasks, engine_error=None):
|
||||||
|
self.engine = engine
|
||||||
|
self.engine_error = engine_error
|
||||||
|
self.tasks = tasks
|
||||||
|
# 游戏只有一个: 所有工具调用串行, 防止两个客户端同时点
|
||||||
|
self.lock = threading.Lock()
|
||||||
|
|
||||||
|
# ---- JSON-RPC ----
|
||||||
|
def handle(self, msg):
|
||||||
|
if not isinstance(msg, dict):
|
||||||
|
return _error(None, -32600, 'request must be a JSON object')
|
||||||
|
method, mid = msg.get('method'), msg.get('id')
|
||||||
|
if method == 'initialize':
|
||||||
|
return _ok(mid, {'protocolVersion': '2024-11-05',
|
||||||
|
'capabilities': {'tools': {'listChanged': False}},
|
||||||
|
'serverInfo': {'name': 'lc-mcp', 'version': VERSION}})
|
||||||
|
if method == 'ping':
|
||||||
|
return _ok(mid, {})
|
||||||
|
if method == 'tools/list':
|
||||||
|
return _ok(mid, {'tools': TOOLS})
|
||||||
|
if method == 'tools/call':
|
||||||
|
params = msg.get('params') or {}
|
||||||
|
result = self.call_tool(params.get('name'), params.get('arguments') or {})
|
||||||
|
return _ok(mid, {'content': [{'type': 'text', 'text': json.dumps(result, ensure_ascii=False)}],
|
||||||
|
'isError': not result.get('ok', False)})
|
||||||
|
if mid is None:
|
||||||
|
return None # 通知 (notifications/*) 不回
|
||||||
|
return _error(mid, -32601, 'method not found: %s' % method)
|
||||||
|
|
||||||
|
def call_tool(self, name, args):
|
||||||
|
try:
|
||||||
|
if name in TASK_TOOLS: # 不抢游戏锁: wait_for_text 期间也能查任务/停任务
|
||||||
|
return self._task_tool(name, args)
|
||||||
|
with self.lock:
|
||||||
|
return self._call(name, args)
|
||||||
|
except (KeyError, TypeError, ValueError) as e:
|
||||||
|
return {'ok': False, 'error': 'bad arguments: %s: %s' % (type(e).__name__, e)}
|
||||||
|
except Exception as e:
|
||||||
|
return {'ok': False, 'error': '%s: %s' % (type(e).__name__, e)}
|
||||||
|
|
||||||
|
# ---- 工具 ----
|
||||||
|
def _task_tool(self, name, a):
|
||||||
|
if name == 'run_task':
|
||||||
|
task = a['task']
|
||||||
|
if task not in TASKS:
|
||||||
|
raise ValueError('unknown task %r' % task)
|
||||||
|
params = {k: int(a[k]) for k in TASKS[task] if a.get(k) is not None}
|
||||||
|
return self.tasks.start(task, params)
|
||||||
|
if name == 'task_status':
|
||||||
|
return self.tasks.status(a.get('lines', 30))
|
||||||
|
return self.tasks.stop()
|
||||||
|
|
||||||
|
def _call(self, name, a):
|
||||||
|
if not any(t['name'] == name for t in TOOLS):
|
||||||
|
return {'ok': False, 'error': 'unknown tool %s' % name}
|
||||||
|
if self.engine is None:
|
||||||
|
return {'ok': False, 'error': 'AALC 引擎不可用: %s' % self.engine_error}
|
||||||
|
if name in INPUT_TOOLS and self.tasks.running():
|
||||||
|
return {'ok': False, 'error': 'AALC 任务 %s 正在运行, 手动输入会打乱它; 先 stop_task' % self.tasks.info['task']}
|
||||||
|
e = self.engine
|
||||||
|
|
||||||
|
if name == 'screenshot':
|
||||||
|
return dict(e.screenshot(), ok=True)
|
||||||
|
if name == 'get_state':
|
||||||
|
shot, items = e.ocr()
|
||||||
|
return {'ok': True, 'screen': shot, 'items': items}
|
||||||
|
if name == 'click_text':
|
||||||
|
text = _text(a)
|
||||||
|
shot, items = e.ocr()
|
||||||
|
hits = find_hits(text, items)
|
||||||
|
if not hits:
|
||||||
|
return {'ok': False, 'error': 'text not found: %s' % text, 'screen': shot, 'items': items}
|
||||||
|
idx = int(a.get('index', 0))
|
||||||
|
if not 0 <= idx < len(hits):
|
||||||
|
return {'ok': False, 'error': 'index %d out of range (%d hits)' % (idx, len(hits)), 'hits': hits[:10]}
|
||||||
|
h = hits[idx]
|
||||||
|
e.click(h['x'], h['y'])
|
||||||
|
return {'ok': True, 'clicked': h, 'other_hits': len(hits) - 1}
|
||||||
|
if name in ('wait_for_text', 'wait_for_text_gone'):
|
||||||
|
return self._wait(_text(a), float(a.get('timeout', 30)), name == 'wait_for_text')
|
||||||
|
if name in ('find_image', 'click_image'):
|
||||||
|
pos = e.find_image(a['name'], float(a.get('threshold', 0.8)), a.get('model', 'normal'))
|
||||||
|
if pos is None:
|
||||||
|
return {'ok': name == 'find_image', 'found': False, 'name': a['name']}
|
||||||
|
if name == 'click_image':
|
||||||
|
e.click(*pos)
|
||||||
|
return {'ok': True, 'found': True, 'name': a['name'], 'x': pos[0], 'y': pos[1]}
|
||||||
|
if name == 'list_images':
|
||||||
|
imgs = e.list_images(a.get('prefix', ''))
|
||||||
|
return {'ok': True, 'count': len(imgs), 'images': imgs[:300], 'truncated': len(imgs) > 300}
|
||||||
|
if name == 'click':
|
||||||
|
e.click(int(a['x']), int(a['y']), max(1, int(a.get('times', 1))))
|
||||||
|
return {'ok': True, 'clicked': [int(a['x']), int(a['y'])]}
|
||||||
|
if name == 'keypress':
|
||||||
|
e.keypress(str(a['key']))
|
||||||
|
return {'ok': True, 'key': str(a['key'])}
|
||||||
|
if name == 'type_text':
|
||||||
|
e.type_text(str(a['text']))
|
||||||
|
return {'ok': True, 'typed': str(a['text'])}
|
||||||
|
return {'ok': False, 'error': 'unknown tool %s' % name}
|
||||||
|
|
||||||
|
def _wait(self, text, timeout, want_present):
|
||||||
|
deadline = time.monotonic() + max(0.0, min(timeout, MAX_WAIT))
|
||||||
|
items, last_err = [], None
|
||||||
|
while True:
|
||||||
|
try:
|
||||||
|
shot, items = self.engine.ocr()
|
||||||
|
hits = find_hits(text, items)
|
||||||
|
if bool(hits) == want_present:
|
||||||
|
return {'ok': True, 'present': want_present, 'hits': hits[:5], 'screen': shot}
|
||||||
|
except Exception as ex:
|
||||||
|
last_err = '%s: %s' % (type(ex).__name__, ex)
|
||||||
|
if time.monotonic() + WAIT_INTERVAL > deadline:
|
||||||
|
break
|
||||||
|
time.sleep(WAIT_INTERVAL)
|
||||||
|
r = {'ok': False, 'error': 'timeout waiting %r (want present=%s)' % (text, want_present),
|
||||||
|
'last_items': items[:15]}
|
||||||
|
if last_err:
|
||||||
|
r['last_error'] = last_err
|
||||||
|
return r
|
||||||
|
|
||||||
|
|
||||||
|
def _text(a):
|
||||||
|
text = str(a.get('text', ''))
|
||||||
|
if not text.strip():
|
||||||
|
raise ValueError('text must be non-empty')
|
||||||
|
return text
|
||||||
|
|
||||||
|
|
||||||
|
def _ok(mid, result):
|
||||||
|
return {'jsonrpc': '2.0', 'id': mid, 'result': result}
|
||||||
|
|
||||||
|
|
||||||
|
def _error(mid, code, message):
|
||||||
|
return {'jsonrpc': '2.0', 'id': mid, 'error': {'code': code, 'message': message}}
|
||||||
|
|
||||||
|
|
||||||
|
def parse_line(line):
|
||||||
|
"""一行 -> (msg, error_response)。空行返回 (None, None)。"""
|
||||||
|
line = line.strip()
|
||||||
|
if not line:
|
||||||
|
return None, None
|
||||||
|
try:
|
||||||
|
return json.loads(line), None
|
||||||
|
except ValueError as e:
|
||||||
|
return None, _error(None, -32700, 'parse error: %s' % e)
|
||||||
@@ -0,0 +1,136 @@
|
|||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
"""lc-mcp 常驻服务: 跑在 Windows 交互桌面会话里(登录时由计划任务启动)。
|
||||||
|
|
||||||
|
python -m lc_mcp.server 监听 127.0.0.1:PORT, MCP 客户端经 bridge.py 连入
|
||||||
|
python -m lc_mcp.server --stdio 直接走 stdio (本机、且本进程就在桌面会话时用)
|
||||||
|
--fake 用 FakeEngine (开发/测试, 不需要 Windows/AALC)
|
||||||
|
|
||||||
|
TCP 协议: 首行 "AUTH <token>", 服务端回 "OK"; 之后逐行 JSON-RPC。
|
||||||
|
token 存在 ~/lc_mcp/token (用户目录默认只有本人可读), 防止同机其他用户/进程连进来操作游戏。
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import hmac
|
||||||
|
import os
|
||||||
|
import secrets
|
||||||
|
import socketserver
|
||||||
|
import sys
|
||||||
|
import threading
|
||||||
|
|
||||||
|
from .protocol import Server, parse_line
|
||||||
|
from .tasks import TaskRunner
|
||||||
|
|
||||||
|
HOME = os.path.join(os.path.expanduser('~'), 'lc_mcp')
|
||||||
|
TOKEN_FILE = os.path.join(HOME, 'token')
|
||||||
|
DEFAULT_PORT = int(os.environ.get('LCP_PORT', '8765'))
|
||||||
|
|
||||||
|
|
||||||
|
def log(msg):
|
||||||
|
sys.stderr.write('[lc_mcp] %s\n' % msg)
|
||||||
|
sys.stderr.flush()
|
||||||
|
|
||||||
|
|
||||||
|
def load_token(create=False):
|
||||||
|
try:
|
||||||
|
with open(TOKEN_FILE, encoding='ascii') as f:
|
||||||
|
tok = f.read().strip()
|
||||||
|
if tok:
|
||||||
|
return tok
|
||||||
|
except OSError:
|
||||||
|
if not create:
|
||||||
|
raise
|
||||||
|
os.makedirs(HOME, exist_ok=True)
|
||||||
|
tok = secrets.token_hex(24)
|
||||||
|
fd = os.open(TOKEN_FILE, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
|
||||||
|
with os.fdopen(fd, 'w', encoding='ascii') as f:
|
||||||
|
f.write(tok)
|
||||||
|
return tok
|
||||||
|
|
||||||
|
|
||||||
|
def build_server(fake=False):
|
||||||
|
os.makedirs(HOME, exist_ok=True)
|
||||||
|
tasks = TaskRunner(os.path.join(HOME, 'runs'))
|
||||||
|
if fake:
|
||||||
|
from .fake_engine import FakeEngine
|
||||||
|
return Server(FakeEngine(), tasks)
|
||||||
|
try:
|
||||||
|
from .aalc_engine import AalcEngine
|
||||||
|
return Server(AalcEngine(os.path.join(HOME, 'shots'), int(os.environ.get('LCP_KEEP_SHOTS', '30'))), tasks)
|
||||||
|
except Exception as e:
|
||||||
|
# 引擎起不来也继续服务: run_task 仍可用, 其他工具返回原因
|
||||||
|
log('AALC engine init failed: %r' % e)
|
||||||
|
return Server(None, tasks, engine_error='%s: %s' % (type(e).__name__, e))
|
||||||
|
|
||||||
|
|
||||||
|
def serve_lines(server, readline, write):
|
||||||
|
"""逐行处理直到 EOF。坏行回 parse error 后继续, 不退出。"""
|
||||||
|
import json
|
||||||
|
while True:
|
||||||
|
line = readline()
|
||||||
|
if not line:
|
||||||
|
return
|
||||||
|
msg, err = parse_line(line)
|
||||||
|
resp = err if err else (server.handle(msg) if msg is not None else None)
|
||||||
|
if resp is not None:
|
||||||
|
write(json.dumps(resp, ensure_ascii=False) + '\n')
|
||||||
|
|
||||||
|
|
||||||
|
def run_stdio(server):
|
||||||
|
out = sys.stdout
|
||||||
|
sys.stdout = sys.stderr # AALC 或第三方库的 print 不能混进协议流
|
||||||
|
|
||||||
|
def write(s):
|
||||||
|
out.write(s)
|
||||||
|
out.flush()
|
||||||
|
serve_lines(server, sys.stdin.readline, write)
|
||||||
|
|
||||||
|
|
||||||
|
class _Handler(socketserver.StreamRequestHandler):
|
||||||
|
|
||||||
|
def handle(self):
|
||||||
|
rfile, wfile = self.rfile, self.wfile
|
||||||
|
first = rfile.readline(256).decode('utf-8', 'replace').strip()
|
||||||
|
if not (first.startswith('AUTH ') and hmac.compare_digest(first[5:], self.server.token)):
|
||||||
|
wfile.write(b'DENIED\n')
|
||||||
|
log('rejected connection from %s' % (self.client_address,))
|
||||||
|
return
|
||||||
|
wfile.write(b'OK\n')
|
||||||
|
wfile.flush()
|
||||||
|
wlock = threading.Lock()
|
||||||
|
|
||||||
|
def write(s):
|
||||||
|
with wlock:
|
||||||
|
wfile.write(s.encode('utf-8'))
|
||||||
|
wfile.flush()
|
||||||
|
serve_lines(self.server.mcp, lambda: rfile.readline().decode('utf-8', 'replace'), write)
|
||||||
|
|
||||||
|
|
||||||
|
class TCPServer(socketserver.ThreadingTCPServer):
|
||||||
|
daemon_threads = True
|
||||||
|
allow_reuse_address = False # Windows 上 SO_REUSEADDR 允许端口劫持
|
||||||
|
|
||||||
|
|
||||||
|
def make_tcp(mcp, host, port, token):
|
||||||
|
srv = TCPServer((host, port), _Handler)
|
||||||
|
srv.mcp, srv.token = mcp, token
|
||||||
|
return srv
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv=None):
|
||||||
|
ap = argparse.ArgumentParser(prog='lc_mcp.server')
|
||||||
|
ap.add_argument('--stdio', action='store_true')
|
||||||
|
ap.add_argument('--fake', action='store_true')
|
||||||
|
ap.add_argument('--host', default='127.0.0.1')
|
||||||
|
ap.add_argument('--port', type=int, default=DEFAULT_PORT)
|
||||||
|
args = ap.parse_args(argv)
|
||||||
|
mcp = build_server(fake=args.fake)
|
||||||
|
if args.stdio:
|
||||||
|
return run_stdio(mcp)
|
||||||
|
if args.host not in ('127.0.0.1', '::1', 'localhost'):
|
||||||
|
log('WARNING: 监听 %s 会把游戏控制权暴露到网络上, 建议走 SSH 隧道' % args.host)
|
||||||
|
srv = make_tcp(mcp, args.host, args.port, load_token(create=True))
|
||||||
|
log('listening on %s:%d (token: %s)' % (args.host, args.port, TOKEN_FILE))
|
||||||
|
srv.serve_forever()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
main()
|
||||||
+107
@@ -0,0 +1,107 @@
|
|||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
"""长任务管理: 同一时刻最多一个 AALC 任务子进程。"""
|
||||||
|
import glob
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import signal
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
|
||||||
|
REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||||
|
IS_WIN = sys.platform.startswith('win')
|
||||||
|
|
||||||
|
|
||||||
|
def default_argv(task, params):
|
||||||
|
return [sys.executable, '-m', 'lc_mcp.aalc_task', task, json.dumps(params)]
|
||||||
|
|
||||||
|
|
||||||
|
def tail(path, n):
|
||||||
|
try:
|
||||||
|
with open(path, 'rb') as f:
|
||||||
|
f.seek(0, os.SEEK_END)
|
||||||
|
f.seek(max(0, f.tell() - 64 * 1024))
|
||||||
|
lines = f.read().decode('utf-8', 'replace').splitlines()
|
||||||
|
return lines[-n:]
|
||||||
|
except OSError:
|
||||||
|
return []
|
||||||
|
|
||||||
|
|
||||||
|
class TaskRunner:
|
||||||
|
|
||||||
|
def __init__(self, run_dir, argv_factory=default_argv, keep_logs=20):
|
||||||
|
self.run_dir = os.path.abspath(run_dir)
|
||||||
|
os.makedirs(self.run_dir, exist_ok=True)
|
||||||
|
self.argv_factory = argv_factory
|
||||||
|
self.keep_logs = keep_logs
|
||||||
|
self._lock = threading.Lock()
|
||||||
|
self.proc = None
|
||||||
|
self.info = None
|
||||||
|
|
||||||
|
def running(self):
|
||||||
|
return self.proc is not None and self.proc.poll() is None
|
||||||
|
|
||||||
|
def start(self, task, params):
|
||||||
|
with self._lock:
|
||||||
|
if self.running():
|
||||||
|
return {'ok': False, 'error': '已有任务在跑: %s (先 stop_task)' % self.info['task']}
|
||||||
|
stamp = time.strftime('%Y%m%d_%H%M%S')
|
||||||
|
log_path = os.path.join(self.run_dir, 'task_%s_%s.log' % (stamp, task))
|
||||||
|
env = dict(os.environ, PYTHONPATH=REPO + os.pathsep + os.environ.get('PYTHONPATH', ''),
|
||||||
|
PYTHONIOENCODING='utf-8', PYTHONUNBUFFERED='1')
|
||||||
|
kw = {'creationflags': subprocess.CREATE_NEW_PROCESS_GROUP} if IS_WIN else {'start_new_session': True}
|
||||||
|
with open(log_path, 'wb') as logf:
|
||||||
|
self.proc = subprocess.Popen(self.argv_factory(task, params), cwd=REPO, env=env,
|
||||||
|
stdin=subprocess.DEVNULL, stdout=logf, stderr=subprocess.STDOUT, **kw)
|
||||||
|
self.info = {'task': task, 'params': params, 'log': log_path.replace('\\', '/'),
|
||||||
|
'started': time.time(), 'stopped_by_user': False}
|
||||||
|
self._prune()
|
||||||
|
return {'ok': True, 'task': task, 'pid': self.proc.pid, 'log': self.info['log']}
|
||||||
|
|
||||||
|
def status(self, lines=30):
|
||||||
|
with self._lock:
|
||||||
|
if self.info is None:
|
||||||
|
return {'ok': True, 'state': 'idle'}
|
||||||
|
code = self.proc.poll()
|
||||||
|
if code is None:
|
||||||
|
state = 'running'
|
||||||
|
elif self.info['stopped_by_user']:
|
||||||
|
state = 'stopped'
|
||||||
|
else:
|
||||||
|
state = 'succeeded' if code == 0 else 'failed'
|
||||||
|
r = {'ok': True, 'state': state, 'task': self.info['task'], 'params': self.info['params'],
|
||||||
|
'elapsed_s': int((self.info.get('ended') or time.time()) - self.info['started']),
|
||||||
|
'log': self.info['log'], 'log_tail': tail(self.info['log'], max(1, min(int(lines), 200)))}
|
||||||
|
if code is not None:
|
||||||
|
r['exit_code'] = code
|
||||||
|
self.info.setdefault('ended', time.time())
|
||||||
|
return r
|
||||||
|
|
||||||
|
def stop(self):
|
||||||
|
with self._lock:
|
||||||
|
if not self.running():
|
||||||
|
return {'ok': True, 'state': 'idle', 'note': '没有在跑的任务'}
|
||||||
|
self.info['stopped_by_user'] = True
|
||||||
|
pid = self.proc.pid
|
||||||
|
if IS_WIN:
|
||||||
|
subprocess.run(['taskkill', '/PID', str(pid), '/T', '/F'], capture_output=True)
|
||||||
|
else:
|
||||||
|
try:
|
||||||
|
os.killpg(pid, signal.SIGKILL)
|
||||||
|
except ProcessLookupError:
|
||||||
|
pass
|
||||||
|
try:
|
||||||
|
self.proc.wait(timeout=10)
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
return {'ok': False, 'error': '进程 %d 10 秒内没退出' % pid}
|
||||||
|
self.info['ended'] = time.time()
|
||||||
|
return {'ok': True, 'state': 'stopped', 'task': self.info['task']}
|
||||||
|
|
||||||
|
def _prune(self):
|
||||||
|
logs = sorted(glob.glob(os.path.join(self.run_dir, 'task_*.log')), key=os.path.getmtime)
|
||||||
|
for p in logs[:-self.keep_logs]:
|
||||||
|
try:
|
||||||
|
os.remove(p)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
@@ -1,75 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
"""本地 OCR:Windows.Media.Ocr(winsdk),零远端依赖。
|
|
||||||
只支持 Windows;Linux/mac 上 import winsdk 会失败 -> 用 remote 后端。
|
|
||||||
"""
|
|
||||||
import asyncio
|
|
||||||
|
|
||||||
|
|
||||||
def available():
|
|
||||||
try:
|
|
||||||
from winsdk.windows.media.ocr import OcrEngine # noqa: F401
|
|
||||||
return True
|
|
||||||
except Exception:
|
|
||||||
return False
|
|
||||||
|
|
||||||
|
|
||||||
def _engine():
|
|
||||||
from winsdk.windows.media.ocr import OcrEngine
|
|
||||||
from winsdk.windows.globalization import Language
|
|
||||||
try:
|
|
||||||
eng = OcrEngine.try_create_from_language(Language('zh-Hans'))
|
|
||||||
if eng:
|
|
||||||
return eng
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
return OcrEngine.try_create_from_user_profile_languages()
|
|
||||||
|
|
||||||
|
|
||||||
async def _ocr_async(path):
|
|
||||||
from winsdk.windows.media.ocr import OcrEngine # noqa: F401
|
|
||||||
from winsdk.windows.storage import StorageFile, FileAccessMode
|
|
||||||
from winsdk.windows.graphics.imaging import BitmapDecoder
|
|
||||||
|
|
||||||
file = await StorageFile.get_file_from_path_async(path)
|
|
||||||
stream = await file.open_async(FileAccessMode.READ)
|
|
||||||
decoder = await BitmapDecoder.create_async(stream)
|
|
||||||
bitmap = await decoder.get_software_bitmap_async()
|
|
||||||
|
|
||||||
eng = _engine()
|
|
||||||
if eng is None:
|
|
||||||
return None, 'no OCR engine (language pack missing?)'
|
|
||||||
result = await eng.recognize_async(bitmap)
|
|
||||||
|
|
||||||
items = []
|
|
||||||
for line in result.lines:
|
|
||||||
words = line.words
|
|
||||||
if not words:
|
|
||||||
continue
|
|
||||||
text = ' '.join(w.text for w in words)
|
|
||||||
w0 = words[0].bounding_rect
|
|
||||||
x0, y0 = w0.x, w0.y
|
|
||||||
x1, y1 = w0.x + w0.width, w0.y + w0.height
|
|
||||||
for w in words[1:]:
|
|
||||||
r = w.bounding_rect
|
|
||||||
if r.x < x0:
|
|
||||||
x0 = r.x
|
|
||||||
if r.y < y0:
|
|
||||||
y0 = r.y
|
|
||||||
if r.x + r.width > x1:
|
|
||||||
x1 = r.x + r.width
|
|
||||||
if r.y + r.height > y1:
|
|
||||||
y1 = r.y + r.height
|
|
||||||
items.append({
|
|
||||||
'text': text,
|
|
||||||
'x': int(x0), 'y': int(y0),
|
|
||||||
'w': int(x1 - x0), 'h': int(y1 - y0),
|
|
||||||
})
|
|
||||||
return items, None
|
|
||||||
|
|
||||||
|
|
||||||
def ocr_image(path):
|
|
||||||
"""同步接口。返回 (items, None) 或 (None, error)。"""
|
|
||||||
try:
|
|
||||||
return asyncio.run(_ocr_async(path))
|
|
||||||
except Exception as e:
|
|
||||||
return None, 'local ocr fail: %s' % e
|
|
||||||
@@ -1,58 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
"""平台抽象:截图 + 输入后端,按平台注册。
|
|
||||||
core (MCP协议/OCR/语义工具) 平台无关。"""
|
|
||||||
|
|
||||||
import os
|
|
||||||
import sys
|
|
||||||
|
|
||||||
|
|
||||||
def detect_platform():
|
|
||||||
p = os.environ.get('LCP_PLATFORM', '')
|
|
||||||
if p:
|
|
||||||
return p.lower()
|
|
||||||
if sys.platform.startswith('win'):
|
|
||||||
return 'win'
|
|
||||||
if sys.platform == 'darwin':
|
|
||||||
return 'mac'
|
|
||||||
return 'linux'
|
|
||||||
|
|
||||||
|
|
||||||
def get_screenshot_provider(platform=None):
|
|
||||||
platform = platform or detect_platform()
|
|
||||||
if platform == 'win':
|
|
||||||
from .win import WinScreenshotProvider
|
|
||||||
return WinScreenshotProvider()
|
|
||||||
if platform == 'linux':
|
|
||||||
from .linux import LinuxScreenshotProvider
|
|
||||||
return LinuxScreenshotProvider()
|
|
||||||
if platform == 'mac':
|
|
||||||
from .mac import MacScreenshotProvider
|
|
||||||
return MacScreenshotProvider()
|
|
||||||
raise ValueError('no screenshot provider for %s' % platform)
|
|
||||||
|
|
||||||
|
|
||||||
def get_input_backend(name=None, platform=None):
|
|
||||||
platform = platform or detect_platform()
|
|
||||||
name = (name or os.environ.get('INPUT_BACKEND', '') or
|
|
||||||
{'win': 'sendinput', 'linux': 'xdotool', 'mac': 'cg'}[platform]).lower()
|
|
||||||
if name == 'mock':
|
|
||||||
from backends.mock import MockBackend
|
|
||||||
return MockBackend()
|
|
||||||
if platform == 'win':
|
|
||||||
if name == 'sendinput':
|
|
||||||
from backends.sendinput_win import SendInputBackend
|
|
||||||
return SendInputBackend()
|
|
||||||
if name == 'interception':
|
|
||||||
from backends.interception_win import InterceptionBackend
|
|
||||||
return InterceptionBackend()
|
|
||||||
if platform == 'linux':
|
|
||||||
if name == 'xdotool':
|
|
||||||
from backends.xdotool_linux import XdotoolBackend
|
|
||||||
return XdotoolBackend()
|
|
||||||
if name == 'uinput':
|
|
||||||
from backends.uinput_linux import UinputBackend
|
|
||||||
return UinputBackend()
|
|
||||||
if platform == 'mac' and name == 'cg':
|
|
||||||
from backends.cg_mac import CGBackend
|
|
||||||
return CGBackend()
|
|
||||||
raise ValueError('no input backend %s for %s' % (name, platform))
|
|
||||||
@@ -1,34 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
"""Linux 截图提供者:X11 (PIL ImageGrab / scrot 兜底)。
|
|
||||||
Wayland 下 ImageGrab 不可用,需额外方案(见 README)。
|
|
||||||
"""
|
|
||||||
import os
|
|
||||||
import shutil
|
|
||||||
import subprocess
|
|
||||||
import tempfile
|
|
||||||
import time
|
|
||||||
|
|
||||||
|
|
||||||
class LinuxScreenshotProvider:
|
|
||||||
|
|
||||||
def grab(self):
|
|
||||||
# 1) PIL ImageGrab (X11)
|
|
||||||
try:
|
|
||||||
from PIL import ImageGrab
|
|
||||||
img = ImageGrab.grab()
|
|
||||||
path = os.path.join(tempfile.gettempdir(), 'lc_shot_%s.png' % time.strftime('%H%M%S'))
|
|
||||||
img.save(path)
|
|
||||||
return {'ok': True, 'path': path}
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
# 2) scrot / import (ImageMagick) / gnome-screenshot 兜底
|
|
||||||
for cmd in (['scrot', '-z'], ['import', '-window', 'root'], ['gnome-screenshot', '-f']):
|
|
||||||
if shutil.which(cmd[0]):
|
|
||||||
path = os.path.join(tempfile.gettempdir(), 'lc_shot_%s.png' % time.strftime('%H%M%S'))
|
|
||||||
try:
|
|
||||||
subprocess.run(cmd + [path], check=True, timeout=20)
|
|
||||||
if os.path.exists(path):
|
|
||||||
return {'ok': True, 'path': path}
|
|
||||||
except Exception:
|
|
||||||
continue
|
|
||||||
return {'ok': False, 'error': 'no working screenshot method on this Linux (need X11 + PIL/scrot)'}
|
|
||||||
@@ -1,20 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
"""macOS 截图提供者:screencapture CLI (系统自带)。"""
|
|
||||||
|
|
||||||
import os
|
|
||||||
import subprocess
|
|
||||||
import tempfile
|
|
||||||
import time
|
|
||||||
|
|
||||||
|
|
||||||
class MacScreenshotProvider:
|
|
||||||
|
|
||||||
def grab(self):
|
|
||||||
path = os.path.join(tempfile.gettempdir(), 'lc_shot_%s.png' % time.strftime('%H%M%S'))
|
|
||||||
try:
|
|
||||||
subprocess.run(['screencapture', '-x', path], check=True, timeout=20)
|
|
||||||
if os.path.exists(path):
|
|
||||||
return {'ok': True, 'path': path}
|
|
||||||
except Exception as e:
|
|
||||||
return {'ok': False, 'error': 'screencapture fail: %s' % e}
|
|
||||||
return {'ok': False, 'error': 'no png produced'}
|
|
||||||
@@ -1,76 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
"""Windows 截图提供者:交互计划任务(LogonType Interactive)。
|
|
||||||
OpenSSH 服务会话(session 0)没有交互桌面,截图/前端输入必须在交互会话执行。
|
|
||||||
"""
|
|
||||||
import os
|
|
||||||
import subprocess
|
|
||||||
import time
|
|
||||||
|
|
||||||
USER = os.environ.get('USERNAME', 'yws-hhh')
|
|
||||||
BASE = os.path.join(os.path.expanduser('~'), 'lc_mcp')
|
|
||||||
os.makedirs(BASE, exist_ok=True)
|
|
||||||
OUT_DIR = r'C:\ProgramData\hduty'
|
|
||||||
os.makedirs(OUT_DIR, exist_ok=True)
|
|
||||||
JOB_PS1 = os.path.join(BASE, 'job.ps1')
|
|
||||||
RUN_VBS = os.path.join(BASE, 'run.vbs')
|
|
||||||
TASK = 'LcMCP'
|
|
||||||
|
|
||||||
if not os.path.exists(RUN_VBS):
|
|
||||||
with open(RUN_VBS, 'w', encoding='utf-8') as f:
|
|
||||||
f.write('CreateObject("WScript.Shell").Run "powershell.exe -NoProfile -ExecutionPolicy Bypass -File "%s"", 0, False\n' % JOB_PS1)
|
|
||||||
|
|
||||||
|
|
||||||
def _pshell(code):
|
|
||||||
r = subprocess.run(['powershell.exe', '-NoProfile', '-ExecutionPolicy', 'Bypass', '-Command', code],
|
|
||||||
capture_output=True, text=True, encoding='utf-8', errors='replace', timeout=60)
|
|
||||||
return r.returncode, (r.stdout or ''), (r.stderr or '')
|
|
||||||
|
|
||||||
|
|
||||||
def _run_interactive(ps1_code, expect_file, timeout=15):
|
|
||||||
with open(JOB_PS1, 'w', encoding='utf-8') as f:
|
|
||||||
f.write('$ErrorActionPreference="Stop"\n' + ps1_code)
|
|
||||||
try:
|
|
||||||
if os.path.exists(expect_file):
|
|
||||||
os.remove(expect_file)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
reg = ("Unregister-ScheduledTask -TaskName %s -Confirm:$false -ErrorAction SilentlyContinue; "
|
|
||||||
"$A=New-ScheduledTaskAction -Execute 'wscript.exe' -Argument '%s'; "
|
|
||||||
"$P=New-ScheduledTaskPrincipal -UserId '%s' -LogonType Interactive -RunLevel Highest; "
|
|
||||||
"Register-ScheduledTask -TaskName %s -Action $A -Principal $P -Force | Out-Null; "
|
|
||||||
"Start-ScheduledTask -TaskName %s; 'OK'"
|
|
||||||
% (TASK, RUN_VBS.replace("'", "''"), USER, TASK, TASK))
|
|
||||||
rc, out, err = _pshell(reg)
|
|
||||||
if 'OK' not in out:
|
|
||||||
return {"ok": False, "error": "task start fail: %s %s" % (out[-150:], err[-150:])}
|
|
||||||
deadline = time.time() + timeout
|
|
||||||
while time.time() < deadline:
|
|
||||||
if os.path.exists(expect_file):
|
|
||||||
try:
|
|
||||||
with open(expect_file, 'r', encoding='utf-8') as f:
|
|
||||||
return {"ok": True, "out": f.read().strip()}
|
|
||||||
except Exception:
|
|
||||||
return {"ok": True, "out": ''}
|
|
||||||
time.sleep(0.4)
|
|
||||||
return {"ok": False, "error": "timeout waiting %s" % expect_file}
|
|
||||||
|
|
||||||
|
|
||||||
class WinScreenshotProvider:
|
|
||||||
|
|
||||||
def grab(self):
|
|
||||||
name = 'shot_%s.png' % time.strftime('%Y%m%d_%H%M%S')
|
|
||||||
img_path = os.path.join(OUT_DIR, name)
|
|
||||||
ps = ("Add-Type -AssemblyName System.Windows.Forms,System.Drawing; "
|
|
||||||
"$b=[System.Windows.Forms.Screen]::PrimaryScreen.Bounds; "
|
|
||||||
"$bmp=New-Object System.Drawing.Bitmap $b.Width,$b.Height; "
|
|
||||||
"$g=[System.Drawing.Graphics]::FromImage($bmp); "
|
|
||||||
"$g.CopyFromScreen($b.X,$b.Y,0,0,$b.Size); "
|
|
||||||
"$bmp.Save('%s'); $g.Dispose(); $bmp.Dispose(); "
|
|
||||||
"$f=Join-Path '%s' 'done.txt'; 'DONE' | Set-Content $f"
|
|
||||||
% (img_path, OUT_DIR))
|
|
||||||
r = _run_interactive(ps, os.path.join(OUT_DIR, 'done.txt'), timeout=20)
|
|
||||||
if not r.get('ok'):
|
|
||||||
return {'ok': False, 'error': r.get('error', 'screenshot fail')}
|
|
||||||
if not os.path.exists(img_path):
|
|
||||||
return {'ok': False, 'error': 'png not produced'}
|
|
||||||
return {'ok': True, 'path': img_path.replace('\\', '/')}
|
|
||||||
@@ -0,0 +1,38 @@
|
|||||||
|
# 注册登录时启动的 lc-mcp 常驻服务 (交互会话, 能看到游戏窗口)。
|
||||||
|
# 用法 (在要跑游戏的那个账号下, 普通 PowerShell 即可):
|
||||||
|
# powershell -ExecutionPolicy Bypass -File scripts\install_logon_task.ps1
|
||||||
|
# 可选: -Python <python.exe 路径> 默认用 vendor\aalc\.venv (uv sync 生成)
|
||||||
|
# -Elevated 游戏以管理员运行时才需要(UIPI 会挡住低权限进程的输入)
|
||||||
|
# -Uninstall
|
||||||
|
param(
|
||||||
|
[string]$Python = "",
|
||||||
|
[switch]$Elevated,
|
||||||
|
[switch]$Uninstall
|
||||||
|
)
|
||||||
|
$ErrorActionPreference = "Stop"
|
||||||
|
$TaskName = "LcMCPServer"
|
||||||
|
$Repo = Split-Path -Parent $PSScriptRoot
|
||||||
|
|
||||||
|
if ($Uninstall) {
|
||||||
|
Unregister-ScheduledTask -TaskName $TaskName -Confirm:$false -ErrorAction SilentlyContinue
|
||||||
|
Write-Output "removed $TaskName"
|
||||||
|
exit 0
|
||||||
|
}
|
||||||
|
|
||||||
|
if (-not $Python) { $Python = Join-Path $Repo "vendor\aalc\.venv\Scripts\pythonw.exe" }
|
||||||
|
if (-not (Test-Path $Python)) {
|
||||||
|
throw "找不到 Python: $Python`n先在 vendor\aalc 里执行 uv sync, 或用 -Python 指定 AALC 依赖齐全的解释器"
|
||||||
|
}
|
||||||
|
|
||||||
|
$user = "$env:USERDOMAIN\$env:USERNAME"
|
||||||
|
$action = New-ScheduledTaskAction -Execute $Python -Argument "-m lc_mcp.server" -WorkingDirectory $Repo
|
||||||
|
$trigger = New-ScheduledTaskTrigger -AtLogOn -User $user
|
||||||
|
$level = if ($Elevated) { "Highest" } else { "Limited" }
|
||||||
|
$principal = New-ScheduledTaskPrincipal -UserId $user -LogonType Interactive -RunLevel $level
|
||||||
|
$settings = New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries `
|
||||||
|
-ExecutionTimeLimit ([TimeSpan]::Zero) -RestartCount 3 -RestartInterval (New-TimeSpan -Minutes 1) `
|
||||||
|
-MultipleInstances IgnoreNew
|
||||||
|
Register-ScheduledTask -TaskName $TaskName -Action $action -Trigger $trigger -Principal $principal `
|
||||||
|
-Settings $settings -Force | Out-Null
|
||||||
|
Start-ScheduledTask -TaskName $TaskName
|
||||||
|
Write-Output "registered + started $TaskName ($level) -> $Python -m lc_mcp.server"
|
||||||
@@ -1,63 +0,0 @@
|
|||||||
# -*- coding: utf-8 -*-
|
|
||||||
"""后端/协议测试(本机直接跑,无需 Windows)。"""
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
|
|
||||||
HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
|
||||||
sys.path.insert(0, HERE)
|
|
||||||
|
|
||||||
from platform import detect_platform, get_input_backend # noqa: E402
|
|
||||||
|
|
||||||
|
|
||||||
def test_mock_backend():
|
|
||||||
b = get_input_backend('mock')
|
|
||||||
assert b.click(100, 200)['ok']
|
|
||||||
assert b.move(10, 10)['ok']
|
|
||||||
assert b.keypress('p')['ok']
|
|
||||||
assert b.type_text('ok')['ok']
|
|
||||||
acts = [x['act'] for x in b.log]
|
|
||||||
assert acts == ['click', 'move', 'keypress', 'type_text'], acts
|
|
||||||
print('mock backend: OK', acts)
|
|
||||||
|
|
||||||
|
|
||||||
def test_scancode_table():
|
|
||||||
from backends.interception_win import SCAN, UPPER_SCAN
|
|
||||||
assert SCAN['a'] == 0x1E and SCAN['p'] == 0x19 and SCAN['z'] == 0x2C
|
|
||||||
assert SCAN['enter'] == 0x1C and SCAN['esc'] == 0x01
|
|
||||||
assert UPPER_SCAN['!'] == 0x02
|
|
||||||
# 连续无缺失
|
|
||||||
for ch in 'abcdefghijklmnopqrstuvwxyz0123456789':
|
|
||||||
assert ch in SCAN, 'missing %s' % ch
|
|
||||||
print('scancode table: OK')
|
|
||||||
|
|
||||||
|
|
||||||
def test_mcp_roundtrip_mock():
|
|
||||||
env = dict(os.environ, INPUT_BACKEND='mock')
|
|
||||||
p = subprocess.Popen([sys.executable, os.path.join(HERE, 'lc_mcp.py')],
|
|
||||||
stdin=subprocess.PIPE, stdout=subprocess.PIPE,
|
|
||||||
stderr=subprocess.PIPE, text=True, encoding='utf-8',
|
|
||||||
errors='replace', env=env)
|
|
||||||
def call(obj):
|
|
||||||
p.stdin.write(json.dumps(obj) + '\n')
|
|
||||||
p.stdin.flush()
|
|
||||||
return json.loads(p.stdout.readline())
|
|
||||||
r = call({'jsonrpc': '2.0', 'id': 1, 'method': 'initialize', 'params': {}})
|
|
||||||
assert r['result']['serverInfo']['name'] == 'lc-mcp'
|
|
||||||
r = call({'jsonrpc': '2.0', 'id': 2, 'method': 'tools/list', 'params': {}})
|
|
||||||
names = [t['name'] for t in r['result']['tools']]
|
|
||||||
assert 'get_state' in names and 'click_text' in names and 'wait_for_text' in names and 'wait_for_text_gone' in names, names
|
|
||||||
assert 'screenshot' in names and 'keypress' in names, names
|
|
||||||
r = call({'jsonrpc': '2.0', 'id': 3, 'method': 'tools/call',
|
|
||||||
'params': {'name': 'move', 'arguments': {'x': 5, 'y': 5}}})
|
|
||||||
assert 'mock' in r['result']['content'][0]['text']
|
|
||||||
p.kill()
|
|
||||||
print('mcp roundtrip(mock): OK', names)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
test_mock_backend()
|
|
||||||
test_scancode_table()
|
|
||||||
test_mcp_roundtrip_mock()
|
|
||||||
print('ALL TESTS PASSED')
|
|
||||||
@@ -0,0 +1,217 @@
|
|||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
"""不需要 Windows/AALC 的测试: python -m unittest discover -s tests"""
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import socket
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
import unittest
|
||||||
|
from unittest import mock
|
||||||
|
|
||||||
|
REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||||
|
sys.path.insert(0, REPO)
|
||||||
|
|
||||||
|
from lc_mcp import server as srv_mod # noqa: E402
|
||||||
|
from lc_mcp.engine import find_hits # noqa: E402
|
||||||
|
from lc_mcp.fake_engine import FakeEngine # noqa: E402
|
||||||
|
from lc_mcp.protocol import Server, TOOLS # noqa: E402
|
||||||
|
from lc_mcp.tasks import TaskRunner # noqa: E402
|
||||||
|
|
||||||
|
START = {'text': '开始', 'x': 100, 'y': 200, 'w': 40, 'h': 20}
|
||||||
|
|
||||||
|
|
||||||
|
def call(server, tool, **args):
|
||||||
|
resp = server.handle({'jsonrpc': '2.0', 'id': 1, 'method': 'tools/call',
|
||||||
|
'params': {'name': tool, 'arguments': args}})
|
||||||
|
res = resp['result']
|
||||||
|
data = json.loads(res['content'][0]['text'])
|
||||||
|
assert res['isError'] == (not data.get('ok', False)), res
|
||||||
|
return data
|
||||||
|
|
||||||
|
|
||||||
|
def sleeper_argv(seconds):
|
||||||
|
code = 'import sys,time; print("child", sys.argv[1:], flush=True); time.sleep(%s)' % seconds
|
||||||
|
return lambda task, params: [sys.executable, '-c', code, task, json.dumps(params)]
|
||||||
|
|
||||||
|
|
||||||
|
class MatchTest(unittest.TestCase):
|
||||||
|
|
||||||
|
def test_ignores_spaces_and_case(self):
|
||||||
|
items = [{'text': 'Hong Lu', 'x': 0, 'y': 0, 'w': 10, 'h': 10}]
|
||||||
|
self.assertEqual(find_hits('honglu', items)[0]['x'], 5)
|
||||||
|
|
||||||
|
def test_exact_first(self):
|
||||||
|
items = [{'text': '开始战斗', 'x': 0, 'y': 0, 'w': 10, 'h': 10}, START]
|
||||||
|
self.assertEqual(find_hits('开始', items)[0]['text'], '开始')
|
||||||
|
|
||||||
|
def test_empty_matches_nothing(self):
|
||||||
|
self.assertEqual(find_hits(' ', [START]), [])
|
||||||
|
|
||||||
|
|
||||||
|
class ProtocolTest(unittest.TestCase):
|
||||||
|
|
||||||
|
def setUp(self):
|
||||||
|
self.tmp = tempfile.TemporaryDirectory()
|
||||||
|
self.tasks = TaskRunner(self.tmp.name, argv_factory=sleeper_argv(5))
|
||||||
|
self.engine = FakeEngine(frames=[[START]], images={'battle/turn_assets.png': (50, 60)})
|
||||||
|
self.s = Server(self.engine, self.tasks)
|
||||||
|
|
||||||
|
def tearDown(self):
|
||||||
|
self.tasks.stop()
|
||||||
|
self.tmp.cleanup()
|
||||||
|
|
||||||
|
def test_initialize_and_list(self):
|
||||||
|
r = self.s.handle({'jsonrpc': '2.0', 'id': 7, 'method': 'initialize', 'params': {}})
|
||||||
|
self.assertEqual(r['id'], 7)
|
||||||
|
names = [t['name'] for t in self.s.handle({'jsonrpc': '2.0', 'id': 8, 'method': 'tools/list'})['result']['tools']]
|
||||||
|
self.assertEqual(names, [t['name'] for t in TOOLS])
|
||||||
|
self.assertIsNone(self.s.handle({'jsonrpc': '2.0', 'method': 'notifications/initialized'}))
|
||||||
|
self.assertEqual(self.s.handle({'jsonrpc': '2.0', 'id': 9, 'method': 'nope'})['error']['code'], -32601)
|
||||||
|
|
||||||
|
def test_click_text(self):
|
||||||
|
r = call(self.s, 'click_text', text='开 始')
|
||||||
|
self.assertTrue(r['ok'])
|
||||||
|
self.assertEqual(self.engine.actions, [('click', 120, 210, 1)])
|
||||||
|
self.assertFalse(call(self.s, 'click_text', text='不存在')['ok'])
|
||||||
|
self.assertFalse(call(self.s, 'click_text', text='开始', index=3)['ok'])
|
||||||
|
self.assertFalse(call(self.s, 'click_text', text='')['ok'])
|
||||||
|
|
||||||
|
def test_bad_arguments_are_tool_errors(self):
|
||||||
|
r = call(self.s, 'click', x='abc', y=1)
|
||||||
|
self.assertIn('bad arguments', r['error'])
|
||||||
|
self.assertFalse(call(self.s, 'click')['ok'])
|
||||||
|
|
||||||
|
def test_images(self):
|
||||||
|
self.assertEqual(call(self.s, 'find_image', name='battle/turn_assets.png')['x'], 50)
|
||||||
|
r = call(self.s, 'find_image', name='home/x.png')
|
||||||
|
self.assertTrue(r['ok'])
|
||||||
|
self.assertFalse(r['found'])
|
||||||
|
self.assertFalse(call(self.s, 'click_image', name='home/x.png')['ok'])
|
||||||
|
call(self.s, 'click_image', name='battle/turn_assets.png')
|
||||||
|
self.assertEqual(self.engine.actions[-1], ('click', 50, 60, 1))
|
||||||
|
self.assertEqual(call(self.s, 'list_images', prefix='battle/')['count'], 1)
|
||||||
|
|
||||||
|
def test_wait_for_text(self):
|
||||||
|
self.engine.frames = [[], [], [START]]
|
||||||
|
with mock.patch('lc_mcp.protocol.WAIT_INTERVAL', 0.01):
|
||||||
|
r = call(self.s, 'wait_for_text', text='开始', timeout=2)
|
||||||
|
self.assertTrue(r['ok'])
|
||||||
|
self.engine.frames = [[START]]
|
||||||
|
r = call(self.s, 'wait_for_text_gone', text='开始', timeout=0.05)
|
||||||
|
self.assertFalse(r['ok'])
|
||||||
|
self.assertIn('timeout', r['error'])
|
||||||
|
|
||||||
|
def test_task_lifecycle_blocks_manual_input(self):
|
||||||
|
self.assertEqual(call(self.s, 'task_status')['state'], 'idle')
|
||||||
|
r = call(self.s, 'run_task', task='mirror', count=2, exp=9) # exp 不属于 mirror, 忽略
|
||||||
|
self.assertTrue(r['ok'], r)
|
||||||
|
self.assertFalse(call(self.s, 'run_task', task='daily')['ok'])
|
||||||
|
self.assertIn('stop_task', call(self.s, 'click', x=1, y=1)['error'])
|
||||||
|
self.assertTrue(call(self.s, 'get_state')['ok']) # 只读工具仍可用
|
||||||
|
time.sleep(0.5)
|
||||||
|
st = call(self.s, 'task_status')
|
||||||
|
self.assertEqual(st['state'], 'running')
|
||||||
|
self.assertEqual(st['params'], {'count': 2})
|
||||||
|
self.assertIn("child ['mirror', '{\"count\": 2}']", '\n'.join(st['log_tail']))
|
||||||
|
self.assertEqual(call(self.s, 'stop_task')['state'], 'stopped')
|
||||||
|
self.assertEqual(call(self.s, 'task_status')['state'], 'stopped')
|
||||||
|
self.assertTrue(call(self.s, 'click', x=1, y=1)['ok'])
|
||||||
|
self.assertFalse(call(self.s, 'run_task', task='bogus')['ok'])
|
||||||
|
|
||||||
|
def test_task_exit_codes(self):
|
||||||
|
self.tasks.argv_factory = lambda t, p: [sys.executable, '-c', 'import sys; sys.exit(3)']
|
||||||
|
call(self.s, 'run_task', task='reward')
|
||||||
|
self.tasks.proc.wait(5)
|
||||||
|
st = call(self.s, 'task_status')
|
||||||
|
self.assertEqual((st['state'], st['exit_code']), ('failed', 3))
|
||||||
|
|
||||||
|
def test_engine_unavailable(self):
|
||||||
|
s = Server(None, self.tasks, engine_error='no windows')
|
||||||
|
self.assertIn('no windows', call(s, 'get_state')['error'])
|
||||||
|
self.assertEqual(call(s, 'task_status')['state'], 'idle')
|
||||||
|
|
||||||
|
|
||||||
|
class AalcTaskArgsTest(unittest.TestCase):
|
||||||
|
|
||||||
|
def test_rejects_unknown_params(self):
|
||||||
|
from lc_mcp import aalc_task
|
||||||
|
self.assertEqual(aalc_task.main(['x', 'mirror', '{"exp": 1}']), 2)
|
||||||
|
self.assertEqual(aalc_task.main(['x', 'nope']), 2)
|
||||||
|
|
||||||
|
|
||||||
|
def free_port():
|
||||||
|
with socket.socket() as s:
|
||||||
|
s.bind(('127.0.0.1', 0))
|
||||||
|
return s.getsockname()[1]
|
||||||
|
|
||||||
|
|
||||||
|
class TransportTest(unittest.TestCase):
|
||||||
|
|
||||||
|
def setUp(self):
|
||||||
|
self.tmp = tempfile.TemporaryDirectory()
|
||||||
|
self.token_file = os.path.join(self.tmp.name, 'token')
|
||||||
|
with open(self.token_file, 'w') as f:
|
||||||
|
f.write('secret-token')
|
||||||
|
self.port = free_port()
|
||||||
|
mcp = Server(FakeEngine(frames=[[START]]), TaskRunner(self.tmp.name))
|
||||||
|
self.tcp = srv_mod.make_tcp(mcp, '127.0.0.1', self.port, 'secret-token')
|
||||||
|
threading.Thread(target=self.tcp.serve_forever, daemon=True).start()
|
||||||
|
|
||||||
|
def tearDown(self):
|
||||||
|
self.tcp.shutdown()
|
||||||
|
self.tcp.server_close()
|
||||||
|
self.tmp.cleanup()
|
||||||
|
|
||||||
|
def test_wrong_token_rejected(self):
|
||||||
|
with socket.create_connection(('127.0.0.1', self.port)) as s:
|
||||||
|
s.sendall(b'AUTH wrong\n')
|
||||||
|
self.assertEqual(s.makefile('rb').readline().strip(), b'DENIED')
|
||||||
|
|
||||||
|
def test_bridge_end_to_end(self):
|
||||||
|
env = dict(os.environ, LCP_PORT=str(self.port), HOME=self.tmp.name, USERPROFILE=self.tmp.name)
|
||||||
|
os.makedirs(os.path.join(self.tmp.name, 'lc_mcp'))
|
||||||
|
os.replace(self.token_file, os.path.join(self.tmp.name, 'lc_mcp', 'token'))
|
||||||
|
lines = [
|
||||||
|
'{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}',
|
||||||
|
'',
|
||||||
|
'not json',
|
||||||
|
'{"jsonrpc":"2.0","method":"notifications/initialized"}',
|
||||||
|
'{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"click_text","arguments":{"text":"开始"}}}',
|
||||||
|
]
|
||||||
|
p = subprocess.run([sys.executable, os.path.join(REPO, 'lc_mcp', 'bridge.py')],
|
||||||
|
input='\n'.join(lines) + '\n', capture_output=True, text=True,
|
||||||
|
encoding='utf-8', env=env, timeout=20)
|
||||||
|
out = [json.loads(x) for x in p.stdout.splitlines()]
|
||||||
|
self.assertEqual([o.get('id') for o in out], [1, None, 2], p.stdout + p.stderr)
|
||||||
|
self.assertEqual(out[1]['error']['code'], -32700)
|
||||||
|
self.assertFalse(out[2]['result']['isError'])
|
||||||
|
|
||||||
|
def test_bridge_offline(self):
|
||||||
|
env = dict(os.environ, LCP_PORT=str(free_port()), HOME=self.tmp.name, USERPROFILE=self.tmp.name)
|
||||||
|
os.makedirs(os.path.join(self.tmp.name, 'lc_mcp'))
|
||||||
|
os.replace(self.token_file, os.path.join(self.tmp.name, 'lc_mcp', 'token'))
|
||||||
|
req = ('{"jsonrpc":"2.0","id":1,"method":"initialize"}\n'
|
||||||
|
'{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"get_state"}}\n')
|
||||||
|
p = subprocess.run([sys.executable, os.path.join(REPO, 'lc_mcp', 'bridge.py')], input=req,
|
||||||
|
capture_output=True, text=True, encoding='utf-8', env=env, timeout=20)
|
||||||
|
out = [json.loads(x) for x in p.stdout.splitlines()]
|
||||||
|
self.assertIn('offline', out[0]['result']['serverInfo']['name'])
|
||||||
|
self.assertIn('未运行', out[1]['error']['message'])
|
||||||
|
|
||||||
|
def test_stdio_server(self):
|
||||||
|
req = ('{"jsonrpc":"2.0","id":1,"method":"ping"}\n'
|
||||||
|
'{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"get_state"}}\n')
|
||||||
|
env = dict(os.environ, HOME=self.tmp.name, USERPROFILE=self.tmp.name)
|
||||||
|
p = subprocess.run([sys.executable, '-m', 'lc_mcp.server', '--stdio', '--fake'], cwd=REPO, input=req,
|
||||||
|
capture_output=True, text=True, encoding='utf-8', env=env, timeout=20)
|
||||||
|
out = [json.loads(x) for x in p.stdout.splitlines()]
|
||||||
|
self.assertEqual(out[0]['result'], {})
|
||||||
|
state = json.loads(out[1]['result']['content'][0]['text'])
|
||||||
|
self.assertEqual((state['ok'], state['items']), (True, []))
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
unittest.main()
|
||||||
+1
Submodule vendor/aalc added at 7452fad5a4
-87
@@ -1,87 +0,0 @@
|
|||||||
# win_ocr.ps1 - Windows 自带 OCR (Windows.Media.Ocr) 识别本地图片
|
|
||||||
# 用法: powershell -NoProfile -ExecutionPolicy Bypass -File win_ocr.ps1 -Image <png> [-Lang zh-Hans]
|
|
||||||
# 输出: UTF-8 JSON => {"ok":true,"items":[{"text","x","y","w","h"},...]}
|
|
||||||
param(
|
|
||||||
[Parameter(Mandatory=$true)][string]$Image,
|
|
||||||
[string]$Lang = "zh-Hans"
|
|
||||||
)
|
|
||||||
$ErrorActionPreference = "Stop"
|
|
||||||
|
|
||||||
$path = (Resolve-Path $Image -ErrorAction SilentlyContinue)
|
|
||||||
if (-not $path) { Write-Output '{"ok":false,"error":"image not found"}'; exit 1 }
|
|
||||||
$path = $path.Path
|
|
||||||
|
|
||||||
# ---- 正确加载 WinRT 类型 ----
|
|
||||||
try {
|
|
||||||
Add-Type -AssemblyName System.Runtime.WindowsRuntime
|
|
||||||
[void][System.WindowsRuntimeSystemExtensions, System.Runtime.WindowsRuntime, ContentType=WindowsRuntime]
|
|
||||||
} catch {
|
|
||||||
Write-Output '{"ok":false,"error":"winrt load fail"}'; exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
$asTaskGeneric = ([System.WindowsRuntimeSystemExtensions].GetMethods() | Where-Object {
|
|
||||||
$_.Name -eq 'AsTask' -and $_.GetParameters().Count -eq 1 -and
|
|
||||||
$_.GetParameters()[0].ParameterType.Name -eq 'IAsyncOperation`1'
|
|
||||||
})[0]
|
|
||||||
|
|
||||||
Function Await($WinRtTask, $ResultType) {
|
|
||||||
$asTask = $asTaskGeneric.MakeGenericMethod($ResultType)
|
|
||||||
$netTask = $asTask.Invoke($null, @($WinRtTask))
|
|
||||||
$netTask.Wait(-1) | Out-Null
|
|
||||||
$netTask.Result
|
|
||||||
}
|
|
||||||
|
|
||||||
function Write-Json($obj) {
|
|
||||||
[Console]::OutputEncoding = [Text.Encoding]::UTF8
|
|
||||||
Write-Output ($obj | ConvertTo-Json -Compress -Depth 5)
|
|
||||||
}
|
|
||||||
|
|
||||||
try {
|
|
||||||
$file = [Windows.Storage.StorageFile, Windows.Storage, ContentType=WindowsRuntime]::GetFileFromPathAsync($path)
|
|
||||||
$storageFile = Await $file ([Windows.Storage.StorageFile])
|
|
||||||
|
|
||||||
$stream = Await ($storageFile.OpenAsync([Windows.Storage.FileAccessMode]::Read)) ([Windows.Storage.Streams.IRandomAccessStream])
|
|
||||||
|
|
||||||
$decoder = Await ([Windows.Graphics.Imaging.BitmapDecoder, Windows.Graphics.Imaging, ContentType=WindowsRuntime]::CreateAsync($stream)) ([Windows.Graphics.Imaging.BitmapDecoder])
|
|
||||||
$bitmap = Await ($decoder.GetSoftwareBitmapAsync()) ([Windows.Graphics.Imaging.SoftwareBitmap])
|
|
||||||
|
|
||||||
$engine = $null
|
|
||||||
try {
|
|
||||||
$langObj = New-Object Windows.Globalization.Language $Lang
|
|
||||||
$engine = [Windows.Media.Ocr.OcrEngine, Windows.Media.Ocr, ContentType=WindowsRuntime]::TryCreateFromLanguage($langObj)
|
|
||||||
} catch { $engine = $null }
|
|
||||||
if ($engine -eq $null) {
|
|
||||||
$engine = [Windows.Media.Ocr.OcrEngine, Windows.Media.Ocr, ContentType=WindowsRuntime]::TryCreateFromUserProfileLanguages()
|
|
||||||
}
|
|
||||||
if ($engine -eq $null) {
|
|
||||||
Write-Json @{ ok = $false; error = 'no OCR engine (need language pack?)' }
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
$result = Await ($engine.RecognizeAsync($bitmap)) ([Windows.Media.Ocr.OcrResult])
|
|
||||||
|
|
||||||
$items = @()
|
|
||||||
foreach ($line in $result.Lines) {
|
|
||||||
if ($line.Words.Count -eq 0) { continue }
|
|
||||||
$t = ($line.Words | ForEach-Object { $_.Text }) -join " "
|
|
||||||
$minX = [math]::Floor($line.Words[0].BoundingRect.X)
|
|
||||||
$minY = [math]::Floor($line.Words[0].BoundingRect.Y)
|
|
||||||
$maxX = [math]::Ceiling($line.Words[0].BoundingRect.X + $line.Words[0].BoundingRect.Width)
|
|
||||||
$maxY = [math]::Ceiling($line.Words[0].BoundingRect.Y + $line.Words[0].BoundingRect.Height)
|
|
||||||
foreach ($word in $line.Words) {
|
|
||||||
$r = $word.BoundingRect
|
|
||||||
if ($r.X -lt $minX) { $minX = [math]::Floor($r.X) }
|
|
||||||
if ($r.Y -lt $minY) { $minY = [math]::Floor($r.Y) }
|
|
||||||
if (($r.X + $r.Width) -gt $maxX) { $maxX = [math]::Ceiling($r.X + $r.Width) }
|
|
||||||
if (($r.Y + $r.Height) -gt $maxY) { $maxY = [math]::Ceiling($r.Y + $r.Height) }
|
|
||||||
}
|
|
||||||
$items += [PSCustomObject]@{
|
|
||||||
text = $t
|
|
||||||
x = $minX; y = $minY; w = ($maxX - $minX); h = ($maxY - $minY)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
Write-Json @{ ok = $true; items = $items }
|
|
||||||
} catch {
|
|
||||||
Write-Json @{ ok = $false; error = $_.Exception.Message }
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
Reference in New Issue
Block a user