v0.7: 以 AALC 为底座重建 MCP server
- AALC 作为 submodule (vendor/aalc @ 7452fad), 不改其代码 - 低层工具走 AALC auto/ocr/模板匹配; 高层 run_task 跑 AALC 现成流程(子进程, 可强杀) - 常驻服务跑在交互桌面会话 + stdio bridge (token 认证), 取代每次截图注册计划任务 - 删除自写的截图/输入/OCR 后端 - 测试: FakeEngine 覆盖协议、任务生命周期、TCP+bridge 端到端 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -1,6 +1,3 @@
|
||||
__pycache__/
|
||||
*.pyc
|
||||
*.log
|
||||
shot_*.png
|
||||
lc_mcp/job.ps1
|
||||
lc_mcp/run.vbs
|
||||
@@ -0,0 +1,3 @@
|
||||
[submodule "vendor/aalc"]
|
||||
path = vendor/aalc
|
||||
url = https://github.com/KIYI671/AhabAssistantLimbusCompany
|
||||
@@ -1,68 +1,92 @@
|
||||
# lc-mcp — Limbus Company 语义化操控 MCP server (跨平台)
|
||||
# lc-mcp — 基于 AALC 的 Limbus Company MCP server
|
||||
|
||||
给 AI agent (codex / deepseek / 任意 MCP 客户端) 提供**语义化**游戏操控,
|
||||
解决"盲看屏幕猜坐标→思考慢、点错率高"。
|
||||
把 [AhabAssistantLimbusCompany (AALC)](https://github.com/KIYI671/AhabAssistantLimbusCompany)
|
||||
的识别与操控能力包装成 MCP 工具,给 AI agent(codex / 任意 MCP 客户端)用。
|
||||
|
||||
## 核心设计
|
||||
- **低层**:截图、RapidOCR、AALC 模板图匹配(几百张按钮/图标模板)、后台点击/按键,全部直接调 AALC 的 `module.automation`
|
||||
- **高层**:`run_task` 直接跑 AALC 现成的完整流程(日常经验本/纽本、领奖励、狂气换体力、镜牢),自带战斗处理、掉线重试、游戏重启
|
||||
|
||||
AI 不用猜坐标。工具层做"文本化 + 语义点击":
|
||||
AALC 以 git submodule 引入(`vendor/aalc`,固定到某个 commit),不改它的代码,升级时只需更新 submodule 指针。
|
||||
|
||||
## 架构
|
||||
|
||||
```
|
||||
get_state() -> 截图+远端OCR:全屏文字元素 + 坐标
|
||||
click_text("开始") -> 内部 OCR 定位"开始"并点击中心
|
||||
wait_for_text("结算") -> 轮询直到文字出现/消失
|
||||
MCP 客户端 ──ssh──> python lc_mcp/bridge.py (stdio, 只用标准库)
|
||||
│ 127.0.0.1:8765, 首行 token 认证
|
||||
▼
|
||||
python -m lc_mcp.server ← 登录时由计划任务在「交互桌面会话」启动
|
||||
├─ AalcEngine → AALC auto / ocr / screen (找窗口、后台截图、PostMessage 输入)
|
||||
└─ TaskRunner → 子进程 python -m lc_mcp.aalc_task <task> (停止 = 杀进程树)
|
||||
```
|
||||
|
||||
任务循环: `get_state -> 决策 -> click_text -> wait_for_text -> 确认 -> 下一步`
|
||||
为什么要常驻服务 + bridge:SSH 登录的进程跑在 session 0,看不到游戏窗口,也发不了输入。
|
||||
旧版靠"每次截图注册一个计划任务"绕过,慢且不能输入;现在整个服务常驻在桌面会话里。
|
||||
|
||||
## 平台矩阵
|
||||
## 部署(Windows,跑游戏的那台机器)
|
||||
|
||||
| 平台 | 截图 | 输入后端 | 备注 |
|
||||
|------|------|----------|------|
|
||||
| win | 交互计划任务 | `sendinput`(默认) / `interception`(物理HID级) | interception 需装驱动+重启 |
|
||||
| linux | X11 PIL / scrot | `xdotool` / `uinput`(内核虚拟设备,物理级) | uinput 需 root 或 uinput 组 |
|
||||
| mac | screencapture | `cg`(CGEvent, PyObjC) | 需辅助功能权限 |
|
||||
```powershell
|
||||
git clone --recursive https://git.lingll.icu/yws/lc-mcp
|
||||
cd lc-mcp\vendor\aalc
|
||||
uv sync # 按 AALC 的 pyproject 装依赖 (Python >= 3.12),生成 .venv
|
||||
# 先用 AALC 自己的界面 (uv run main.py) 配好队伍、镜牢设置等,lc-mcp 读同一个 config.yaml
|
||||
cd ..\..
|
||||
powershell -ExecutionPolicy Bypass -File scripts\install_logon_task.ps1
|
||||
```
|
||||
|
||||
`LCP_PLATFORM=win|linux|mac` 强制指定,默认按系统自动检测。
|
||||
`install_logon_task.ps1` 会注册 `LcMCPServer` 计划任务(登录时启动、普通权限、失败自动重启)。
|
||||
游戏以管理员身份运行时才需要加 `-Elevated`。卸载:`-Uninstall`。
|
||||
|
||||
OCR 全本地:`ocr_local.py` 调 Windows 自带 OCR(Windows.Media.Ocr,需 `pip install --user winsdk`)。
|
||||
无任何云端/远端服务依赖——MCP 是纯本地"外骨骼"。
|
||||
|
||||
## 部署
|
||||
|
||||
1. 拷贝 `lc-mcp-dev/` 到目标机器
|
||||
2. 装依赖(只用标准库;linux xdotool 后端需 `apt install xdotool`;mac cg 需 `pip install pyobjc-framework-Quartz`)
|
||||
3. 启动:`python lc_mcp.py`(stdio),MCP 客户端 spawn
|
||||
## MCP 客户端配置
|
||||
|
||||
```jsonc
|
||||
// codex MCP 配置 (codex/mcp.json 或项目 .mcp.json)
|
||||
{
|
||||
"mcpServers": {
|
||||
"limbus": {
|
||||
"type": "stdio",
|
||||
"command": "ssh",
|
||||
"args": ["win", "python", "C:/Users/yws-hhh/lc-mcp-dev/lc_mcp.py"],
|
||||
"env": { "INPUT_BACKEND": "interception" }
|
||||
"args": ["win", "python", "C:/Users/<you>/lc-mcp/lc_mcp/bridge.py"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
bridge 读取 `~/lc_mcp/token` 认证(服务首次启动时生成),所以 SSH 用户必须和登录桌面的用户是同一个账号。
|
||||
服务没起来时 bridge 仍会完成握手,并对每个调用返回“服务未运行”的原因。
|
||||
|
||||
本机直接用(客户端本身就在桌面会话里):`python -m lc_mcp.server --stdio`。
|
||||
|
||||
## 工具
|
||||
|
||||
| 工具 | 说明 |
|
||||
|------|------|
|
||||
| `get_state` | 截图 + OCR,返回窗口内全部文字 `{text,x,y,w,h,score}` |
|
||||
| `click_text(text, index)` | 按文字点击;忽略空格/大小写,完全匹配优先 |
|
||||
| `wait_for_text` / `wait_for_text_gone` | 轮询 OCR 等文字出现/消失(上限 300s) |
|
||||
| `find_image` / `click_image(name, threshold, model)` | AALC 模板图匹配,适合没有文字的按钮 |
|
||||
| `list_images(prefix)` | 列出模板图,如 `battle/`、`mirror/` |
|
||||
| `screenshot` | 窗口截图(`~/lc_mcp/shots`,只保留最近 30 张) |
|
||||
| `click(x,y,times)` / `keypress` / `type_text` | 低层输入(窗口客户区坐标) |
|
||||
| `run_task(task, …)` | `all` / `daily(exp,thread)` / `reward` / `enkephalin(times)` / `mirror(count)` |
|
||||
| `task_status(lines)` / `stop_task` | 任务状态、日志尾部(`~/lc_mcp/runs`)、强制停止 |
|
||||
|
||||
任务运行期间,点击/按键类工具会被拒绝(防止打乱 AALC 的流程),只读工具仍可用。
|
||||
`run_task` 传入的次数只对本次运行生效,结束后恢复 `config.yaml` 原值。
|
||||
|
||||
## 环境变量
|
||||
|
||||
| 变量 | 默认 | 说明 |
|
||||
|------|------|------|
|
||||
| `LCP_PLATFORM` | 自动 | win/linux/mac |
|
||||
| `INPUT_BACKEND` | 平台默认 | mock/sendinput/interception/xdotool/uinput/cg |
|
||||
| `OCR_BACKEND` | local | 本地 OCR(仅 local;Win 需 winsdk) |
|
||||
| `OCR_MIN_SCORE` | 0.5 | 过滤低置信度 OCR 结果 |
|
||||
| `AALC_DIR` | `vendor/aalc` | 使用其他位置的 AALC 源码 |
|
||||
| `LCP_PORT` | 8765 | 服务/bridge 端口 |
|
||||
| `LCP_KEEP_SHOTS` | 30 | 截图保留张数 |
|
||||
|
||||
## 工具
|
||||
## 开发 / 测试
|
||||
|
||||
get_state · click_text · wait_for_text · wait_for_text_gone · screenshot · click · move · keypress · type_text
|
||||
```bash
|
||||
python -m unittest discover -s tests # 不需要 Windows/AALC (FakeEngine)
|
||||
python -m lc_mcp.server --stdio --fake # 假引擎手动调试协议
|
||||
```
|
||||
|
||||
## 安全提醒
|
||||
## 许可证
|
||||
|
||||
- 输入会真实操作你已登录的桌面,跑任务时别动鼠标
|
||||
- win 的截图走交互计划任务(Registry + wscript),杀软可能告警
|
||||
- interception/uinput 是"物理化"输入,用于降低虚拟输入痕迹;真·硬件级可外接 Arduino/RPi Pico HID
|
||||
AALC 使用 AGPL-3.0。本项目会导入并运行 AALC 的代码,分发时需遵守 AGPL-3.0。
|
||||
|
||||
@@ -1,35 +0,0 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""输入后端接口。每种后端实现同一套动作:click/move/keypress/type_text。"""
|
||||
from abc import ABC, abstractmethod
|
||||
|
||||
|
||||
class InputBackend(ABC):
|
||||
|
||||
@abstractmethod
|
||||
def click(self, x, y, button='left', clicks=1):
|
||||
...
|
||||
|
||||
@abstractmethod
|
||||
def move(self, x, y):
|
||||
...
|
||||
|
||||
@abstractmethod
|
||||
def keypress(self, key):
|
||||
...
|
||||
|
||||
@abstractmethod
|
||||
def type_text(self, text):
|
||||
...
|
||||
|
||||
|
||||
def backend_factory(name, **kwargs):
|
||||
if name == 'mock':
|
||||
from .mock import MockBackend
|
||||
return MockBackend(**kwargs)
|
||||
if name == 'sendinput':
|
||||
from .sendinput_win import SendInputBackend
|
||||
return SendInputBackend(**kwargs)
|
||||
if name == 'interception':
|
||||
from .interception_win import InterceptionBackend
|
||||
return InterceptionBackend(**kwargs)
|
||||
raise ValueError('unknown backend: %s' % name)
|
||||
@@ -1,32 +0,0 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""后端共用小工具。"""
|
||||
import ctypes
|
||||
|
||||
KEY_ALIASES = {'return': 'enter', 'escape': 'esc', 'del': 'delete', 'ins': 'insert',
|
||||
'pgup': 'pageup', 'pgdn': 'pagedown', 'control': 'ctrl', 'win': 'lwin',
|
||||
'arrowup': 'up', 'arrowdown': 'down', 'arrowleft': 'left', 'arrowright': 'right'}
|
||||
|
||||
|
||||
def parse_combo(key):
|
||||
"""'Alt+F4' -> ['alt', 'f4']; 单个 '+' 视为加号键。"""
|
||||
k = str(key).strip().lower()
|
||||
if k == '+' or '+' not in k:
|
||||
parts = [k]
|
||||
else:
|
||||
parts = [p.strip() for p in k.split('+')]
|
||||
if any(not p for p in parts):
|
||||
raise ValueError('bad key combo %r' % key)
|
||||
return [KEY_ALIASES.get(p, p) for p in parts]
|
||||
|
||||
|
||||
def win_dpi_aware(user32):
|
||||
"""声明 per-monitor DPI 感知, 让 SetCursorPos/GetSystemMetrics 用物理像素, 和截图一致。"""
|
||||
try:
|
||||
ctypes.windll.shcore.SetProcessDpiAwareness(2)
|
||||
return
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
user32.SetProcessDPIAware()
|
||||
except Exception:
|
||||
pass
|
||||
@@ -1,64 +0,0 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""macOS 输入后端:Quartz CGEvent (需 PyObjC: pip install pyobjc-framework-Quartz)。"""
|
||||
import time
|
||||
|
||||
|
||||
class CGBackend:
|
||||
|
||||
def __init__(self):
|
||||
try:
|
||||
import Quartz
|
||||
self.q = Quartz
|
||||
except ImportError:
|
||||
raise RuntimeError('PyObjC Quartz not installed (pip install pyobjc-framework-Quartz)')
|
||||
|
||||
def _post(self, event):
|
||||
self.q.CGEventPost(self.q.kCGHIDEventTap, event)
|
||||
time.sleep(0.02)
|
||||
|
||||
def move(self, x, y):
|
||||
e = self.q.CGEventCreateMouseEvent(None, self.q.kCGEventMouseMoved, (float(x), float(y)), self.q.kCGMouseButtonLeft)
|
||||
self._post(e)
|
||||
return {'ok': True}
|
||||
|
||||
def click(self, x, y, button='left', times=1):
|
||||
btn = self.q.kCGMouseButtonLeft if button == 'left' else self.q.kCGMouseButtonRight
|
||||
self.move(x, y)
|
||||
for _ in range(times):
|
||||
down = self.q.CGEventCreateMouseEvent(None, self.q.kCGEventLeftMouseDown if button == 'left' else self.q.kCGEventRightMouseDown, (float(x), float(y)), btn)
|
||||
up = self.q.CGEventCreateMouseEvent(None, self.q.kCGEventLeftMouseUp if button == 'left' else self.q.kCGEventRightMouseUp, (float(x), float(y)), btn)
|
||||
self._post(down)
|
||||
self._post(up)
|
||||
return {'ok': True}
|
||||
|
||||
def keypress(self, key):
|
||||
mapping = {'enter': 36, 'esc': 53, 'space': 49, 'tab': 48, 'f4': 96,
|
||||
'ctrl': 59, 'shift': 56, 'alt': 58, 'backspace': 51}
|
||||
k = key.lower()
|
||||
if k in mapping:
|
||||
self._key(mapping[k])
|
||||
return {'ok': True}
|
||||
if len(k) == 1 and 'a' <= k <= 'z':
|
||||
# mac 虚拟键码: a=0, b=11, ... 非连续, 用简单表
|
||||
table = {'a': 0, 'b': 11, 'c': 8, 'd': 2, 'e': 14, 'f': 3, 'g': 5, 'h': 4,
|
||||
'i': 34, 'j': 38, 'k': 40, 'l': 37, 'm': 46, 'n': 45, 'o': 31,
|
||||
'p': 35, 'q': 12, 'r': 15, 's': 1, 't': 17, 'u': 32, 'v': 9,
|
||||
'w': 13, 'x': 7, 'y': 16, 'z': 6}
|
||||
if k in table:
|
||||
self._key(table[k])
|
||||
return {'ok': True}
|
||||
return {'ok': False, 'error': 'unsupported key %s' % key}
|
||||
|
||||
def _key(self, vk):
|
||||
down = self.q.CGEventCreateKeyboardEvent(None, vk, True)
|
||||
up = self.q.CGEventCreateKeyboardEvent(None, vk, False)
|
||||
self._post(down)
|
||||
self._post(up)
|
||||
|
||||
def type_text(self, text):
|
||||
for ch in text:
|
||||
r = self.keypress(ch)
|
||||
if not r.get('ok'):
|
||||
return r
|
||||
time.sleep(0.03)
|
||||
return {'ok': True}
|
||||
@@ -1,239 +0,0 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Interception 物理输入后端(Windows + Interception 驱动)。
|
||||
|
||||
输入在驱动层转为真实 HID 事件(对应用透明,非 SendInput 虚拟输入)。
|
||||
需要: install-interception.exe /install 后重启系统,且本 dll 与驱动共存。
|
||||
|
||||
API 以 oblitum/Interception interception.h 为准:
|
||||
设备号 1..10 = 键盘, 11..20 = 鼠标; 没有 open_device 之类的函数。
|
||||
interception_send(ctx, device, stroke*, nstroke) 第 4 个参数是 stroke 个数, 不是字节数。
|
||||
"""
|
||||
import ctypes
|
||||
import time
|
||||
|
||||
from ._util import parse_combo, win_dpi_aware
|
||||
|
||||
# ---- Interception 常量 ----
|
||||
MOUSE_LEFTBUTTON_DOWN = 0x001
|
||||
MOUSE_LEFTBUTTON_UP = 0x002
|
||||
MOUSE_RIGHTBUTTON_DOWN = 0x004
|
||||
MOUSE_RIGHTBUTTON_UP = 0x008
|
||||
MOUSE_MIDDLEBUTTON_DOWN = 0x010
|
||||
MOUSE_MIDDLEBUTTON_UP = 0x020
|
||||
|
||||
MOUSE_MOVE_RELATIVE = 0x000
|
||||
MOUSE_MOVE_ABSOLUTE = 0x001
|
||||
MOUSE_VIRTUAL_DESKTOP = 0x002
|
||||
|
||||
KEY_DOWN = 0x00
|
||||
KEY_UP = 0x01
|
||||
KEY_E0 = 0x02
|
||||
|
||||
MAX_KEYBOARD = 10
|
||||
MAX_MOUSE = 10
|
||||
KEYBOARD_DEVS = range(1, MAX_KEYBOARD + 1)
|
||||
MOUSE_DEVS = range(MAX_KEYBOARD + 1, MAX_KEYBOARD + MAX_MOUSE + 1)
|
||||
|
||||
# ---- Set 1 扫描码 ----
|
||||
SCAN = {
|
||||
'a': 0x1E, 'b': 0x30, 'c': 0x2E, 'd': 0x20, 'e': 0x12, 'f': 0x21,
|
||||
'g': 0x22, 'h': 0x23, 'i': 0x17, 'j': 0x24, 'k': 0x25, 'l': 0x26,
|
||||
'm': 0x32, 'n': 0x31, 'o': 0x18, 'p': 0x19, 'q': 0x10, 'r': 0x13,
|
||||
's': 0x1F, 't': 0x14, 'u': 0x16, 'v': 0x2F, 'w': 0x11, 'x': 0x2D,
|
||||
'y': 0x15, 'z': 0x2C,
|
||||
'1': 0x02, '2': 0x03, '3': 0x04, '4': 0x05, '5': 0x06,
|
||||
'6': 0x07, '7': 0x08, '8': 0x09, '9': 0x0A, '0': 0x0B,
|
||||
'enter': 0x1C, 'esc': 0x01, 'space': 0x39, 'tab': 0x0F, 'backspace': 0x0E,
|
||||
'ctrl': 0x1D, 'alt': 0x38, 'lshift': 0x2A, 'rshift': 0x36, 'shift': 0x2A,
|
||||
'f1': 0x3B, 'f2': 0x3C, 'f3': 0x3D, 'f4': 0x3E, 'f5': 0x3F, 'f6': 0x40,
|
||||
'f7': 0x41, 'f8': 0x42, 'f9': 0x43, 'f10': 0x44, 'f11': 0x57, 'f12': 0x58,
|
||||
',': 0x33, '.': 0x34, '/': 0x35, ';': 0x27, "'": 0x28, '[': 0x1A,
|
||||
']': 0x1B, '-': 0x0C, '=': 0x0D, '`': 0x29, '\\': 0x2B,
|
||||
}
|
||||
# 扩展键: 扫描码同小键盘, 需带 E0 前缀
|
||||
SCAN_E0 = {
|
||||
'up': 0x48, 'down': 0x50, 'left': 0x4B, 'right': 0x4D,
|
||||
'home': 0x47, 'end': 0x4F, 'pageup': 0x49, 'pagedown': 0x51,
|
||||
'insert': 0x52, 'delete': 0x53, 'rctrl': 0x1D, 'ralt': 0x38, 'lwin': 0x5B,
|
||||
}
|
||||
UPPER_SCAN = { # 需按住 Shift 的符号
|
||||
'!': 0x02, '@': 0x03, '#': 0x04, '$': 0x05, '%': 0x06, '^': 0x07,
|
||||
'&': 0x08, '*': 0x09, '(': 0x0A, ')': 0x0B, '_': 0x0C, '+': 0x0D,
|
||||
'{': 0x1A, '}': 0x1B, ':': 0x27, '"': 0x28, '<': 0x33, '>': 0x34,
|
||||
'?': 0x35, '~': 0x29, '|': 0x2B,
|
||||
}
|
||||
|
||||
|
||||
class MouseStroke(ctypes.Structure):
|
||||
# interception.h: unsigned short state, flags; short rolling; int x, y; unsigned int information
|
||||
_fields_ = [("state", ctypes.c_ushort), ("flags", ctypes.c_ushort),
|
||||
("rolling", ctypes.c_short), ("x", ctypes.c_int),
|
||||
("y", ctypes.c_int), ("information", ctypes.c_uint)]
|
||||
|
||||
|
||||
class KeyStroke(ctypes.Structure):
|
||||
_fields_ = [("code", ctypes.c_ushort), ("state", ctypes.c_ushort),
|
||||
("information", ctypes.c_uint)]
|
||||
|
||||
|
||||
def key_strokes(key):
|
||||
"""键名 -> (scancode, e0) ; 未知返回 None。"""
|
||||
if key in SCAN:
|
||||
return SCAN[key], False
|
||||
if key in SCAN_E0:
|
||||
return SCAN_E0[key], True
|
||||
return None
|
||||
|
||||
|
||||
def to_absolute(v, size):
|
||||
"""像素 -> Interception 绝对坐标 0..65535 (主屏)。"""
|
||||
if size <= 1:
|
||||
return 0
|
||||
v = min(max(int(v), 0), size - 1)
|
||||
return int(round(v * 65535.0 / (size - 1)))
|
||||
|
||||
|
||||
class InterceptionBackend:
|
||||
|
||||
def __init__(self, dll_path=None, keyboard=None, mouse=None):
|
||||
if dll_path is None:
|
||||
dll_path = r'C:\Program Files\Interception\interception.dll'
|
||||
try:
|
||||
self.dll = ctypes.CDLL(dll_path)
|
||||
except OSError:
|
||||
# 兜底:尝试系统 PATH / 常见目录
|
||||
self.dll = ctypes.CDLL('interception.dll')
|
||||
self._api()
|
||||
self._user32 = ctypes.WinDLL('user32', use_last_error=True)
|
||||
win_dpi_aware(self._user32)
|
||||
self.ctx = self.dll.interception_create_context()
|
||||
if not self.ctx:
|
||||
raise RuntimeError('interception_create_context failed (驱动未安装/未重启?)')
|
||||
# 只往一个设备发: 广播到所有键盘会让同一个键被按多次
|
||||
self.kbd = keyboard or self._first_present(KEYBOARD_DEVS)
|
||||
self.mou = mouse or self._first_present(MOUSE_DEVS)
|
||||
self._scr_w = self._user32.GetSystemMetrics(0)
|
||||
self._scr_h = self._user32.GetSystemMetrics(1)
|
||||
|
||||
def __del__(self):
|
||||
try:
|
||||
if getattr(self, 'ctx', None):
|
||||
self.dll.interception_destroy_context(self.ctx)
|
||||
self.ctx = None
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def _api(self):
|
||||
d = self.dll
|
||||
d.interception_create_context.restype = ctypes.c_void_p
|
||||
d.interception_create_context.argtypes = []
|
||||
d.interception_destroy_context.restype = None
|
||||
d.interception_destroy_context.argtypes = [ctypes.c_void_p]
|
||||
d.interception_get_hardware_id.restype = ctypes.c_uint
|
||||
d.interception_get_hardware_id.argtypes = [ctypes.c_void_p, ctypes.c_int,
|
||||
ctypes.c_void_p, ctypes.c_uint]
|
||||
d.interception_send.restype = ctypes.c_int
|
||||
d.interception_send.argtypes = [ctypes.c_void_p, ctypes.c_int,
|
||||
ctypes.c_void_p, ctypes.c_uint]
|
||||
|
||||
def hardware_id(self, dev):
|
||||
buf = ctypes.create_string_buffer(512)
|
||||
n = self.dll.interception_get_hardware_id(self.ctx, dev, buf, ctypes.sizeof(buf))
|
||||
if not n or n > ctypes.sizeof(buf):
|
||||
return ''
|
||||
# 驱动返回 UTF-16 (REG_MULTI_SZ), 取第一个
|
||||
return buf.raw[:n].decode('utf-16-le', 'replace').split('\x00')[0]
|
||||
|
||||
def _first_present(self, devs):
|
||||
for dev in devs:
|
||||
if self.hardware_id(dev):
|
||||
return dev
|
||||
return devs[0]
|
||||
|
||||
def set_screen(self, w, h):
|
||||
"""以截图尺寸为准(会话 0 / DPI 虚拟化下 GetSystemMetrics 不可信)。"""
|
||||
self._scr_w, self._scr_h = int(w), int(h)
|
||||
|
||||
# ---- 底层发送 ----
|
||||
def _send(self, dev, stroke):
|
||||
n = self.dll.interception_send(self.ctx, dev, ctypes.byref(stroke), 1)
|
||||
if n != 1:
|
||||
raise RuntimeError('interception_send(dev=%d) returned %d' % (dev, n))
|
||||
|
||||
def _kbd_stroke(self, code, state, e0=False):
|
||||
ks = KeyStroke(code=code, state=state | (KEY_E0 if e0 else 0))
|
||||
self._send(self.kbd, ks)
|
||||
|
||||
def _mouse_stroke(self, state=0, flags=0, x=0, y=0):
|
||||
self._send(self.mou, MouseStroke(state=state, flags=flags, x=int(x), y=int(y)))
|
||||
|
||||
def _tap(self, code, e0=False, hold=0.04):
|
||||
self._kbd_stroke(code, KEY_DOWN, e0)
|
||||
time.sleep(hold)
|
||||
self._kbd_stroke(code, KEY_UP, e0)
|
||||
|
||||
# ---- 公开动作 ----
|
||||
def move(self, x, y):
|
||||
nx = to_absolute(x, self._scr_w)
|
||||
ny = to_absolute(y, self._scr_h)
|
||||
self._mouse_stroke(state=0, flags=MOUSE_MOVE_ABSOLUTE, x=nx, y=ny)
|
||||
time.sleep(0.03)
|
||||
return {'ok': True, 'backend': 'interception'}
|
||||
|
||||
def click(self, x, y, button='left', clicks=1):
|
||||
self.move(x, y)
|
||||
time.sleep(0.05)
|
||||
if button == 'left':
|
||||
down, up = MOUSE_LEFTBUTTON_DOWN, MOUSE_LEFTBUTTON_UP
|
||||
else:
|
||||
down, up = MOUSE_RIGHTBUTTON_DOWN, MOUSE_RIGHTBUTTON_UP
|
||||
for _ in range(max(1, int(clicks))):
|
||||
self._mouse_stroke(state=down)
|
||||
time.sleep(0.06)
|
||||
self._mouse_stroke(state=up)
|
||||
time.sleep(0.06)
|
||||
return {'ok': True, 'clicked': [int(x), int(y)], 'backend': 'interception'}
|
||||
|
||||
def keypress(self, key):
|
||||
try:
|
||||
parts = parse_combo(key)
|
||||
except ValueError as e:
|
||||
return {'ok': False, 'error': str(e)}
|
||||
strokes = [key_strokes(p) for p in parts]
|
||||
if None in strokes:
|
||||
return {'ok': False, 'error': 'unknown key %s' % key}
|
||||
for code, e0 in strokes[:-1]: # 修饰键按下
|
||||
self._kbd_stroke(code, KEY_DOWN, e0)
|
||||
time.sleep(0.03)
|
||||
self._tap(*strokes[-1])
|
||||
for code, e0 in reversed(strokes[:-1]):
|
||||
time.sleep(0.03)
|
||||
self._kbd_stroke(code, KEY_UP, e0)
|
||||
return {'ok': True, 'key': str(key), 'backend': 'interception'}
|
||||
|
||||
def type_text(self, text):
|
||||
text = str(text)
|
||||
plan = []
|
||||
for ch in text:
|
||||
low = ch.lower()
|
||||
if ch == ' ':
|
||||
plan.append((SCAN['space'], False))
|
||||
elif ch == '\n':
|
||||
plan.append((SCAN['enter'], False))
|
||||
elif low in SCAN and len(low) == 1:
|
||||
plan.append((SCAN[low], ch.isupper()))
|
||||
elif ch in UPPER_SCAN:
|
||||
plan.append((UPPER_SCAN[ch], True))
|
||||
else:
|
||||
# 扫描码只能打 US 布局 ASCII; 中文请用 sendinput 后端(KEYEVENTF_UNICODE)
|
||||
return {'ok': False, 'error': 'untypable char %r (interception 只支持 ASCII)' % ch}
|
||||
for code, shift in plan:
|
||||
if shift:
|
||||
self._kbd_stroke(SCAN['shift'], KEY_DOWN)
|
||||
time.sleep(0.02)
|
||||
self._tap(code, hold=0.03)
|
||||
if shift:
|
||||
time.sleep(0.02)
|
||||
self._kbd_stroke(SCAN['shift'], KEY_UP)
|
||||
time.sleep(0.03)
|
||||
return {'ok': True, 'typed': text, 'backend': 'interception'}
|
||||
@@ -1,31 +0,0 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Mock 后端:记录动作,无真实输入。开发/测试用。"""
|
||||
import time
|
||||
|
||||
|
||||
class MockBackend:
|
||||
|
||||
def __init__(self, log=None):
|
||||
self.log = log if log is not None else []
|
||||
self._screen = (1920, 1080)
|
||||
|
||||
def click(self, x, y, button='left', clicks=1):
|
||||
self.log.append({'act': 'click', 'x': int(x), 'y': int(y),
|
||||
'button': button, 'clicks': int(clicks)})
|
||||
time.sleep(0.02)
|
||||
return {'ok': True, 'clicked': [int(x), int(y)], 'backend': 'mock'}
|
||||
|
||||
def move(self, x, y):
|
||||
self.log.append({'act': 'move', 'x': int(x), 'y': int(y)})
|
||||
time.sleep(0.01)
|
||||
return {'ok': True, 'backend': 'mock'}
|
||||
|
||||
def keypress(self, key):
|
||||
self.log.append({'act': 'keypress', 'key': str(key)})
|
||||
time.sleep(0.01)
|
||||
return {'ok': True, 'key': str(key), 'backend': 'mock'}
|
||||
|
||||
def type_text(self, text):
|
||||
self.log.append({'act': 'type_text', 'text': str(text)})
|
||||
time.sleep(0.01 * len(text))
|
||||
return {'ok': True, 'typed': str(text), 'backend': 'mock'}
|
||||
@@ -1,147 +0,0 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""SendInput 后端(Windows,ctypes user32)。
|
||||
当前线上 lc_mcp.py 用的机制;作为 interception 不可用时的兜底。
|
||||
注意: SendInput 只能注入到调用进程所在的交互桌面; 从 SSH(session 0)里跑不会作用到游戏窗口。
|
||||
"""
|
||||
import ctypes
|
||||
import time
|
||||
|
||||
from ctypes import wintypes
|
||||
|
||||
from ._util import parse_combo, win_dpi_aware
|
||||
|
||||
INPUT_MOUSE = 0
|
||||
INPUT_KEYBOARD = 1
|
||||
KEYEVENTF_EXTENDEDKEY = 0x0001
|
||||
KEYEVENTF_KEYUP = 0x0002
|
||||
KEYEVENTF_UNICODE = 0x0004
|
||||
MOUSEEVENTF_MOVE = 0x0001
|
||||
MOUSEEVENTF_LEFTDOWN = 0x0002
|
||||
MOUSEEVENTF_LEFTUP = 0x0004
|
||||
MOUSEEVENTF_RIGHTDOWN = 0x0008
|
||||
MOUSEEVENTF_RIGHTUP = 0x0010
|
||||
MAPVK_VK_TO_VSC = 0
|
||||
|
||||
|
||||
class MOUSEINPUT(ctypes.Structure):
|
||||
_fields_ = [("dx", wintypes.LONG), ("dy", wintypes.LONG),
|
||||
("mouseData", wintypes.DWORD), ("dwFlags", wintypes.DWORD),
|
||||
("time", wintypes.DWORD), ("dwExtraInfo", ctypes.POINTER(ctypes.c_ulong))]
|
||||
|
||||
|
||||
class KEYBDINPUT(ctypes.Structure):
|
||||
_fields_ = [("wVk", wintypes.WORD), ("wScan", wintypes.WORD),
|
||||
("dwFlags", wintypes.DWORD), ("time", wintypes.DWORD),
|
||||
("dwExtraInfo", ctypes.POINTER(ctypes.c_ulong))]
|
||||
|
||||
|
||||
class INPUT(ctypes.Structure):
|
||||
class _I(ctypes.Union):
|
||||
_fields_ = [("mi", MOUSEINPUT), ("ki", KEYBDINPUT)]
|
||||
_anonymous_ = ("i",)
|
||||
_fields_ = [("type", wintypes.DWORD), ("i", _I)]
|
||||
|
||||
|
||||
KEYMAP = {'enter': 0x0D, 'esc': 0x1B, 'space': 0x20, 'tab': 0x09, 'backspace': 0x08,
|
||||
'ctrl': 0x11, 'alt': 0x12, 'shift': 0x10, 'lshift': 0xA0, 'rshift': 0xA1,
|
||||
'rctrl': 0xA3, 'ralt': 0xA5, 'lwin': 0x5B,
|
||||
'up': 0x26, 'down': 0x28, 'left': 0x25, 'right': 0x27,
|
||||
'home': 0x24, 'end': 0x23, 'pageup': 0x21, 'pagedown': 0x22,
|
||||
'insert': 0x2D, 'delete': 0x2E}
|
||||
KEYMAP.update({'f%d' % i: 0x6F + i for i in range(1, 13)})
|
||||
EXTENDED = {0x26, 0x28, 0x25, 0x27, 0x24, 0x23, 0x21, 0x22, 0x2D, 0x2E, 0xA3, 0xA5, 0x5B}
|
||||
|
||||
|
||||
def utf16_units(text):
|
||||
"""str -> UTF-16 码元列表(BMP 外字符拆成代理对)。"""
|
||||
data = str(text).encode('utf-16-le')
|
||||
return [int.from_bytes(data[i:i + 2], 'little') for i in range(0, len(data), 2)]
|
||||
|
||||
|
||||
class SendInputBackend:
|
||||
|
||||
def __init__(self):
|
||||
self._user32 = ctypes.WinDLL('user32', use_last_error=True)
|
||||
win_dpi_aware(self._user32)
|
||||
self._user32.VkKeyScanW.restype = ctypes.c_short
|
||||
self._user32.VkKeyScanW.argtypes = [wintypes.WCHAR]
|
||||
self._user32.MapVirtualKeyW.restype = wintypes.UINT
|
||||
self._user32.MapVirtualKeyW.argtypes = [wintypes.UINT, wintypes.UINT]
|
||||
|
||||
def _send_input(self, *inputs):
|
||||
n = len(inputs)
|
||||
arr = (INPUT * n)(*inputs)
|
||||
sent = self._user32.SendInput(n, arr, ctypes.sizeof(INPUT))
|
||||
if sent != n:
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
return sent
|
||||
|
||||
def _mouse(self, flags):
|
||||
m = INPUT(); m.type = INPUT_MOUSE; m.mi.dwFlags = flags
|
||||
return m
|
||||
|
||||
def click(self, x, y, button='left', clicks=1):
|
||||
self._user32.SetCursorPos(int(x), int(y))
|
||||
# 补一个 0 位移的 MOVE, 让只看原始输入/WM_MOUSEMOVE 的游戏刷新 hover
|
||||
self._send_input(self._mouse(MOUSEEVENTF_MOVE))
|
||||
time.sleep(0.03)
|
||||
down_flag = MOUSEEVENTF_LEFTDOWN if button == 'left' else MOUSEEVENTF_RIGHTDOWN
|
||||
up_flag = MOUSEEVENTF_LEFTUP if button == 'left' else MOUSEEVENTF_RIGHTUP
|
||||
for _ in range(max(1, int(clicks))):
|
||||
self._send_input(self._mouse(down_flag))
|
||||
time.sleep(0.05)
|
||||
self._send_input(self._mouse(up_flag))
|
||||
time.sleep(0.05)
|
||||
return {'ok': True, 'clicked': [int(x), int(y)], 'backend': 'sendinput'}
|
||||
|
||||
def move(self, x, y):
|
||||
self._user32.SetCursorPos(int(x), int(y))
|
||||
self._send_input(self._mouse(MOUSEEVENTF_MOVE))
|
||||
time.sleep(0.01)
|
||||
return {'ok': True, 'backend': 'sendinput'}
|
||||
|
||||
def _vk(self, name):
|
||||
if name in KEYMAP:
|
||||
return KEYMAP[name]
|
||||
if len(name) == 1:
|
||||
r = self._user32.VkKeyScanW(name)
|
||||
if r != -1:
|
||||
return r & 0xFF
|
||||
return None
|
||||
|
||||
def _key(self, vk, up):
|
||||
ki = INPUT(); ki.type = INPUT_KEYBOARD
|
||||
ki.ki.wVk = vk
|
||||
ki.ki.wScan = self._user32.MapVirtualKeyW(vk, MAPVK_VK_TO_VSC)
|
||||
ki.ki.dwFlags = (KEYEVENTF_KEYUP if up else 0) | (KEYEVENTF_EXTENDEDKEY if vk in EXTENDED else 0)
|
||||
self._send_input(ki)
|
||||
|
||||
def keypress(self, key):
|
||||
try:
|
||||
parts = parse_combo(key)
|
||||
except ValueError as e:
|
||||
return {'ok': False, 'error': str(e)}
|
||||
vks = [self._vk(p) for p in parts]
|
||||
if None in vks:
|
||||
return {'ok': False, 'error': 'unknown key %s' % key}
|
||||
for vk in vks[:-1]:
|
||||
self._key(vk, False)
|
||||
time.sleep(0.03)
|
||||
self._key(vks[-1], False)
|
||||
time.sleep(0.04)
|
||||
self._key(vks[-1], True)
|
||||
for vk in reversed(vks[:-1]):
|
||||
time.sleep(0.03)
|
||||
self._key(vk, True)
|
||||
return {'ok': True, 'key': str(key), 'backend': 'sendinput'}
|
||||
|
||||
def type_text(self, text):
|
||||
for unit in utf16_units(text):
|
||||
ki = INPUT(); ki.type = INPUT_KEYBOARD
|
||||
ki.ki.wScan = unit; ki.ki.dwFlags = KEYEVENTF_UNICODE
|
||||
ki2 = INPUT(); ki2.type = INPUT_KEYBOARD
|
||||
ki2.ki.wScan = unit
|
||||
ki2.ki.dwFlags = KEYEVENTF_UNICODE | KEYEVENTF_KEYUP
|
||||
self._send_input(ki, ki2)
|
||||
time.sleep(0.01)
|
||||
return {'ok': True, 'typed': str(text), 'backend': 'sendinput'}
|
||||
@@ -1,133 +0,0 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Linux uinput 物理输入后端(虚拟内核 evdev 设备,需 root 或 uinput 组权限)。
|
||||
比 xdotool 更接近硬件:内核层面人为物理设备,无 X 层痕迹。
|
||||
依赖: python-uinput (pip install python-uinput)。
|
||||
|
||||
鼠标用绝对坐标轴(ABS_X/ABS_Y): 相对位移(REL_X/REL_Y)会被 libinput 指针加速放大, 点不准。
|
||||
"""
|
||||
import time
|
||||
|
||||
from ._util import parse_combo
|
||||
|
||||
try:
|
||||
import uinput
|
||||
HAVE_UINPUT = True
|
||||
except ImportError:
|
||||
uinput = None
|
||||
HAVE_UINPUT = False
|
||||
|
||||
ABS_MAX = 65535
|
||||
|
||||
# 键名 -> python-uinput 常量名
|
||||
_NAMED = {
|
||||
'enter': 'ENTER', 'esc': 'ESC', 'space': 'SPACE', 'tab': 'TAB', 'backspace': 'BACKSPACE',
|
||||
'ctrl': 'LEFTCTRL', 'rctrl': 'RIGHTCTRL', 'alt': 'LEFTALT', 'ralt': 'RIGHTALT',
|
||||
'shift': 'LEFTSHIFT', 'lshift': 'LEFTSHIFT', 'rshift': 'RIGHTSHIFT', 'lwin': 'LEFTMETA',
|
||||
'up': 'UP', 'down': 'DOWN', 'left': 'LEFT', 'right': 'RIGHT',
|
||||
'home': 'HOME', 'end': 'END', 'pageup': 'PAGEUP', 'pagedown': 'PAGEDOWN',
|
||||
'insert': 'INSERT', 'delete': 'DELETE',
|
||||
',': 'COMMA', '.': 'DOT', '/': 'SLASH', ';': 'SEMICOLON', "'": 'APOSTROPHE',
|
||||
'[': 'LEFTBRACE', ']': 'RIGHTBRACE', '-': 'MINUS', '=': 'EQUAL', '`': 'GRAVE', '\\': 'BACKSLASH',
|
||||
}
|
||||
_NAMED.update({c: c.upper() for c in 'abcdefghijklmnopqrstuvwxyz0123456789'})
|
||||
_NAMED.update({'f%d' % i: 'F%d' % i for i in range(1, 13)})
|
||||
# US 布局下需要 Shift 的符号 -> 基础键
|
||||
_SHIFTED = {'!': '1', '@': '2', '#': '3', '$': '4', '%': '5', '^': '6', '&': '7', '*': '8',
|
||||
'(': '9', ')': '0', '_': '-', '+': '=', '{': '[', '}': ']', ':': ';', '"': "'",
|
||||
'<': ',', '>': '.', '?': '/', '~': '`', '|': '\\'}
|
||||
|
||||
|
||||
def _key_event(name):
|
||||
const = _NAMED.get(name)
|
||||
return getattr(uinput, 'KEY_' + const) if const else None
|
||||
|
||||
|
||||
def _x_screen_size():
|
||||
try:
|
||||
from Xlib import display
|
||||
s = display.Display().screen()
|
||||
return s.width_in_pixels, s.height_in_pixels
|
||||
except Exception:
|
||||
return 1920, 1080
|
||||
|
||||
|
||||
class UinputBackend:
|
||||
|
||||
def __init__(self, screen=None):
|
||||
if not HAVE_UINPUT:
|
||||
raise RuntimeError('python-uinput not installed (pip install python-uinput)')
|
||||
self._scr_w, self._scr_h = screen or _x_screen_size()
|
||||
events = [getattr(uinput, 'KEY_' + c) for c in sorted(set(_NAMED.values()))]
|
||||
events += [uinput.BTN_LEFT, uinput.BTN_RIGHT,
|
||||
uinput.ABS_X + (0, ABS_MAX, 0, 0), uinput.ABS_Y + (0, ABS_MAX, 0, 0)]
|
||||
self._dev = uinput.Device(events, name='lc-mcp-uinput')
|
||||
time.sleep(0.5) # 等 udev / 显示服务器识别新设备
|
||||
|
||||
def set_screen(self, w, h):
|
||||
self._scr_w, self._scr_h = int(w), int(h)
|
||||
|
||||
def _abs(self, v, size):
|
||||
v = min(max(int(v), 0), max(size - 1, 0))
|
||||
return int(round(v * ABS_MAX / max(size - 1, 1)))
|
||||
|
||||
def move(self, x, y):
|
||||
self._dev.emit(uinput.ABS_X, self._abs(x, self._scr_w), syn=False)
|
||||
self._dev.emit(uinput.ABS_Y, self._abs(y, self._scr_h))
|
||||
time.sleep(0.03)
|
||||
return {'ok': True, 'backend': 'uinput'}
|
||||
|
||||
def click(self, x, y, button='left', clicks=1):
|
||||
self.move(x, y)
|
||||
time.sleep(0.03)
|
||||
btn = uinput.BTN_LEFT if button == 'left' else uinput.BTN_RIGHT
|
||||
for _ in range(max(1, int(clicks))):
|
||||
self._dev.emit(btn, 1)
|
||||
time.sleep(0.05)
|
||||
self._dev.emit(btn, 0)
|
||||
time.sleep(0.05)
|
||||
return {'ok': True, 'clicked': [int(x), int(y)], 'backend': 'uinput'}
|
||||
|
||||
def _tap(self, ev, hold=0.03):
|
||||
self._dev.emit(ev, 1)
|
||||
time.sleep(hold)
|
||||
self._dev.emit(ev, 0)
|
||||
|
||||
def keypress(self, key):
|
||||
try:
|
||||
parts = parse_combo(key)
|
||||
except ValueError as e:
|
||||
return {'ok': False, 'error': str(e)}
|
||||
evs = [_key_event(p) for p in parts]
|
||||
if None in evs:
|
||||
return {'ok': False, 'error': 'unsupported key %s' % key}
|
||||
for ev in evs[:-1]:
|
||||
self._dev.emit(ev, 1)
|
||||
time.sleep(0.03)
|
||||
self._tap(evs[-1], 0.04)
|
||||
for ev in reversed(evs[:-1]):
|
||||
time.sleep(0.03)
|
||||
self._dev.emit(ev, 0)
|
||||
return {'ok': True, 'key': str(key), 'backend': 'uinput'}
|
||||
|
||||
def type_text(self, text):
|
||||
plan = []
|
||||
for ch in str(text):
|
||||
if ch == ' ':
|
||||
plan.append(('space', False))
|
||||
elif ch == '\n':
|
||||
plan.append(('enter', False))
|
||||
elif ch.lower() in _NAMED and len(ch) == 1:
|
||||
plan.append((ch.lower(), ch.isupper()))
|
||||
elif ch in _SHIFTED:
|
||||
plan.append((_SHIFTED[ch], True))
|
||||
else:
|
||||
return {'ok': False, 'error': 'unsupported char %r (uinput 只支持 US 布局 ASCII)' % ch}
|
||||
shift = uinput.KEY_LEFTSHIFT
|
||||
for name, need_shift in plan:
|
||||
if need_shift:
|
||||
self._dev.emit(shift, 1)
|
||||
self._tap(_key_event(name))
|
||||
if need_shift:
|
||||
self._dev.emit(shift, 0)
|
||||
time.sleep(0.03)
|
||||
return {'ok': True, 'typed': str(text), 'backend': 'uinput'}
|
||||
@@ -1,55 +0,0 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Linux X11 输入后端:xdotool(X 事件注入,xdotool 需已安装)。
|
||||
物理化升级路径: uinput 后端(虚拟内核 evdev 设备)。
|
||||
"""
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
from ._util import parse_combo
|
||||
|
||||
# 内部键名 -> X keysym
|
||||
KEYSYM = {'enter': 'Return', 'esc': 'Escape', 'space': 'space', 'tab': 'Tab',
|
||||
'backspace': 'BackSpace', 'alt': 'Alt_L', 'ralt': 'Alt_R', 'ctrl': 'Control_L',
|
||||
'rctrl': 'Control_R', 'shift': 'Shift_L', 'lshift': 'Shift_L', 'rshift': 'Shift_R',
|
||||
'lwin': 'Super_L', 'up': 'Up', 'down': 'Down', 'left': 'Left', 'right': 'Right',
|
||||
'home': 'Home', 'end': 'End', 'pageup': 'Prior', 'pagedown': 'Next',
|
||||
'insert': 'Insert', 'delete': 'Delete'}
|
||||
KEYSYM.update({'f%d' % i: 'F%d' % i for i in range(1, 13)})
|
||||
|
||||
|
||||
def _run(args):
|
||||
r = subprocess.run(args, capture_output=True, text=True, timeout=30)
|
||||
if r.returncode != 0:
|
||||
raise RuntimeError('%s failed: %s' % (' '.join(args[:2]), (r.stderr or '').strip()[-200:]))
|
||||
return r
|
||||
|
||||
|
||||
class XdotoolBackend:
|
||||
|
||||
def __init__(self):
|
||||
if not shutil.which('xdotool'):
|
||||
raise RuntimeError('xdotool not found (apt install xdotool)')
|
||||
|
||||
def move(self, x, y):
|
||||
_run(['xdotool', 'mousemove', '--sync', str(int(x)), str(int(y))])
|
||||
return {'ok': True, 'backend': 'xdotool'}
|
||||
|
||||
def click(self, x, y, button='left', clicks=1):
|
||||
btn = {'left': '1', 'right': '3', 'middle': '2'}.get(button, '1')
|
||||
_run(['xdotool', 'mousemove', '--sync', str(int(x)), str(int(y)),
|
||||
'click', '--repeat', str(max(1, int(clicks))), '--delay', '60', btn])
|
||||
return {'ok': True, 'clicked': [int(x), int(y)], 'backend': 'xdotool'}
|
||||
|
||||
def keypress(self, key):
|
||||
try:
|
||||
parts = parse_combo(key)
|
||||
except ValueError as e:
|
||||
return {'ok': False, 'error': str(e)}
|
||||
combo = '+'.join(KEYSYM.get(p, p) for p in parts)
|
||||
_run(['xdotool', 'key', '--', combo])
|
||||
return {'ok': True, 'key': str(key), 'backend': 'xdotool'}
|
||||
|
||||
def type_text(self, text):
|
||||
# '--' 防止以 '-' 开头的文本被当成选项
|
||||
_run(['xdotool', 'type', '--delay', '60', '--', str(text)])
|
||||
return {'ok': True, 'typed': str(text), 'backend': 'xdotool'}
|
||||
@@ -1,322 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Limbus Company 语义化操控 MCP server v0.6 (跨平台)。
|
||||
|
||||
核心思路: 不让 AI 盲猜坐标。工具层做"文本化 + 语义点击"——
|
||||
get_state : 截图 + OCR -> 全页文字元素(带坐标)
|
||||
click_text : 说文字/按钮名, 内部 OCR 定位后点击
|
||||
wait_for_text / wait_for_text_gone : 等条件出现/消失再回报
|
||||
平台无关: 截图/输入按 LCP_PLATFORM (win|linux|mac, 默认自动检测) 加载实现。
|
||||
win : 截图=交互计划任务; 输入=sendinput|interception
|
||||
linux : 截图=X11 PIL/scrot; 输入=xdotool|uinput(物理)
|
||||
mac : 截图=screencapture; 输入=cg
|
||||
OCR: 纯本地 (Windows.Media.Ocr / winsdk)。
|
||||
协议: MCP stdio (newline-delimited JSON-RPC)。
|
||||
"""
|
||||
import glob
|
||||
import os
|
||||
import re
|
||||
import struct
|
||||
import sys
|
||||
import json
|
||||
import time
|
||||
|
||||
from lcplatform import detect_platform, get_input_backend, get_screenshot_provider
|
||||
|
||||
VERSION = '0.6.0'
|
||||
BACKEND_NAME = os.environ.get('INPUT_BACKEND', '').strip().lower()
|
||||
OCR_BACKEND = os.environ.get('OCR_BACKEND', 'local').strip().lower()
|
||||
MIN_SCORE = float(os.environ.get('OCR_MIN_SCORE', '0.5'))
|
||||
KEEP_SHOTS = int(os.environ.get('LCP_KEEP_SHOTS', '30'))
|
||||
MAX_WAIT = 300.0
|
||||
WAIT_INTERVAL = 1.2
|
||||
BASE = os.path.join(os.path.expanduser('~'), 'lc_mcp')
|
||||
os.makedirs(BASE, exist_ok=True)
|
||||
|
||||
NEED_SCREEN = {'screenshot', 'get_state', 'wait_for_text', 'wait_for_text_gone', 'click_text'}
|
||||
NEED_INPUT = {'click_text', 'click', 'move', 'keypress', 'type_text'}
|
||||
|
||||
|
||||
def log(msg):
|
||||
try:
|
||||
with open(os.path.join(BASE, 'lc_mcp.log'), 'a', encoding='utf-8') as f:
|
||||
f.write('[%s] %s\n' % (time.strftime('%H:%M:%S'), msg))
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def ocr_image(img_path):
|
||||
"""本地 OCR(Windows 自带 OCR,winsdk;无任何云端依赖)。"""
|
||||
try:
|
||||
import ocr_local
|
||||
except Exception as e:
|
||||
return None, 'ocr_local import fail: %s' % e
|
||||
if not ocr_local.available():
|
||||
return None, 'winsdk 不可用: pip install --user winsdk'
|
||||
items, err = ocr_local.ocr_image(img_path)
|
||||
if items is None:
|
||||
return None, err
|
||||
# Windows.Media.Ocr 不给置信度; 只有带 score 的后端才会被过滤
|
||||
items = [it for it in items if float(it.get('score', 1)) >= MIN_SCORE]
|
||||
return items, None
|
||||
|
||||
|
||||
def png_size(path):
|
||||
"""读 PNG IHDR 拿宽高(纯标准库)。失败返回 None。"""
|
||||
try:
|
||||
with open(path, 'rb') as f:
|
||||
head = f.read(24)
|
||||
if head[:8] != b'\x89PNG\r\n\x1a\n':
|
||||
return None
|
||||
return struct.unpack('>II', head[16:24])
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def prune_shots(path, keep=KEEP_SHOTS):
|
||||
"""只保留同目录下最新 keep 张截图,防止轮询把磁盘写满。"""
|
||||
if keep <= 0:
|
||||
return
|
||||
d = os.path.dirname(path)
|
||||
shots = glob.glob(os.path.join(d, 'shot_*.png')) + glob.glob(os.path.join(d, 'lc_shot_*.png'))
|
||||
shots.sort(key=lambda p: os.path.getmtime(p) if os.path.exists(p) else 0)
|
||||
for p in shots[:-keep]:
|
||||
try:
|
||||
os.remove(p)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
_WS = re.compile(r'\s+')
|
||||
|
||||
|
||||
def norm(s):
|
||||
"""匹配用归一化: 去全部空白 + 小写。
|
||||
Windows OCR 对中文按单字切词, 拼行时会变成 "开 始"。"""
|
||||
return _WS.sub('', str(s)).lower()
|
||||
|
||||
|
||||
def calc_click(txt, items):
|
||||
"""在 OCR items 里找包含 txt 的项,返回中心列表。完全相等的排前面。"""
|
||||
key = norm(txt)
|
||||
if not key:
|
||||
return []
|
||||
exact, partial = [], []
|
||||
for it in items:
|
||||
t = norm(it['text'])
|
||||
if key not in t:
|
||||
continue
|
||||
hit = {'text': it['text'], 'x': it['x'] + it['w'] // 2, 'y': it['y'] + it['h'] // 2}
|
||||
(exact if t == key else partial).append(hit)
|
||||
return exact + partial
|
||||
|
||||
|
||||
class MCPServer:
|
||||
|
||||
def __init__(self, platform, provider, backend):
|
||||
self.platform = platform
|
||||
self.provider = provider
|
||||
self.backend = backend
|
||||
self.tools = [
|
||||
{"name": "get_state", "description": "截图并OCR, 返回当前屏幕全部文字元素(含坐标)。AI 决策前先调这个, 不需要自己猜坐标",
|
||||
"inputSchema": {"type": "object", "properties": {}}},
|
||||
{"name": "click_text", "description": "语义点击: 给按钮文字(支持子串, 忽略空格; 完全匹配优先), 内部OCR定位后点击其中心。例: click_text('开始')",
|
||||
"inputSchema": {"type": "object", "properties": {
|
||||
"text": {"type": "string"}, "index": {"type": "number", "default": 0}},
|
||||
"required": ["text"]}},
|
||||
{"name": "wait_for_text", "description": "循环截图+OCR, 等到指定文字出现(默认30秒, 上限300秒)", "inputSchema": {"type": "object", "properties": {
|
||||
"text": {"type": "string"}, "timeout": {"type": "number", "default": 30}},
|
||||
"required": ["text"]}},
|
||||
{"name": "wait_for_text_gone", "description": "循环截图+OCR, 等到指定文字消失(默认30秒, 上限300秒)", "inputSchema": {"type": "object", "properties": {
|
||||
"text": {"type": "string"}, "timeout": {"type": "number", "default": 30}},
|
||||
"required": ["text"]}},
|
||||
{"name": "screenshot", "description": "截图并返回PNG路径(用于模板/人工查看)", "inputSchema": {"type": "object", "properties": {}}},
|
||||
{"name": "click", "description": "直接按坐标点击(截图像素坐标; 一般用 click_text 更稳)", "inputSchema": {"type": "object", "properties": {
|
||||
"x": {"type": "number"}, "y": {"type": "number"},
|
||||
"button": {"type": "string", "enum": ["left", "right"], "default": "left"}},
|
||||
"required": ["x", "y"]}},
|
||||
{"name": "move", "description": "移动鼠标到(x,y)", "inputSchema": {"type": "object", "properties": {
|
||||
"x": {"type": "number"}, "y": {"type": "number"}}, "required": ["x", "y"]}},
|
||||
{"name": "keypress", "description": "按一次键(p/enter/esc/space/tab/alt/f1-f12/单字符等)", "inputSchema": {"type": "object", "properties": {
|
||||
"key": {"type": "string"}}, "required": ["key"]}},
|
||||
{"name": "type_text", "description": "输入一串文本", "inputSchema": {"type": "object", "properties": {
|
||||
"text": {"type": "string"}}, "required": ["text"]}},
|
||||
]
|
||||
|
||||
def _grab(self):
|
||||
"""截图 + 清理旧图 + 把真实屏幕像素尺寸告诉输入后端(坐标换算以截图为准)。"""
|
||||
sh = self.provider.grab()
|
||||
if sh.get('ok'):
|
||||
prune_shots(sh['path'])
|
||||
size = png_size(sh['path'])
|
||||
if size and hasattr(self.backend, 'set_screen'):
|
||||
self.backend.set_screen(*size)
|
||||
return sh
|
||||
|
||||
def _ocr_now(self):
|
||||
sh = self._grab()
|
||||
if not sh.get('ok'):
|
||||
return sh, None
|
||||
items, err = ocr_image(sh['path'])
|
||||
if err:
|
||||
return {'ok': False, 'error': err, 'screen': sh['path']}, None
|
||||
return sh, items or []
|
||||
|
||||
def call(self, name, args):
|
||||
args = args or {}
|
||||
if name == 'screenshot':
|
||||
return self._grab()
|
||||
if name == 'get_state':
|
||||
sh, items = self._ocr_now()
|
||||
if items is None:
|
||||
return sh
|
||||
return {'ok': True, 'screen': sh['path'], 'items': items}
|
||||
if name == 'click_text':
|
||||
txt = str(args.get('text', ''))
|
||||
if not norm(txt):
|
||||
return {'ok': False, 'error': 'text must be non-empty'}
|
||||
sh, items = self._ocr_now()
|
||||
if items is None:
|
||||
return sh
|
||||
hits = calc_click(txt, items)
|
||||
if not hits:
|
||||
return {'ok': False, 'error': 'text not found: %s' % txt,
|
||||
'screen': sh['path'], 'items': items}
|
||||
idx = int(args.get('index', 0))
|
||||
if not 0 <= idx < len(hits):
|
||||
return {'ok': False, 'error': 'index %d out of range (%d hits)' % (idx, len(hits)),
|
||||
'hits': hits[:10]}
|
||||
h = hits[idx]
|
||||
r = self.backend.click(h['x'], h['y'])
|
||||
r['on'] = h['text']
|
||||
if len(hits) > 1:
|
||||
r['other_hits'] = len(hits) - 1
|
||||
return r
|
||||
if name in ('wait_for_text', 'wait_for_text_gone'):
|
||||
txt = str(args.get('text', ''))
|
||||
if not norm(txt):
|
||||
return {'ok': False, 'error': 'text must be non-empty'}
|
||||
timeout = max(0.0, min(float(args.get('timeout', 30)), MAX_WAIT))
|
||||
return self._wait(txt, timeout, name == 'wait_for_text')
|
||||
if name == 'click':
|
||||
button = args.get('button', 'left')
|
||||
if button not in ('left', 'right'):
|
||||
return {'ok': False, 'error': 'button must be left/right'}
|
||||
return self.backend.click(int(args['x']), int(args['y']), button, 1)
|
||||
if name == 'move':
|
||||
return self.backend.move(int(args['x']), int(args['y']))
|
||||
if name == 'keypress':
|
||||
return self.backend.keypress(str(args.get('key', '')))
|
||||
if name == 'type_text':
|
||||
return self.backend.type_text(str(args.get('text', '')))
|
||||
return {'ok': False, 'error': 'unknown tool %s' % name}
|
||||
|
||||
def _wait(self, txt, timeout, want_present):
|
||||
deadline = time.time() + timeout
|
||||
last_items, last_err = [], None
|
||||
while True:
|
||||
sh, items = self._ocr_now()
|
||||
if items is not None:
|
||||
last_items = items
|
||||
hits = calc_click(txt, items)
|
||||
if bool(hits) == want_present:
|
||||
return {'ok': True, 'present': bool(hits),
|
||||
'hits': hits[:5], 'screen': sh['path']}
|
||||
else:
|
||||
last_err = sh.get('error')
|
||||
if time.time() + WAIT_INTERVAL > deadline:
|
||||
break
|
||||
time.sleep(WAIT_INTERVAL)
|
||||
r = {'ok': False, 'error': 'timeout waiting %s (present=%s)' % (txt, want_present),
|
||||
'last_items': last_items[:10]}
|
||||
if last_err:
|
||||
r['last_error'] = last_err
|
||||
return r
|
||||
|
||||
|
||||
def read_msg():
|
||||
"""读一行 JSON-RPC。EOF 返回 None; 空行/坏 JSON 抛 ValueError 由调用方回错误后继续。"""
|
||||
line = sys.stdin.readline()
|
||||
if not line:
|
||||
return None
|
||||
line = line.strip()
|
||||
if not line:
|
||||
return {}
|
||||
msg = json.loads(line)
|
||||
if not isinstance(msg, dict):
|
||||
raise ValueError('request must be a JSON object')
|
||||
return msg
|
||||
|
||||
|
||||
def send(obj):
|
||||
sys.stdout.write(json.dumps(obj, ensure_ascii=False) + '\n')
|
||||
sys.stdout.flush()
|
||||
|
||||
|
||||
def tool_result(mid, result, is_error=False):
|
||||
send({"jsonrpc": "2.0", "id": mid, "result": {
|
||||
"content": [{"type": "text", "text": json.dumps(result, ensure_ascii=False)}],
|
||||
"isError": bool(is_error)}})
|
||||
|
||||
|
||||
def main():
|
||||
platform = detect_platform()
|
||||
log('lc_mcp v%s platform=%s backend=%s ocr=%s' % (VERSION, platform, BACKEND_NAME or 'auto', OCR_BACKEND))
|
||||
provider = backend = None
|
||||
try:
|
||||
provider = get_screenshot_provider(platform)
|
||||
except Exception as e:
|
||||
log('screenshot provider init fail: %s' % e)
|
||||
try:
|
||||
backend = get_input_backend(BACKEND_NAME, platform)
|
||||
except Exception as e:
|
||||
log('input backend init fail: %s' % e)
|
||||
server = MCPServer(platform, provider, backend)
|
||||
while True:
|
||||
try:
|
||||
msg = read_msg()
|
||||
except ValueError as e:
|
||||
log('parse err %s' % e)
|
||||
send({"jsonrpc": "2.0", "id": None, "error": {"code": -32700, "message": "parse error: %s" % e}})
|
||||
continue
|
||||
if msg is None:
|
||||
break
|
||||
if not msg:
|
||||
continue
|
||||
method = msg.get('method')
|
||||
mid = msg.get('id')
|
||||
if method == 'initialize':
|
||||
send({"jsonrpc": "2.0", "id": mid, "result": {
|
||||
"protocolVersion": "2024-11-05",
|
||||
"capabilities": {"tools": {"listChanged": False}},
|
||||
"serverInfo": {"name": "lc-mcp", "version": VERSION}}})
|
||||
elif method == 'ping':
|
||||
send({"jsonrpc": "2.0", "id": mid, "result": {}})
|
||||
elif method == 'tools/list':
|
||||
send({"jsonrpc": "2.0", "id": mid, "result": {"tools": server.tools}})
|
||||
elif method == 'tools/call':
|
||||
params = msg.get('params') or {}
|
||||
name = params.get('name')
|
||||
args = params.get('arguments') or {}
|
||||
log('call %s %s' % (name, args))
|
||||
if name in NEED_SCREEN and server.provider is None:
|
||||
tool_result(mid, {'ok': False, 'error': 'screenshot provider unavailable (see lc_mcp.log)'}, True)
|
||||
continue
|
||||
if name in NEED_INPUT and server.backend is None:
|
||||
tool_result(mid, {'ok': False, 'error': 'input backend unavailable (see lc_mcp.log)'}, True)
|
||||
continue
|
||||
try:
|
||||
result = server.call(name, args)
|
||||
tool_result(mid, result, not result.get('ok', True))
|
||||
except Exception as e:
|
||||
log('tool err %s: %r' % (name, e))
|
||||
tool_result(mid, {'ok': False, 'error': '%s: %s' % (type(e).__name__, e)}, True)
|
||||
elif mid is not None:
|
||||
send({"jsonrpc": "2.0", "id": mid,
|
||||
"error": {"code": -32601, "message": "method not found"}})
|
||||
# 其余无 id 的通知(notifications/*)按规范忽略
|
||||
log('lc_mcp exit')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,4 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""lc-mcp: 以 AALC (AhabAssistantLimbusCompany) 为底座的 Limbus Company MCP server。"""
|
||||
|
||||
VERSION = '0.7.0'
|
||||
@@ -0,0 +1,143 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""AALC 引擎: 截图/OCR/模板匹配/输入全部走 AALC 的 module.automation。
|
||||
|
||||
只能在 Windows 的交互桌面会话里运行(找窗口句柄、后台截图、PostMessage 都依赖它)。
|
||||
AALC 代码按相对路径读 assets/ 和 config.yaml, 所以 bootstrap() 会 chdir 到 AALC 目录。
|
||||
"""
|
||||
import glob
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
|
||||
from .engine import Engine
|
||||
|
||||
REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
DEFAULT_AALC_DIR = os.path.join(REPO, 'vendor', 'aalc')
|
||||
_booted = None
|
||||
|
||||
|
||||
def aalc_dir():
|
||||
return os.path.abspath(os.environ.get('AALC_DIR') or DEFAULT_AALC_DIR)
|
||||
|
||||
|
||||
def bootstrap():
|
||||
"""进程内初始化 AALC (幂等)。必须在 import 任何 AALC 模块之前调用。"""
|
||||
global _booted
|
||||
if _booted:
|
||||
return _booted
|
||||
d = aalc_dir()
|
||||
if not os.path.isfile(os.path.join(d, 'main.py')):
|
||||
raise RuntimeError('AALC 目录无效: %s (git submodule update --init, 或设置 AALC_DIR)' % d)
|
||||
os.chdir(d)
|
||||
if d not in sys.path:
|
||||
sys.path.insert(0, d)
|
||||
# 与 AALC main.py 相同的 DPI 设置, 否则截图/点击坐标在缩放下错位
|
||||
from ctypes import c_void_p, windll
|
||||
try:
|
||||
windll.user32.SetProcessDpiAwarenessContext(c_void_p(-4))
|
||||
except (AttributeError, OSError):
|
||||
try:
|
||||
windll.shcore.SetProcessDpiAwareness(2)
|
||||
except (AttributeError, OSError):
|
||||
windll.user32.SetProcessDPIAware()
|
||||
from module.logger.my_log import Logger
|
||||
Logger()
|
||||
_booted = d
|
||||
return d
|
||||
|
||||
|
||||
def prune(directory, pattern, keep):
|
||||
files = sorted(glob.glob(os.path.join(directory, pattern)), key=os.path.getmtime)
|
||||
for p in files[:-keep] if keep > 0 else []:
|
||||
try:
|
||||
os.remove(p)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
class AalcEngine(Engine):
|
||||
|
||||
def __init__(self, shot_dir, keep_shots=30):
|
||||
self.shot_dir = os.path.abspath(shot_dir)
|
||||
os.makedirs(self.shot_dir, exist_ok=True)
|
||||
self.keep_shots = keep_shots
|
||||
root = bootstrap()
|
||||
from module.automation import auto
|
||||
from module.automation.input_handlers.input import key_list
|
||||
from module.automation.screenshot import ScreenShot
|
||||
from module.config import cfg
|
||||
from module.game_and_screen import screen
|
||||
from module.ocr import ocr
|
||||
from utils.path_manager import path_manager
|
||||
path_manager.initialize_paths()
|
||||
self.auto, self.cfg, self.screen, self.ocr_engine = auto, cfg, screen, ocr
|
||||
self.ScreenShot = ScreenShot
|
||||
self.key_names = sorted(key_list)
|
||||
self.images_root = os.path.join(root, 'assets', 'images')
|
||||
|
||||
# ---- 看 ----
|
||||
def _grab(self, gray):
|
||||
if not self.cfg.simulator and not self.screen.handle.hwnd:
|
||||
raise RuntimeError('未找到游戏窗口 %r: 先启动游戏(run_task 会自动启动)' % self.cfg.game_title_name)
|
||||
# 不用 auto.take_screenshot: 它连续失败 60 秒会直接 taskkill 游戏
|
||||
with self.auto._screenshot_lock:
|
||||
img = self.ScreenShot.take_screenshot(gray)
|
||||
if img is None:
|
||||
raise RuntimeError('AALC 截图失败(窗口最小化/被遮挡? 见 AALC logs)')
|
||||
self.auto.screenshot = img # find_image_element 读这个
|
||||
return img
|
||||
|
||||
def _save(self, img):
|
||||
stamp = time.strftime('%Y%m%d_%H%M%S') + '_%03d' % (int(time.time() * 1000) % 1000)
|
||||
path = os.path.join(self.shot_dir, 'shot_%s.png' % stamp)
|
||||
img.save(path)
|
||||
prune(self.shot_dir, 'shot_*.png', self.keep_shots)
|
||||
return {'path': path.replace('\\', '/'), 'w': img.width, 'h': img.height}
|
||||
|
||||
def screenshot(self):
|
||||
return self._save(self._grab(gray=False))
|
||||
|
||||
def ocr(self):
|
||||
img = self._grab(gray=False)
|
||||
shot = self._save(img)
|
||||
res = self.ocr_engine.run(img)
|
||||
items = []
|
||||
boxes = res.boxes if res.boxes is not None else []
|
||||
scores = res.scores if res.scores is not None else [1.0] * len(boxes)
|
||||
for box, text, score in zip(boxes, res.txts or (), scores):
|
||||
xs = [p[0] for p in box]
|
||||
ys = [p[1] for p in box]
|
||||
x0, y0 = int(min(xs)), int(min(ys))
|
||||
items.append({'text': text, 'x': x0, 'y': y0,
|
||||
'w': int(max(xs)) - x0, 'h': int(max(ys)) - y0,
|
||||
'score': round(float(score), 3)})
|
||||
return shot, items
|
||||
|
||||
def list_images(self, prefix=''):
|
||||
names = set()
|
||||
for theme in ('dark', 'default'):
|
||||
for lang in ('zh_cn', 'en', 'share'):
|
||||
base = os.path.join(self.images_root, theme, lang)
|
||||
for p in glob.glob(os.path.join(base, '**', '*.png'), recursive=True):
|
||||
names.add(os.path.relpath(p, base).replace('\\', '/'))
|
||||
return sorted(n for n in names if n.startswith(prefix))
|
||||
|
||||
def find_image(self, name, threshold=0.8, model='normal'):
|
||||
if name not in self.list_images(name):
|
||||
raise ValueError('没有模板图 %r (用 list_images 查)' % name)
|
||||
self._grab(gray=True)
|
||||
pos = self.auto.find_image_element(name, threshold, model=model)
|
||||
return (int(pos[0]), int(pos[1])) if pos else None
|
||||
|
||||
# ---- 动手 (AALC 输入后端, 由 config.yaml win_input_type 决定: 默认后台 PostMessage) ----
|
||||
def click(self, x, y, times=1):
|
||||
self.auto.mouse_click(int(x), int(y), times=int(times))
|
||||
|
||||
def keypress(self, key):
|
||||
k = str(key).lower()
|
||||
if k not in self.key_names:
|
||||
raise ValueError('AALC 不支持按键 %r, 可用: %s' % (key, ' '.join(self.key_names)))
|
||||
self.auto.key_press(k)
|
||||
|
||||
def type_text(self, text):
|
||||
self.auto.input_text(str(text))
|
||||
@@ -0,0 +1,81 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""子进程入口: 跑 AALC 现成的任务流程。
|
||||
|
||||
用法: python -m lc_mcp.aalc_task <task> [json_params]
|
||||
放在子进程里跑, 停止 = 杀进程树; 不会像 QThread.terminate 那样留下被占用的锁。
|
||||
"""
|
||||
import json
|
||||
import sys
|
||||
|
||||
from .aalc_engine import bootstrap
|
||||
|
||||
# 任务 -> {参数名: AALC 配置项}; 参数只在本次运行内生效
|
||||
TASKS = {
|
||||
'all': {}, # 完全按 config.yaml 里勾选的任务跑, 等价于 AALC 界面点"开始"
|
||||
'daily': {'exp': 'set_EXP_count', 'thread': 'set_thread_count'},
|
||||
'reward': {},
|
||||
'enkephalin': {'times': 'set_lunacy_to_enkephalin'},
|
||||
'mirror': {'count': 'set_mirror_count'},
|
||||
}
|
||||
|
||||
|
||||
def run(task, params):
|
||||
bootstrap()
|
||||
from module.automation import auto
|
||||
from module.config import cfg
|
||||
from tasks.base import script_task_scheme as scheme
|
||||
from tasks.base.retry_monitor import retry_monitor
|
||||
from tasks.battle import battle
|
||||
from utils.path_manager import path_manager
|
||||
|
||||
originals = {}
|
||||
for name, value in params.items():
|
||||
key = TASKS[task][name]
|
||||
originals[key] = cfg.get_value(key)
|
||||
cfg.unsaved_set_value(key, int(value))
|
||||
try:
|
||||
if task == 'all':
|
||||
scheme.script_task()
|
||||
return
|
||||
# 与 script_task() 相同的前置步骤
|
||||
scheme.init_game()
|
||||
path_manager.initialize_paths()
|
||||
auto.clear_img_cache()
|
||||
retry_monitor.start()
|
||||
get_reward = None
|
||||
if auto.click_element('battle/turn_assets.png', take_screenshot=True):
|
||||
print('[lc_mcp] 检测到正在战斗, 先打完', flush=True)
|
||||
get_reward = battle.fight()
|
||||
{
|
||||
'daily': scheme.Daily_task_wrapper(get_reward=get_reward),
|
||||
'reward': scheme.to_get_reward,
|
||||
'enkephalin': scheme.Buy_enkephalin,
|
||||
'mirror': scheme.Mirror_task,
|
||||
}[task]()
|
||||
finally:
|
||||
retry_monitor.stop()
|
||||
# AALC 运行中会自行保存配置(换队伍队列等), 把临时覆盖的次数写回原值
|
||||
for key, value in originals.items():
|
||||
cfg.set_value(key, value)
|
||||
if originals:
|
||||
cfg.flush()
|
||||
|
||||
|
||||
def main(argv):
|
||||
if len(argv) < 2 or argv[1] not in TASKS:
|
||||
print('usage: python -m lc_mcp.aalc_task <%s> [json]' % '|'.join(TASKS), file=sys.stderr)
|
||||
return 2
|
||||
task = argv[1]
|
||||
params = json.loads(argv[2]) if len(argv) > 2 else {}
|
||||
unknown = set(params) - set(TASKS[task])
|
||||
if unknown:
|
||||
print('unknown params for %s: %s' % (task, ', '.join(sorted(unknown))), file=sys.stderr)
|
||||
return 2
|
||||
print('[lc_mcp] task %s start %s' % (task, json.dumps(params)), flush=True)
|
||||
run(task, params)
|
||||
print('[lc_mcp] task %s done' % task, flush=True)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main(sys.argv))
|
||||
@@ -0,0 +1,84 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""stdio <-> 常驻服务 的中继。MCP 客户端(经 ssh)启动的是这个, 只用标准库。
|
||||
|
||||
ssh win python C:/path/lc-mcp/lc_mcp/bridge.py
|
||||
|
||||
为什么不直接 ssh 跑 server: SSH 会话在 session 0, 看不到游戏窗口、也发不了输入。
|
||||
服务没起来时, 对每个请求回明确的错误, 而不是让客户端干等。
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import socket
|
||||
import sys
|
||||
import threading
|
||||
|
||||
HOME = os.path.join(os.path.expanduser('~'), 'lc_mcp')
|
||||
TOKEN_FILE = os.path.join(HOME, 'token')
|
||||
PORT = int(os.environ.get('LCP_PORT', '8765'))
|
||||
|
||||
|
||||
def connect(port=PORT, token_file=TOKEN_FILE):
|
||||
with open(token_file, encoding='ascii') as f:
|
||||
token = f.read().strip()
|
||||
sock = socket.create_connection(('127.0.0.1', port), timeout=5)
|
||||
sock.settimeout(None)
|
||||
sock.sendall(('AUTH %s\n' % token).encode('ascii'))
|
||||
rfile = sock.makefile('rb')
|
||||
if rfile.readline().strip() != b'OK':
|
||||
raise ConnectionError('token 被拒绝 (%s 与服务端不一致?)' % token_file)
|
||||
return sock, rfile
|
||||
|
||||
|
||||
def relay(sock, rfile, stdin, stdout):
|
||||
def up():
|
||||
try:
|
||||
for line in iter(stdin.readline, b''):
|
||||
sock.sendall(line)
|
||||
finally:
|
||||
try:
|
||||
sock.shutdown(socket.SHUT_WR)
|
||||
except OSError:
|
||||
pass
|
||||
threading.Thread(target=up, daemon=True).start()
|
||||
for line in iter(rfile.readline, b''):
|
||||
stdout.write(line)
|
||||
stdout.flush()
|
||||
|
||||
|
||||
def answer_offline(reason, stdin, stdout):
|
||||
"""服务不可达: 仍完成握手, 但每个工具调用都返回原因。"""
|
||||
for raw in iter(stdin.readline, b''):
|
||||
try:
|
||||
msg = json.loads(raw)
|
||||
except ValueError:
|
||||
continue
|
||||
if not isinstance(msg, dict) or msg.get('id') is None:
|
||||
continue
|
||||
if msg.get('method') == 'initialize':
|
||||
resp = {'protocolVersion': '2024-11-05', 'capabilities': {'tools': {}},
|
||||
'serverInfo': {'name': 'lc-mcp (offline)', 'version': '0'}}
|
||||
out = {'jsonrpc': '2.0', 'id': msg['id'], 'result': resp}
|
||||
elif msg.get('method') == 'tools/list':
|
||||
out = {'jsonrpc': '2.0', 'id': msg['id'], 'result': {'tools': []}}
|
||||
else:
|
||||
out = {'jsonrpc': '2.0', 'id': msg['id'], 'error': {'code': -32000, 'message': reason}}
|
||||
stdout.write((json.dumps(out, ensure_ascii=False) + '\n').encode('utf-8'))
|
||||
stdout.flush()
|
||||
|
||||
|
||||
def main():
|
||||
stdin, stdout = sys.stdin.buffer, sys.stdout.buffer
|
||||
try:
|
||||
sock, rfile = connect()
|
||||
except (OSError, ConnectionError) as e:
|
||||
reason = ('lc-mcp 服务未运行或不可达 (%s)。在 Windows 桌面会话里启动: '
|
||||
'scripts/install_logon_task.ps1 或 python -m lc_mcp.server' % e)
|
||||
sys.stderr.write(reason + '\n')
|
||||
answer_offline(reason, stdin, stdout)
|
||||
return 1
|
||||
relay(sock, rfile, stdin, stdout)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,58 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""引擎接口 + 平台无关的文字匹配。
|
||||
|
||||
引擎 = "看屏幕 + 动手" 的最小集合, 坐标一律是游戏窗口客户区截图里的像素坐标。
|
||||
实现: AalcEngine(Windows, 调 AALC) / FakeEngine(测试)。
|
||||
"""
|
||||
import re
|
||||
|
||||
_WS = re.compile(r'\s+')
|
||||
|
||||
|
||||
def norm(s):
|
||||
"""匹配用归一化: 去全部空白 + 小写 (OCR 常把 "Hong Lu" 识别成 "HongLu" 或反之)。"""
|
||||
return _WS.sub('', str(s)).lower()
|
||||
|
||||
|
||||
def find_hits(text, items):
|
||||
"""在 OCR items 里找包含 text 的项, 返回中心点列表。完全相等的排前面。"""
|
||||
key = norm(text)
|
||||
if not key:
|
||||
return []
|
||||
exact, partial = [], []
|
||||
for it in items:
|
||||
t = norm(it['text'])
|
||||
if key not in t:
|
||||
continue
|
||||
hit = {'text': it['text'], 'x': it['x'] + it['w'] // 2, 'y': it['y'] + it['h'] // 2}
|
||||
(exact if t == key else partial).append(hit)
|
||||
return exact + partial
|
||||
|
||||
|
||||
class Engine:
|
||||
"""所有方法失败时抛异常, 由协议层转成 isError 结果。"""
|
||||
|
||||
def screenshot(self):
|
||||
"""截图并落盘。返回 {'path': str, 'w': int, 'h': int}。"""
|
||||
raise NotImplementedError
|
||||
|
||||
def ocr(self):
|
||||
"""截图 + OCR。返回 (shot, items); items = [{text, x, y, w, h, score}]。"""
|
||||
raise NotImplementedError
|
||||
|
||||
def find_image(self, name, threshold=0.8, model='normal'):
|
||||
"""在当前画面找 AALC 模板图 (如 'battle/turn_assets.png')。返回 (x, y) 或 None。"""
|
||||
raise NotImplementedError
|
||||
|
||||
def list_images(self, prefix=''):
|
||||
"""列出可用模板图的相对路径。"""
|
||||
raise NotImplementedError
|
||||
|
||||
def click(self, x, y, times=1):
|
||||
raise NotImplementedError
|
||||
|
||||
def keypress(self, key):
|
||||
raise NotImplementedError
|
||||
|
||||
def type_text(self, text):
|
||||
raise NotImplementedError
|
||||
@@ -0,0 +1,37 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""测试用引擎: 屏幕内容由测试脚本预设, 记录所有输入动作。"""
|
||||
from .engine import Engine
|
||||
|
||||
|
||||
class FakeEngine(Engine):
|
||||
|
||||
def __init__(self, frames=None, images=None):
|
||||
# frames: 每次 ocr() 依次返回的 items 列表; 用完后停在最后一帧
|
||||
self.frames = list(frames or [[]])
|
||||
self.images = dict(images or {}) # name -> (x, y)
|
||||
self.actions = []
|
||||
self.shots = 0
|
||||
|
||||
def screenshot(self):
|
||||
self.shots += 1
|
||||
return {'path': 'fake/shot_%d.png' % self.shots, 'w': 1920, 'h': 1080}
|
||||
|
||||
def ocr(self):
|
||||
shot = self.screenshot()
|
||||
items = self.frames.pop(0) if len(self.frames) > 1 else self.frames[0]
|
||||
return shot, [dict(it, score=it.get('score', 0.99)) for it in items]
|
||||
|
||||
def find_image(self, name, threshold=0.8, model='normal'):
|
||||
return self.images.get(name)
|
||||
|
||||
def list_images(self, prefix=''):
|
||||
return sorted(n for n in self.images if n.startswith(prefix))
|
||||
|
||||
def click(self, x, y, times=1):
|
||||
self.actions.append(('click', int(x), int(y), int(times)))
|
||||
|
||||
def keypress(self, key):
|
||||
self.actions.append(('key', key))
|
||||
|
||||
def type_text(self, text):
|
||||
self.actions.append(('type', text))
|
||||
@@ -0,0 +1,209 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""MCP (JSON-RPC 2.0) 处理: 与传输方式无关, handle(msg) -> 响应 dict 或 None(通知)。"""
|
||||
import json
|
||||
import threading
|
||||
import time
|
||||
|
||||
from . import VERSION
|
||||
from .aalc_task import TASKS
|
||||
from .engine import find_hits
|
||||
|
||||
MAX_WAIT = 300.0
|
||||
WAIT_INTERVAL = 1.0
|
||||
INPUT_TOOLS = {'click', 'click_text', 'click_image', 'keypress', 'type_text'}
|
||||
TASK_TOOLS = {'run_task', 'task_status', 'stop_task'}
|
||||
|
||||
|
||||
def _obj(props=None, required=()):
|
||||
s = {'type': 'object', 'properties': props or {}}
|
||||
if required:
|
||||
s['required'] = list(required)
|
||||
return s
|
||||
|
||||
|
||||
_STR = {'type': 'string'}
|
||||
_NUM = {'type': 'number'}
|
||||
_TIMEOUT = {'type': 'number', 'default': 30, 'description': '秒, 上限 300'}
|
||||
_IMG = {'name': {'type': 'string', 'description': 'AALC 模板图相对路径, 如 battle/turn_assets.png'},
|
||||
'threshold': {'type': 'number', 'default': 0.8},
|
||||
'model': {'type': 'string', 'enum': ['clam', 'normal', 'aggressive'], 'default': 'normal',
|
||||
'description': 'clam=只在模板原位置找, normal=扩大范围, aggressive=全屏'}}
|
||||
|
||||
TOOLS = [
|
||||
{'name': 'get_state', 'description': '截图 + OCR, 返回游戏窗口内全部文字(含坐标/置信度)。决策前先调这个, 不要猜坐标',
|
||||
'inputSchema': _obj()},
|
||||
{'name': 'click_text', 'description': '按文字点击: OCR 定位后点中心。子串匹配、忽略空格, 完全相同的优先; index 选第几个命中',
|
||||
'inputSchema': _obj({'text': _STR, 'index': {'type': 'number', 'default': 0}}, ['text'])},
|
||||
{'name': 'wait_for_text', 'description': '轮询 OCR 直到文字出现', 'inputSchema': _obj({'text': _STR, 'timeout': _TIMEOUT}, ['text'])},
|
||||
{'name': 'wait_for_text_gone', 'description': '轮询 OCR 直到文字消失', 'inputSchema': _obj({'text': _STR, 'timeout': _TIMEOUT}, ['text'])},
|
||||
{'name': 'find_image', 'description': '用 AALC 模板图在当前画面里找图标/按钮(比 OCR 稳, 适合没有文字的按钮)',
|
||||
'inputSchema': _obj(_IMG, ['name'])},
|
||||
{'name': 'click_image', 'description': '找到 AALC 模板图并点击它', 'inputSchema': _obj(_IMG, ['name'])},
|
||||
{'name': 'list_images', 'description': '列出 AALC 模板图(按前缀过滤, 如 mirror/ battle/ home/)',
|
||||
'inputSchema': _obj({'prefix': {'type': 'string', 'default': ''}})},
|
||||
{'name': 'screenshot', 'description': '截游戏窗口, 返回 PNG 路径和尺寸', 'inputSchema': _obj()},
|
||||
{'name': 'click', 'description': '按窗口截图像素坐标点击(优先用 click_text / click_image)',
|
||||
'inputSchema': _obj({'x': _NUM, 'y': _NUM, 'times': {'type': 'number', 'default': 1}}, ['x', 'y'])},
|
||||
{'name': 'keypress', 'description': '按一次键: a-z 0-9 enter esc space tab shift ctrl alt up down left right',
|
||||
'inputSchema': _obj({'key': _STR}, ['key'])},
|
||||
{'name': 'type_text', 'description': '向游戏窗口输入文本', 'inputSchema': _obj({'text': _STR}, ['text'])},
|
||||
{'name': 'run_task', 'description': (
|
||||
'后台运行 AALC 现成的完整任务流程(会自动启动游戏、处理战斗/掉线重试)。立即返回, 用 task_status 看进度。'
|
||||
'all=按 AALC config.yaml 勾选的全部任务; daily=经验本+纽本(exp/thread 次数); reward=领奖励; '
|
||||
'enkephalin=狂气换体力(times); mirror=镜牢(count 次数)。任务运行期间禁止手动点击类工具'),
|
||||
'inputSchema': _obj({'task': {'type': 'string', 'enum': sorted(TASKS)},
|
||||
'exp': _NUM, 'thread': _NUM, 'times': _NUM, 'count': _NUM}, ['task'])},
|
||||
{'name': 'task_status', 'description': '查看任务状态(idle/running/succeeded/failed/stopped)和日志尾部',
|
||||
'inputSchema': _obj({'lines': {'type': 'number', 'default': 30}})},
|
||||
{'name': 'stop_task', 'description': '强制停止当前任务(杀掉任务进程)', 'inputSchema': _obj()},
|
||||
]
|
||||
|
||||
|
||||
class Server:
|
||||
|
||||
def __init__(self, engine, tasks, engine_error=None):
|
||||
self.engine = engine
|
||||
self.engine_error = engine_error
|
||||
self.tasks = tasks
|
||||
# 游戏只有一个: 所有工具调用串行, 防止两个客户端同时点
|
||||
self.lock = threading.Lock()
|
||||
|
||||
# ---- JSON-RPC ----
|
||||
def handle(self, msg):
|
||||
if not isinstance(msg, dict):
|
||||
return _error(None, -32600, 'request must be a JSON object')
|
||||
method, mid = msg.get('method'), msg.get('id')
|
||||
if method == 'initialize':
|
||||
return _ok(mid, {'protocolVersion': '2024-11-05',
|
||||
'capabilities': {'tools': {'listChanged': False}},
|
||||
'serverInfo': {'name': 'lc-mcp', 'version': VERSION}})
|
||||
if method == 'ping':
|
||||
return _ok(mid, {})
|
||||
if method == 'tools/list':
|
||||
return _ok(mid, {'tools': TOOLS})
|
||||
if method == 'tools/call':
|
||||
params = msg.get('params') or {}
|
||||
result = self.call_tool(params.get('name'), params.get('arguments') or {})
|
||||
return _ok(mid, {'content': [{'type': 'text', 'text': json.dumps(result, ensure_ascii=False)}],
|
||||
'isError': not result.get('ok', False)})
|
||||
if mid is None:
|
||||
return None # 通知 (notifications/*) 不回
|
||||
return _error(mid, -32601, 'method not found: %s' % method)
|
||||
|
||||
def call_tool(self, name, args):
|
||||
try:
|
||||
if name in TASK_TOOLS: # 不抢游戏锁: wait_for_text 期间也能查任务/停任务
|
||||
return self._task_tool(name, args)
|
||||
with self.lock:
|
||||
return self._call(name, args)
|
||||
except (KeyError, TypeError, ValueError) as e:
|
||||
return {'ok': False, 'error': 'bad arguments: %s: %s' % (type(e).__name__, e)}
|
||||
except Exception as e:
|
||||
return {'ok': False, 'error': '%s: %s' % (type(e).__name__, e)}
|
||||
|
||||
# ---- 工具 ----
|
||||
def _task_tool(self, name, a):
|
||||
if name == 'run_task':
|
||||
task = a['task']
|
||||
if task not in TASKS:
|
||||
raise ValueError('unknown task %r' % task)
|
||||
params = {k: int(a[k]) for k in TASKS[task] if a.get(k) is not None}
|
||||
return self.tasks.start(task, params)
|
||||
if name == 'task_status':
|
||||
return self.tasks.status(a.get('lines', 30))
|
||||
return self.tasks.stop()
|
||||
|
||||
def _call(self, name, a):
|
||||
if not any(t['name'] == name for t in TOOLS):
|
||||
return {'ok': False, 'error': 'unknown tool %s' % name}
|
||||
if self.engine is None:
|
||||
return {'ok': False, 'error': 'AALC 引擎不可用: %s' % self.engine_error}
|
||||
if name in INPUT_TOOLS and self.tasks.running():
|
||||
return {'ok': False, 'error': 'AALC 任务 %s 正在运行, 手动输入会打乱它; 先 stop_task' % self.tasks.info['task']}
|
||||
e = self.engine
|
||||
|
||||
if name == 'screenshot':
|
||||
return dict(e.screenshot(), ok=True)
|
||||
if name == 'get_state':
|
||||
shot, items = e.ocr()
|
||||
return {'ok': True, 'screen': shot, 'items': items}
|
||||
if name == 'click_text':
|
||||
text = _text(a)
|
||||
shot, items = e.ocr()
|
||||
hits = find_hits(text, items)
|
||||
if not hits:
|
||||
return {'ok': False, 'error': 'text not found: %s' % text, 'screen': shot, 'items': items}
|
||||
idx = int(a.get('index', 0))
|
||||
if not 0 <= idx < len(hits):
|
||||
return {'ok': False, 'error': 'index %d out of range (%d hits)' % (idx, len(hits)), 'hits': hits[:10]}
|
||||
h = hits[idx]
|
||||
e.click(h['x'], h['y'])
|
||||
return {'ok': True, 'clicked': h, 'other_hits': len(hits) - 1}
|
||||
if name in ('wait_for_text', 'wait_for_text_gone'):
|
||||
return self._wait(_text(a), float(a.get('timeout', 30)), name == 'wait_for_text')
|
||||
if name in ('find_image', 'click_image'):
|
||||
pos = e.find_image(a['name'], float(a.get('threshold', 0.8)), a.get('model', 'normal'))
|
||||
if pos is None:
|
||||
return {'ok': name == 'find_image', 'found': False, 'name': a['name']}
|
||||
if name == 'click_image':
|
||||
e.click(*pos)
|
||||
return {'ok': True, 'found': True, 'name': a['name'], 'x': pos[0], 'y': pos[1]}
|
||||
if name == 'list_images':
|
||||
imgs = e.list_images(a.get('prefix', ''))
|
||||
return {'ok': True, 'count': len(imgs), 'images': imgs[:300], 'truncated': len(imgs) > 300}
|
||||
if name == 'click':
|
||||
e.click(int(a['x']), int(a['y']), max(1, int(a.get('times', 1))))
|
||||
return {'ok': True, 'clicked': [int(a['x']), int(a['y'])]}
|
||||
if name == 'keypress':
|
||||
e.keypress(str(a['key']))
|
||||
return {'ok': True, 'key': str(a['key'])}
|
||||
if name == 'type_text':
|
||||
e.type_text(str(a['text']))
|
||||
return {'ok': True, 'typed': str(a['text'])}
|
||||
return {'ok': False, 'error': 'unknown tool %s' % name}
|
||||
|
||||
def _wait(self, text, timeout, want_present):
|
||||
deadline = time.monotonic() + max(0.0, min(timeout, MAX_WAIT))
|
||||
items, last_err = [], None
|
||||
while True:
|
||||
try:
|
||||
shot, items = self.engine.ocr()
|
||||
hits = find_hits(text, items)
|
||||
if bool(hits) == want_present:
|
||||
return {'ok': True, 'present': want_present, 'hits': hits[:5], 'screen': shot}
|
||||
except Exception as ex:
|
||||
last_err = '%s: %s' % (type(ex).__name__, ex)
|
||||
if time.monotonic() + WAIT_INTERVAL > deadline:
|
||||
break
|
||||
time.sleep(WAIT_INTERVAL)
|
||||
r = {'ok': False, 'error': 'timeout waiting %r (want present=%s)' % (text, want_present),
|
||||
'last_items': items[:15]}
|
||||
if last_err:
|
||||
r['last_error'] = last_err
|
||||
return r
|
||||
|
||||
|
||||
def _text(a):
|
||||
text = str(a.get('text', ''))
|
||||
if not text.strip():
|
||||
raise ValueError('text must be non-empty')
|
||||
return text
|
||||
|
||||
|
||||
def _ok(mid, result):
|
||||
return {'jsonrpc': '2.0', 'id': mid, 'result': result}
|
||||
|
||||
|
||||
def _error(mid, code, message):
|
||||
return {'jsonrpc': '2.0', 'id': mid, 'error': {'code': code, 'message': message}}
|
||||
|
||||
|
||||
def parse_line(line):
|
||||
"""一行 -> (msg, error_response)。空行返回 (None, None)。"""
|
||||
line = line.strip()
|
||||
if not line:
|
||||
return None, None
|
||||
try:
|
||||
return json.loads(line), None
|
||||
except ValueError as e:
|
||||
return None, _error(None, -32700, 'parse error: %s' % e)
|
||||
@@ -0,0 +1,136 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""lc-mcp 常驻服务: 跑在 Windows 交互桌面会话里(登录时由计划任务启动)。
|
||||
|
||||
python -m lc_mcp.server 监听 127.0.0.1:PORT, MCP 客户端经 bridge.py 连入
|
||||
python -m lc_mcp.server --stdio 直接走 stdio (本机、且本进程就在桌面会话时用)
|
||||
--fake 用 FakeEngine (开发/测试, 不需要 Windows/AALC)
|
||||
|
||||
TCP 协议: 首行 "AUTH <token>", 服务端回 "OK"; 之后逐行 JSON-RPC。
|
||||
token 存在 ~/lc_mcp/token (用户目录默认只有本人可读), 防止同机其他用户/进程连进来操作游戏。
|
||||
"""
|
||||
import argparse
|
||||
import hmac
|
||||
import os
|
||||
import secrets
|
||||
import socketserver
|
||||
import sys
|
||||
import threading
|
||||
|
||||
from .protocol import Server, parse_line
|
||||
from .tasks import TaskRunner
|
||||
|
||||
HOME = os.path.join(os.path.expanduser('~'), 'lc_mcp')
|
||||
TOKEN_FILE = os.path.join(HOME, 'token')
|
||||
DEFAULT_PORT = int(os.environ.get('LCP_PORT', '8765'))
|
||||
|
||||
|
||||
def log(msg):
|
||||
sys.stderr.write('[lc_mcp] %s\n' % msg)
|
||||
sys.stderr.flush()
|
||||
|
||||
|
||||
def load_token(create=False):
|
||||
try:
|
||||
with open(TOKEN_FILE, encoding='ascii') as f:
|
||||
tok = f.read().strip()
|
||||
if tok:
|
||||
return tok
|
||||
except OSError:
|
||||
if not create:
|
||||
raise
|
||||
os.makedirs(HOME, exist_ok=True)
|
||||
tok = secrets.token_hex(24)
|
||||
fd = os.open(TOKEN_FILE, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
|
||||
with os.fdopen(fd, 'w', encoding='ascii') as f:
|
||||
f.write(tok)
|
||||
return tok
|
||||
|
||||
|
||||
def build_server(fake=False):
|
||||
os.makedirs(HOME, exist_ok=True)
|
||||
tasks = TaskRunner(os.path.join(HOME, 'runs'))
|
||||
if fake:
|
||||
from .fake_engine import FakeEngine
|
||||
return Server(FakeEngine(), tasks)
|
||||
try:
|
||||
from .aalc_engine import AalcEngine
|
||||
return Server(AalcEngine(os.path.join(HOME, 'shots'), int(os.environ.get('LCP_KEEP_SHOTS', '30'))), tasks)
|
||||
except Exception as e:
|
||||
# 引擎起不来也继续服务: run_task 仍可用, 其他工具返回原因
|
||||
log('AALC engine init failed: %r' % e)
|
||||
return Server(None, tasks, engine_error='%s: %s' % (type(e).__name__, e))
|
||||
|
||||
|
||||
def serve_lines(server, readline, write):
|
||||
"""逐行处理直到 EOF。坏行回 parse error 后继续, 不退出。"""
|
||||
import json
|
||||
while True:
|
||||
line = readline()
|
||||
if not line:
|
||||
return
|
||||
msg, err = parse_line(line)
|
||||
resp = err if err else (server.handle(msg) if msg is not None else None)
|
||||
if resp is not None:
|
||||
write(json.dumps(resp, ensure_ascii=False) + '\n')
|
||||
|
||||
|
||||
def run_stdio(server):
|
||||
out = sys.stdout
|
||||
sys.stdout = sys.stderr # AALC 或第三方库的 print 不能混进协议流
|
||||
|
||||
def write(s):
|
||||
out.write(s)
|
||||
out.flush()
|
||||
serve_lines(server, sys.stdin.readline, write)
|
||||
|
||||
|
||||
class _Handler(socketserver.StreamRequestHandler):
|
||||
|
||||
def handle(self):
|
||||
rfile, wfile = self.rfile, self.wfile
|
||||
first = rfile.readline(256).decode('utf-8', 'replace').strip()
|
||||
if not (first.startswith('AUTH ') and hmac.compare_digest(first[5:], self.server.token)):
|
||||
wfile.write(b'DENIED\n')
|
||||
log('rejected connection from %s' % (self.client_address,))
|
||||
return
|
||||
wfile.write(b'OK\n')
|
||||
wfile.flush()
|
||||
wlock = threading.Lock()
|
||||
|
||||
def write(s):
|
||||
with wlock:
|
||||
wfile.write(s.encode('utf-8'))
|
||||
wfile.flush()
|
||||
serve_lines(self.server.mcp, lambda: rfile.readline().decode('utf-8', 'replace'), write)
|
||||
|
||||
|
||||
class TCPServer(socketserver.ThreadingTCPServer):
|
||||
daemon_threads = True
|
||||
allow_reuse_address = False # Windows 上 SO_REUSEADDR 允许端口劫持
|
||||
|
||||
|
||||
def make_tcp(mcp, host, port, token):
|
||||
srv = TCPServer((host, port), _Handler)
|
||||
srv.mcp, srv.token = mcp, token
|
||||
return srv
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
ap = argparse.ArgumentParser(prog='lc_mcp.server')
|
||||
ap.add_argument('--stdio', action='store_true')
|
||||
ap.add_argument('--fake', action='store_true')
|
||||
ap.add_argument('--host', default='127.0.0.1')
|
||||
ap.add_argument('--port', type=int, default=DEFAULT_PORT)
|
||||
args = ap.parse_args(argv)
|
||||
mcp = build_server(fake=args.fake)
|
||||
if args.stdio:
|
||||
return run_stdio(mcp)
|
||||
if args.host not in ('127.0.0.1', '::1', 'localhost'):
|
||||
log('WARNING: 监听 %s 会把游戏控制权暴露到网络上, 建议走 SSH 隧道' % args.host)
|
||||
srv = make_tcp(mcp, args.host, args.port, load_token(create=True))
|
||||
log('listening on %s:%d (token: %s)' % (args.host, args.port, TOKEN_FILE))
|
||||
srv.serve_forever()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
+107
@@ -0,0 +1,107 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""长任务管理: 同一时刻最多一个 AALC 任务子进程。"""
|
||||
import glob
|
||||
import json
|
||||
import os
|
||||
import signal
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
|
||||
REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
IS_WIN = sys.platform.startswith('win')
|
||||
|
||||
|
||||
def default_argv(task, params):
|
||||
return [sys.executable, '-m', 'lc_mcp.aalc_task', task, json.dumps(params)]
|
||||
|
||||
|
||||
def tail(path, n):
|
||||
try:
|
||||
with open(path, 'rb') as f:
|
||||
f.seek(0, os.SEEK_END)
|
||||
f.seek(max(0, f.tell() - 64 * 1024))
|
||||
lines = f.read().decode('utf-8', 'replace').splitlines()
|
||||
return lines[-n:]
|
||||
except OSError:
|
||||
return []
|
||||
|
||||
|
||||
class TaskRunner:
|
||||
|
||||
def __init__(self, run_dir, argv_factory=default_argv, keep_logs=20):
|
||||
self.run_dir = os.path.abspath(run_dir)
|
||||
os.makedirs(self.run_dir, exist_ok=True)
|
||||
self.argv_factory = argv_factory
|
||||
self.keep_logs = keep_logs
|
||||
self._lock = threading.Lock()
|
||||
self.proc = None
|
||||
self.info = None
|
||||
|
||||
def running(self):
|
||||
return self.proc is not None and self.proc.poll() is None
|
||||
|
||||
def start(self, task, params):
|
||||
with self._lock:
|
||||
if self.running():
|
||||
return {'ok': False, 'error': '已有任务在跑: %s (先 stop_task)' % self.info['task']}
|
||||
stamp = time.strftime('%Y%m%d_%H%M%S')
|
||||
log_path = os.path.join(self.run_dir, 'task_%s_%s.log' % (stamp, task))
|
||||
env = dict(os.environ, PYTHONPATH=REPO + os.pathsep + os.environ.get('PYTHONPATH', ''),
|
||||
PYTHONIOENCODING='utf-8', PYTHONUNBUFFERED='1')
|
||||
kw = {'creationflags': subprocess.CREATE_NEW_PROCESS_GROUP} if IS_WIN else {'start_new_session': True}
|
||||
with open(log_path, 'wb') as logf:
|
||||
self.proc = subprocess.Popen(self.argv_factory(task, params), cwd=REPO, env=env,
|
||||
stdin=subprocess.DEVNULL, stdout=logf, stderr=subprocess.STDOUT, **kw)
|
||||
self.info = {'task': task, 'params': params, 'log': log_path.replace('\\', '/'),
|
||||
'started': time.time(), 'stopped_by_user': False}
|
||||
self._prune()
|
||||
return {'ok': True, 'task': task, 'pid': self.proc.pid, 'log': self.info['log']}
|
||||
|
||||
def status(self, lines=30):
|
||||
with self._lock:
|
||||
if self.info is None:
|
||||
return {'ok': True, 'state': 'idle'}
|
||||
code = self.proc.poll()
|
||||
if code is None:
|
||||
state = 'running'
|
||||
elif self.info['stopped_by_user']:
|
||||
state = 'stopped'
|
||||
else:
|
||||
state = 'succeeded' if code == 0 else 'failed'
|
||||
r = {'ok': True, 'state': state, 'task': self.info['task'], 'params': self.info['params'],
|
||||
'elapsed_s': int((self.info.get('ended') or time.time()) - self.info['started']),
|
||||
'log': self.info['log'], 'log_tail': tail(self.info['log'], max(1, min(int(lines), 200)))}
|
||||
if code is not None:
|
||||
r['exit_code'] = code
|
||||
self.info.setdefault('ended', time.time())
|
||||
return r
|
||||
|
||||
def stop(self):
|
||||
with self._lock:
|
||||
if not self.running():
|
||||
return {'ok': True, 'state': 'idle', 'note': '没有在跑的任务'}
|
||||
self.info['stopped_by_user'] = True
|
||||
pid = self.proc.pid
|
||||
if IS_WIN:
|
||||
subprocess.run(['taskkill', '/PID', str(pid), '/T', '/F'], capture_output=True)
|
||||
else:
|
||||
try:
|
||||
os.killpg(pid, signal.SIGKILL)
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
try:
|
||||
self.proc.wait(timeout=10)
|
||||
except subprocess.TimeoutExpired:
|
||||
return {'ok': False, 'error': '进程 %d 10 秒内没退出' % pid}
|
||||
self.info['ended'] = time.time()
|
||||
return {'ok': True, 'state': 'stopped', 'task': self.info['task']}
|
||||
|
||||
def _prune(self):
|
||||
logs = sorted(glob.glob(os.path.join(self.run_dir, 'task_*.log')), key=os.path.getmtime)
|
||||
for p in logs[:-self.keep_logs]:
|
||||
try:
|
||||
os.remove(p)
|
||||
except OSError:
|
||||
pass
|
||||
@@ -1,58 +0,0 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""平台抽象:截图 + 输入后端,按平台注册。
|
||||
core (MCP协议/OCR/语义工具) 平台无关。"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
|
||||
def detect_platform():
|
||||
p = os.environ.get('LCP_PLATFORM', '')
|
||||
if p:
|
||||
return p.lower()
|
||||
if sys.platform.startswith('win'):
|
||||
return 'win'
|
||||
if sys.platform == 'darwin':
|
||||
return 'mac'
|
||||
return 'linux'
|
||||
|
||||
|
||||
def get_screenshot_provider(platform=None):
|
||||
platform = platform or detect_platform()
|
||||
if platform == 'win':
|
||||
from .win import WinScreenshotProvider
|
||||
return WinScreenshotProvider()
|
||||
if platform == 'linux':
|
||||
from .linux import LinuxScreenshotProvider
|
||||
return LinuxScreenshotProvider()
|
||||
if platform == 'mac':
|
||||
from .mac import MacScreenshotProvider
|
||||
return MacScreenshotProvider()
|
||||
raise ValueError('no screenshot provider for %s' % platform)
|
||||
|
||||
|
||||
def get_input_backend(name=None, platform=None):
|
||||
platform = platform or detect_platform()
|
||||
name = (name or os.environ.get('INPUT_BACKEND', '') or
|
||||
{'win': 'sendinput', 'linux': 'xdotool', 'mac': 'cg'}[platform]).lower()
|
||||
if name == 'mock':
|
||||
from backends.mock import MockBackend
|
||||
return MockBackend()
|
||||
if platform == 'win':
|
||||
if name == 'sendinput':
|
||||
from backends.sendinput_win import SendInputBackend
|
||||
return SendInputBackend()
|
||||
if name == 'interception':
|
||||
from backends.interception_win import InterceptionBackend
|
||||
return InterceptionBackend()
|
||||
if platform == 'linux':
|
||||
if name == 'xdotool':
|
||||
from backends.xdotool_linux import XdotoolBackend
|
||||
return XdotoolBackend()
|
||||
if name == 'uinput':
|
||||
from backends.uinput_linux import UinputBackend
|
||||
return UinputBackend()
|
||||
if platform == 'mac' and name == 'cg':
|
||||
from backends.cg_mac import CGBackend
|
||||
return CGBackend()
|
||||
raise ValueError('no input backend %s for %s' % (name, platform))
|
||||
@@ -1,34 +0,0 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Linux 截图提供者:X11 (PIL ImageGrab / scrot 兜底)。
|
||||
Wayland 下 ImageGrab 不可用,需额外方案(见 README)。
|
||||
"""
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
|
||||
class LinuxScreenshotProvider:
|
||||
|
||||
def grab(self):
|
||||
# 1) PIL ImageGrab (X11)
|
||||
try:
|
||||
from PIL import ImageGrab
|
||||
img = ImageGrab.grab()
|
||||
path = os.path.join(tempfile.gettempdir(), 'lc_shot_%s.png' % time.strftime('%H%M%S'))
|
||||
img.save(path)
|
||||
return {'ok': True, 'path': path}
|
||||
except Exception:
|
||||
pass
|
||||
# 2) scrot / import (ImageMagick) / gnome-screenshot 兜底
|
||||
for cmd in (['scrot', '-z'], ['import', '-window', 'root'], ['gnome-screenshot', '-f']):
|
||||
if shutil.which(cmd[0]):
|
||||
path = os.path.join(tempfile.gettempdir(), 'lc_shot_%s.png' % time.strftime('%H%M%S'))
|
||||
try:
|
||||
subprocess.run(cmd + [path], check=True, timeout=20)
|
||||
if os.path.exists(path):
|
||||
return {'ok': True, 'path': path}
|
||||
except Exception:
|
||||
continue
|
||||
return {'ok': False, 'error': 'no working screenshot method on this Linux (need X11 + PIL/scrot)'}
|
||||
@@ -1,20 +0,0 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""macOS 截图提供者:screencapture CLI (系统自带)。"""
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
|
||||
class MacScreenshotProvider:
|
||||
|
||||
def grab(self):
|
||||
path = os.path.join(tempfile.gettempdir(), 'lc_shot_%s.png' % time.strftime('%H%M%S'))
|
||||
try:
|
||||
subprocess.run(['screencapture', '-x', path], check=True, timeout=20)
|
||||
if os.path.exists(path):
|
||||
return {'ok': True, 'path': path}
|
||||
except Exception as e:
|
||||
return {'ok': False, 'error': 'screencapture fail: %s' % e}
|
||||
return {'ok': False, 'error': 'no png produced'}
|
||||
@@ -1,122 +0,0 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Windows 截图提供者:交互计划任务(LogonType Interactive)。
|
||||
OpenSSH 服务会话(session 0)没有交互桌面,截图/前端输入必须在交互会话执行。
|
||||
|
||||
任务只注册一次(RunLevel Limited, 截图不需要管理员), 之后每次只改 job.ps1 + Start。
|
||||
"""
|
||||
import os
|
||||
import subprocess
|
||||
import time
|
||||
|
||||
USER = os.environ.get('USERNAME', 'yws-hhh')
|
||||
BASE = os.path.join(os.path.expanduser('~'), 'lc_mcp')
|
||||
# 默认放用户目录(只有本人可写); ProgramData 子目录对其他本地用户可写
|
||||
OUT_DIR = os.environ.get('LCP_SHOT_DIR') or os.path.join(BASE, 'shots')
|
||||
JOB_PS1 = os.path.join(BASE, 'job.ps1')
|
||||
RUN_VBS = os.path.join(BASE, 'run.vbs')
|
||||
TASK = 'LcMCP'
|
||||
|
||||
|
||||
def _ps_quote(s):
|
||||
"""PowerShell 单引号字面量。"""
|
||||
return "'" + str(s).replace("'", "''") + "'"
|
||||
|
||||
|
||||
def _vbs_content(job_ps1):
|
||||
# VBS 字符串里的双引号要写成 ""
|
||||
cmd = 'powershell.exe -NoProfile -ExecutionPolicy Bypass -File "%s"' % job_ps1
|
||||
return 'CreateObject("WScript.Shell").Run "%s", 0, False\n' % cmd.replace('"', '""')
|
||||
|
||||
|
||||
def _pshell(code):
|
||||
r = subprocess.run(['powershell.exe', '-NoProfile', '-ExecutionPolicy', 'Bypass', '-Command', code],
|
||||
capture_output=True, text=True, encoding='utf-8', errors='replace', timeout=60)
|
||||
return r.returncode, (r.stdout or ''), (r.stderr or '')
|
||||
|
||||
|
||||
def _register_task():
|
||||
"""注册(或覆盖)交互计划任务。进程生命周期内只做一次。"""
|
||||
reg = ("$A=New-ScheduledTaskAction -Execute 'wscript.exe' -Argument %s; "
|
||||
"$P=New-ScheduledTaskPrincipal -UserId %s -LogonType Interactive -RunLevel Limited; "
|
||||
"$S=New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries "
|
||||
"-ExecutionTimeLimit (New-TimeSpan -Minutes 2) -MultipleInstances Parallel; "
|
||||
"Register-ScheduledTask -TaskName %s -Action $A -Principal $P -Settings $S -Force | Out-Null; 'OK'"
|
||||
% (_ps_quote('"%s"' % RUN_VBS), _ps_quote(USER), _ps_quote(TASK)))
|
||||
rc, out, err = _pshell(reg)
|
||||
if 'OK' not in out:
|
||||
raise RuntimeError('register task fail: %s %s' % (out[-150:], err[-150:]))
|
||||
|
||||
|
||||
def _run_interactive(ps1_code, expect_file, timeout=15):
|
||||
# 带 BOM: Windows PowerShell 5.1 把无 BOM 的 .ps1 当 ANSI 读, 中文路径会乱码
|
||||
with open(JOB_PS1, 'w', encoding='utf-8-sig') as f:
|
||||
f.write('$ErrorActionPreference="Stop"\n' + ps1_code)
|
||||
try:
|
||||
if os.path.exists(expect_file):
|
||||
os.remove(expect_file)
|
||||
except Exception:
|
||||
pass
|
||||
rc, out, err = _pshell("Start-ScheduledTask -TaskName %s; 'OK'" % _ps_quote(TASK))
|
||||
if 'OK' not in out:
|
||||
return {"ok": False, "error": "task start fail: %s %s" % (out[-150:], err[-150:])}
|
||||
deadline = time.time() + timeout
|
||||
while time.time() < deadline:
|
||||
if os.path.exists(expect_file):
|
||||
try:
|
||||
with open(expect_file, 'r', encoding='utf-8-sig') as f:
|
||||
out = f.read().strip()
|
||||
except Exception:
|
||||
out = ''
|
||||
try:
|
||||
os.remove(expect_file)
|
||||
except Exception:
|
||||
pass
|
||||
return {"ok": True, "out": out}
|
||||
time.sleep(0.2)
|
||||
return {"ok": False, "error": "timeout waiting %s" % expect_file}
|
||||
|
||||
|
||||
# 截图前声明 DPI 感知, 否则缩放 != 100% 时 Bounds 是逻辑尺寸, 截出来是裁切的
|
||||
_SHOT_PS = r"""
|
||||
Add-Type -AssemblyName System.Windows.Forms,System.Drawing
|
||||
Add-Type -Namespace LcMcp -Name Dpi -MemberDefinition '[DllImport("user32.dll")] public static extern bool SetProcessDPIAware();'
|
||||
[void][LcMcp.Dpi]::SetProcessDPIAware()
|
||||
$img = %(img)s
|
||||
$done = %(done)s
|
||||
try {
|
||||
$b=[System.Windows.Forms.Screen]::PrimaryScreen.Bounds
|
||||
$bmp=New-Object System.Drawing.Bitmap $b.Width,$b.Height
|
||||
$g=[System.Drawing.Graphics]::FromImage($bmp)
|
||||
$g.CopyFromScreen($b.X,$b.Y,0,0,$b.Size)
|
||||
$bmp.Save($img, [System.Drawing.Imaging.ImageFormat]::Png)
|
||||
$g.Dispose(); $bmp.Dispose()
|
||||
'DONE' | Set-Content -Encoding UTF8 $done
|
||||
} catch {
|
||||
('ERR ' + $_.Exception.Message) | Set-Content -Encoding UTF8 $done
|
||||
}
|
||||
"""
|
||||
|
||||
|
||||
class WinScreenshotProvider:
|
||||
|
||||
def __init__(self):
|
||||
os.makedirs(BASE, exist_ok=True)
|
||||
os.makedirs(OUT_DIR, exist_ok=True)
|
||||
# WSH 只认 ANSI / UTF-16(BOM), UTF-8 下中文路径会坏
|
||||
with open(RUN_VBS, 'w', encoding='utf-16') as f:
|
||||
f.write(_vbs_content(JOB_PS1))
|
||||
_register_task()
|
||||
|
||||
def grab(self):
|
||||
stamp = time.strftime('%Y%m%d_%H%M%S') + '_%03d' % (int(time.time() * 1000) % 1000)
|
||||
img_path = os.path.join(OUT_DIR, 'shot_%s.png' % stamp)
|
||||
done = os.path.join(OUT_DIR, 'done_%s.txt' % stamp)
|
||||
ps = _SHOT_PS % {'img': _ps_quote(img_path), 'done': _ps_quote(done)}
|
||||
r = _run_interactive(ps, done, timeout=20)
|
||||
if not r.get('ok'):
|
||||
return {'ok': False, 'error': r.get('error', 'screenshot fail')}
|
||||
if r.get('out', '').startswith('ERR'):
|
||||
return {'ok': False, 'error': r['out']}
|
||||
if not os.path.exists(img_path):
|
||||
return {'ok': False, 'error': 'png not produced'}
|
||||
return {'ok': True, 'path': img_path.replace('\\', '/')}
|
||||
@@ -1,75 +0,0 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""本地 OCR:Windows.Media.Ocr(winsdk),零远端依赖。
|
||||
只支持 Windows;Linux/mac 上 import winsdk 会失败 -> 用 remote 后端。
|
||||
"""
|
||||
import asyncio
|
||||
|
||||
|
||||
def available():
|
||||
try:
|
||||
from winsdk.windows.media.ocr import OcrEngine # noqa: F401
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def _engine():
|
||||
from winsdk.windows.media.ocr import OcrEngine
|
||||
from winsdk.windows.globalization import Language
|
||||
try:
|
||||
eng = OcrEngine.try_create_from_language(Language('zh-Hans'))
|
||||
if eng:
|
||||
return eng
|
||||
except Exception:
|
||||
pass
|
||||
return OcrEngine.try_create_from_user_profile_languages()
|
||||
|
||||
|
||||
async def _ocr_async(path):
|
||||
from winsdk.windows.media.ocr import OcrEngine # noqa: F401
|
||||
from winsdk.windows.storage import StorageFile, FileAccessMode
|
||||
from winsdk.windows.graphics.imaging import BitmapDecoder
|
||||
|
||||
file = await StorageFile.get_file_from_path_async(path)
|
||||
stream = await file.open_async(FileAccessMode.READ)
|
||||
decoder = await BitmapDecoder.create_async(stream)
|
||||
bitmap = await decoder.get_software_bitmap_async()
|
||||
|
||||
eng = _engine()
|
||||
if eng is None:
|
||||
return None, 'no OCR engine (language pack missing?)'
|
||||
result = await eng.recognize_async(bitmap)
|
||||
|
||||
items = []
|
||||
for line in result.lines:
|
||||
words = line.words
|
||||
if not words:
|
||||
continue
|
||||
text = ' '.join(w.text for w in words)
|
||||
w0 = words[0].bounding_rect
|
||||
x0, y0 = w0.x, w0.y
|
||||
x1, y1 = w0.x + w0.width, w0.y + w0.height
|
||||
for w in words[1:]:
|
||||
r = w.bounding_rect
|
||||
if r.x < x0:
|
||||
x0 = r.x
|
||||
if r.y < y0:
|
||||
y0 = r.y
|
||||
if r.x + r.width > x1:
|
||||
x1 = r.x + r.width
|
||||
if r.y + r.height > y1:
|
||||
y1 = r.y + r.height
|
||||
items.append({
|
||||
'text': text,
|
||||
'x': int(x0), 'y': int(y0),
|
||||
'w': int(x1 - x0), 'h': int(y1 - y0),
|
||||
})
|
||||
return items, None
|
||||
|
||||
|
||||
def ocr_image(path):
|
||||
"""同步接口。返回 (items, None) 或 (None, error)。"""
|
||||
try:
|
||||
return asyncio.run(_ocr_async(path))
|
||||
except Exception as e:
|
||||
return None, 'local ocr fail: %s' % e
|
||||
@@ -0,0 +1,38 @@
|
||||
# 注册登录时启动的 lc-mcp 常驻服务 (交互会话, 能看到游戏窗口)。
|
||||
# 用法 (在要跑游戏的那个账号下, 普通 PowerShell 即可):
|
||||
# powershell -ExecutionPolicy Bypass -File scripts\install_logon_task.ps1
|
||||
# 可选: -Python <python.exe 路径> 默认用 vendor\aalc\.venv (uv sync 生成)
|
||||
# -Elevated 游戏以管理员运行时才需要(UIPI 会挡住低权限进程的输入)
|
||||
# -Uninstall
|
||||
param(
|
||||
[string]$Python = "",
|
||||
[switch]$Elevated,
|
||||
[switch]$Uninstall
|
||||
)
|
||||
$ErrorActionPreference = "Stop"
|
||||
$TaskName = "LcMCPServer"
|
||||
$Repo = Split-Path -Parent $PSScriptRoot
|
||||
|
||||
if ($Uninstall) {
|
||||
Unregister-ScheduledTask -TaskName $TaskName -Confirm:$false -ErrorAction SilentlyContinue
|
||||
Write-Output "removed $TaskName"
|
||||
exit 0
|
||||
}
|
||||
|
||||
if (-not $Python) { $Python = Join-Path $Repo "vendor\aalc\.venv\Scripts\pythonw.exe" }
|
||||
if (-not (Test-Path $Python)) {
|
||||
throw "找不到 Python: $Python`n先在 vendor\aalc 里执行 uv sync, 或用 -Python 指定 AALC 依赖齐全的解释器"
|
||||
}
|
||||
|
||||
$user = "$env:USERDOMAIN\$env:USERNAME"
|
||||
$action = New-ScheduledTaskAction -Execute $Python -Argument "-m lc_mcp.server" -WorkingDirectory $Repo
|
||||
$trigger = New-ScheduledTaskTrigger -AtLogOn -User $user
|
||||
$level = if ($Elevated) { "Highest" } else { "Limited" }
|
||||
$principal = New-ScheduledTaskPrincipal -UserId $user -LogonType Interactive -RunLevel $level
|
||||
$settings = New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries `
|
||||
-ExecutionTimeLimit ([TimeSpan]::Zero) -RestartCount 3 -RestartInterval (New-TimeSpan -Minutes 1) `
|
||||
-MultipleInstances IgnoreNew
|
||||
Register-ScheduledTask -TaskName $TaskName -Action $action -Trigger $trigger -Principal $principal `
|
||||
-Settings $settings -Force | Out-Null
|
||||
Start-ScheduledTask -TaskName $TaskName
|
||||
Write-Output "registered + started $TaskName ($level) -> $Python -m lc_mcp.server"
|
||||
@@ -1,63 +0,0 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""后端/协议测试(本机直接跑,无需 Windows)。"""
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
sys.path.insert(0, HERE)
|
||||
|
||||
from lcplatform import detect_platform, get_input_backend # noqa: E402
|
||||
|
||||
|
||||
def test_mock_backend():
|
||||
b = get_input_backend('mock')
|
||||
assert b.click(100, 200)['ok']
|
||||
assert b.move(10, 10)['ok']
|
||||
assert b.keypress('p')['ok']
|
||||
assert b.type_text('ok')['ok']
|
||||
acts = [x['act'] for x in b.log]
|
||||
assert acts == ['click', 'move', 'keypress', 'type_text'], acts
|
||||
print('mock backend: OK', acts)
|
||||
|
||||
|
||||
def test_scancode_table():
|
||||
from backends.interception_win import SCAN, UPPER_SCAN
|
||||
assert SCAN['a'] == 0x1E and SCAN['p'] == 0x19 and SCAN['z'] == 0x2C
|
||||
assert SCAN['enter'] == 0x1C and SCAN['esc'] == 0x01
|
||||
assert UPPER_SCAN['!'] == 0x02
|
||||
# 连续无缺失
|
||||
for ch in 'abcdefghijklmnopqrstuvwxyz0123456789':
|
||||
assert ch in SCAN, 'missing %s' % ch
|
||||
print('scancode table: OK')
|
||||
|
||||
|
||||
def test_mcp_roundtrip_mock():
|
||||
env = dict(os.environ, INPUT_BACKEND='mock')
|
||||
p = subprocess.Popen([sys.executable, os.path.join(HERE, 'lc_mcp.py')],
|
||||
stdin=subprocess.PIPE, stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE, text=True, encoding='utf-8',
|
||||
errors='replace', env=env)
|
||||
def call(obj):
|
||||
p.stdin.write(json.dumps(obj) + '\n')
|
||||
p.stdin.flush()
|
||||
return json.loads(p.stdout.readline())
|
||||
r = call({'jsonrpc': '2.0', 'id': 1, 'method': 'initialize', 'params': {}})
|
||||
assert r['result']['serverInfo']['name'] == 'lc-mcp'
|
||||
r = call({'jsonrpc': '2.0', 'id': 2, 'method': 'tools/list', 'params': {}})
|
||||
names = [t['name'] for t in r['result']['tools']]
|
||||
assert 'get_state' in names and 'click_text' in names and 'wait_for_text' in names and 'wait_for_text_gone' in names, names
|
||||
assert 'screenshot' in names and 'keypress' in names, names
|
||||
r = call({'jsonrpc': '2.0', 'id': 3, 'method': 'tools/call',
|
||||
'params': {'name': 'move', 'arguments': {'x': 5, 'y': 5}}})
|
||||
assert 'mock' in r['result']['content'][0]['text']
|
||||
p.kill()
|
||||
print('mcp roundtrip(mock): OK', names)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
test_mock_backend()
|
||||
test_scancode_table()
|
||||
test_mcp_roundtrip_mock()
|
||||
print('ALL TESTS PASSED')
|
||||
@@ -0,0 +1,217 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""不需要 Windows/AALC 的测试: python -m unittest discover -s tests"""
|
||||
import json
|
||||
import os
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
sys.path.insert(0, REPO)
|
||||
|
||||
from lc_mcp import server as srv_mod # noqa: E402
|
||||
from lc_mcp.engine import find_hits # noqa: E402
|
||||
from lc_mcp.fake_engine import FakeEngine # noqa: E402
|
||||
from lc_mcp.protocol import Server, TOOLS # noqa: E402
|
||||
from lc_mcp.tasks import TaskRunner # noqa: E402
|
||||
|
||||
START = {'text': '开始', 'x': 100, 'y': 200, 'w': 40, 'h': 20}
|
||||
|
||||
|
||||
def call(server, tool, **args):
|
||||
resp = server.handle({'jsonrpc': '2.0', 'id': 1, 'method': 'tools/call',
|
||||
'params': {'name': tool, 'arguments': args}})
|
||||
res = resp['result']
|
||||
data = json.loads(res['content'][0]['text'])
|
||||
assert res['isError'] == (not data.get('ok', False)), res
|
||||
return data
|
||||
|
||||
|
||||
def sleeper_argv(seconds):
|
||||
code = 'import sys,time; print("child", sys.argv[1:], flush=True); time.sleep(%s)' % seconds
|
||||
return lambda task, params: [sys.executable, '-c', code, task, json.dumps(params)]
|
||||
|
||||
|
||||
class MatchTest(unittest.TestCase):
|
||||
|
||||
def test_ignores_spaces_and_case(self):
|
||||
items = [{'text': 'Hong Lu', 'x': 0, 'y': 0, 'w': 10, 'h': 10}]
|
||||
self.assertEqual(find_hits('honglu', items)[0]['x'], 5)
|
||||
|
||||
def test_exact_first(self):
|
||||
items = [{'text': '开始战斗', 'x': 0, 'y': 0, 'w': 10, 'h': 10}, START]
|
||||
self.assertEqual(find_hits('开始', items)[0]['text'], '开始')
|
||||
|
||||
def test_empty_matches_nothing(self):
|
||||
self.assertEqual(find_hits(' ', [START]), [])
|
||||
|
||||
|
||||
class ProtocolTest(unittest.TestCase):
|
||||
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.tasks = TaskRunner(self.tmp.name, argv_factory=sleeper_argv(5))
|
||||
self.engine = FakeEngine(frames=[[START]], images={'battle/turn_assets.png': (50, 60)})
|
||||
self.s = Server(self.engine, self.tasks)
|
||||
|
||||
def tearDown(self):
|
||||
self.tasks.stop()
|
||||
self.tmp.cleanup()
|
||||
|
||||
def test_initialize_and_list(self):
|
||||
r = self.s.handle({'jsonrpc': '2.0', 'id': 7, 'method': 'initialize', 'params': {}})
|
||||
self.assertEqual(r['id'], 7)
|
||||
names = [t['name'] for t in self.s.handle({'jsonrpc': '2.0', 'id': 8, 'method': 'tools/list'})['result']['tools']]
|
||||
self.assertEqual(names, [t['name'] for t in TOOLS])
|
||||
self.assertIsNone(self.s.handle({'jsonrpc': '2.0', 'method': 'notifications/initialized'}))
|
||||
self.assertEqual(self.s.handle({'jsonrpc': '2.0', 'id': 9, 'method': 'nope'})['error']['code'], -32601)
|
||||
|
||||
def test_click_text(self):
|
||||
r = call(self.s, 'click_text', text='开 始')
|
||||
self.assertTrue(r['ok'])
|
||||
self.assertEqual(self.engine.actions, [('click', 120, 210, 1)])
|
||||
self.assertFalse(call(self.s, 'click_text', text='不存在')['ok'])
|
||||
self.assertFalse(call(self.s, 'click_text', text='开始', index=3)['ok'])
|
||||
self.assertFalse(call(self.s, 'click_text', text='')['ok'])
|
||||
|
||||
def test_bad_arguments_are_tool_errors(self):
|
||||
r = call(self.s, 'click', x='abc', y=1)
|
||||
self.assertIn('bad arguments', r['error'])
|
||||
self.assertFalse(call(self.s, 'click')['ok'])
|
||||
|
||||
def test_images(self):
|
||||
self.assertEqual(call(self.s, 'find_image', name='battle/turn_assets.png')['x'], 50)
|
||||
r = call(self.s, 'find_image', name='home/x.png')
|
||||
self.assertTrue(r['ok'])
|
||||
self.assertFalse(r['found'])
|
||||
self.assertFalse(call(self.s, 'click_image', name='home/x.png')['ok'])
|
||||
call(self.s, 'click_image', name='battle/turn_assets.png')
|
||||
self.assertEqual(self.engine.actions[-1], ('click', 50, 60, 1))
|
||||
self.assertEqual(call(self.s, 'list_images', prefix='battle/')['count'], 1)
|
||||
|
||||
def test_wait_for_text(self):
|
||||
self.engine.frames = [[], [], [START]]
|
||||
with mock.patch('lc_mcp.protocol.WAIT_INTERVAL', 0.01):
|
||||
r = call(self.s, 'wait_for_text', text='开始', timeout=2)
|
||||
self.assertTrue(r['ok'])
|
||||
self.engine.frames = [[START]]
|
||||
r = call(self.s, 'wait_for_text_gone', text='开始', timeout=0.05)
|
||||
self.assertFalse(r['ok'])
|
||||
self.assertIn('timeout', r['error'])
|
||||
|
||||
def test_task_lifecycle_blocks_manual_input(self):
|
||||
self.assertEqual(call(self.s, 'task_status')['state'], 'idle')
|
||||
r = call(self.s, 'run_task', task='mirror', count=2, exp=9) # exp 不属于 mirror, 忽略
|
||||
self.assertTrue(r['ok'], r)
|
||||
self.assertFalse(call(self.s, 'run_task', task='daily')['ok'])
|
||||
self.assertIn('stop_task', call(self.s, 'click', x=1, y=1)['error'])
|
||||
self.assertTrue(call(self.s, 'get_state')['ok']) # 只读工具仍可用
|
||||
time.sleep(0.5)
|
||||
st = call(self.s, 'task_status')
|
||||
self.assertEqual(st['state'], 'running')
|
||||
self.assertEqual(st['params'], {'count': 2})
|
||||
self.assertIn("child ['mirror', '{\"count\": 2}']", '\n'.join(st['log_tail']))
|
||||
self.assertEqual(call(self.s, 'stop_task')['state'], 'stopped')
|
||||
self.assertEqual(call(self.s, 'task_status')['state'], 'stopped')
|
||||
self.assertTrue(call(self.s, 'click', x=1, y=1)['ok'])
|
||||
self.assertFalse(call(self.s, 'run_task', task='bogus')['ok'])
|
||||
|
||||
def test_task_exit_codes(self):
|
||||
self.tasks.argv_factory = lambda t, p: [sys.executable, '-c', 'import sys; sys.exit(3)']
|
||||
call(self.s, 'run_task', task='reward')
|
||||
self.tasks.proc.wait(5)
|
||||
st = call(self.s, 'task_status')
|
||||
self.assertEqual((st['state'], st['exit_code']), ('failed', 3))
|
||||
|
||||
def test_engine_unavailable(self):
|
||||
s = Server(None, self.tasks, engine_error='no windows')
|
||||
self.assertIn('no windows', call(s, 'get_state')['error'])
|
||||
self.assertEqual(call(s, 'task_status')['state'], 'idle')
|
||||
|
||||
|
||||
class AalcTaskArgsTest(unittest.TestCase):
|
||||
|
||||
def test_rejects_unknown_params(self):
|
||||
from lc_mcp import aalc_task
|
||||
self.assertEqual(aalc_task.main(['x', 'mirror', '{"exp": 1}']), 2)
|
||||
self.assertEqual(aalc_task.main(['x', 'nope']), 2)
|
||||
|
||||
|
||||
def free_port():
|
||||
with socket.socket() as s:
|
||||
s.bind(('127.0.0.1', 0))
|
||||
return s.getsockname()[1]
|
||||
|
||||
|
||||
class TransportTest(unittest.TestCase):
|
||||
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.token_file = os.path.join(self.tmp.name, 'token')
|
||||
with open(self.token_file, 'w') as f:
|
||||
f.write('secret-token')
|
||||
self.port = free_port()
|
||||
mcp = Server(FakeEngine(frames=[[START]]), TaskRunner(self.tmp.name))
|
||||
self.tcp = srv_mod.make_tcp(mcp, '127.0.0.1', self.port, 'secret-token')
|
||||
threading.Thread(target=self.tcp.serve_forever, daemon=True).start()
|
||||
|
||||
def tearDown(self):
|
||||
self.tcp.shutdown()
|
||||
self.tcp.server_close()
|
||||
self.tmp.cleanup()
|
||||
|
||||
def test_wrong_token_rejected(self):
|
||||
with socket.create_connection(('127.0.0.1', self.port)) as s:
|
||||
s.sendall(b'AUTH wrong\n')
|
||||
self.assertEqual(s.makefile('rb').readline().strip(), b'DENIED')
|
||||
|
||||
def test_bridge_end_to_end(self):
|
||||
env = dict(os.environ, LCP_PORT=str(self.port), HOME=self.tmp.name, USERPROFILE=self.tmp.name)
|
||||
os.makedirs(os.path.join(self.tmp.name, 'lc_mcp'))
|
||||
os.replace(self.token_file, os.path.join(self.tmp.name, 'lc_mcp', 'token'))
|
||||
lines = [
|
||||
'{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}',
|
||||
'',
|
||||
'not json',
|
||||
'{"jsonrpc":"2.0","method":"notifications/initialized"}',
|
||||
'{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"click_text","arguments":{"text":"开始"}}}',
|
||||
]
|
||||
p = subprocess.run([sys.executable, os.path.join(REPO, 'lc_mcp', 'bridge.py')],
|
||||
input='\n'.join(lines) + '\n', capture_output=True, text=True,
|
||||
encoding='utf-8', env=env, timeout=20)
|
||||
out = [json.loads(x) for x in p.stdout.splitlines()]
|
||||
self.assertEqual([o.get('id') for o in out], [1, None, 2], p.stdout + p.stderr)
|
||||
self.assertEqual(out[1]['error']['code'], -32700)
|
||||
self.assertFalse(out[2]['result']['isError'])
|
||||
|
||||
def test_bridge_offline(self):
|
||||
env = dict(os.environ, LCP_PORT=str(free_port()), HOME=self.tmp.name, USERPROFILE=self.tmp.name)
|
||||
os.makedirs(os.path.join(self.tmp.name, 'lc_mcp'))
|
||||
os.replace(self.token_file, os.path.join(self.tmp.name, 'lc_mcp', 'token'))
|
||||
req = ('{"jsonrpc":"2.0","id":1,"method":"initialize"}\n'
|
||||
'{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"get_state"}}\n')
|
||||
p = subprocess.run([sys.executable, os.path.join(REPO, 'lc_mcp', 'bridge.py')], input=req,
|
||||
capture_output=True, text=True, encoding='utf-8', env=env, timeout=20)
|
||||
out = [json.loads(x) for x in p.stdout.splitlines()]
|
||||
self.assertIn('offline', out[0]['result']['serverInfo']['name'])
|
||||
self.assertIn('未运行', out[1]['error']['message'])
|
||||
|
||||
def test_stdio_server(self):
|
||||
req = ('{"jsonrpc":"2.0","id":1,"method":"ping"}\n'
|
||||
'{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"get_state"}}\n')
|
||||
env = dict(os.environ, HOME=self.tmp.name, USERPROFILE=self.tmp.name)
|
||||
p = subprocess.run([sys.executable, '-m', 'lc_mcp.server', '--stdio', '--fake'], cwd=REPO, input=req,
|
||||
capture_output=True, text=True, encoding='utf-8', env=env, timeout=20)
|
||||
out = [json.loads(x) for x in p.stdout.splitlines()]
|
||||
self.assertEqual(out[0]['result'], {})
|
||||
state = json.loads(out[1]['result']['content'][0]['text'])
|
||||
self.assertEqual((state['ok'], state['items']), (True, []))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
+1
Submodule vendor/aalc added at 7452fad5a4
-87
@@ -1,87 +0,0 @@
|
||||
# win_ocr.ps1 - Windows 自带 OCR (Windows.Media.Ocr) 识别本地图片
|
||||
# 用法: powershell -NoProfile -ExecutionPolicy Bypass -File win_ocr.ps1 -Image <png> [-Lang zh-Hans]
|
||||
# 输出: UTF-8 JSON => {"ok":true,"items":[{"text","x","y","w","h"},...]}
|
||||
param(
|
||||
[Parameter(Mandatory=$true)][string]$Image,
|
||||
[string]$Lang = "zh-Hans"
|
||||
)
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
$path = (Resolve-Path $Image -ErrorAction SilentlyContinue)
|
||||
if (-not $path) { Write-Output '{"ok":false,"error":"image not found"}'; exit 1 }
|
||||
$path = $path.Path
|
||||
|
||||
# ---- 正确加载 WinRT 类型 ----
|
||||
try {
|
||||
Add-Type -AssemblyName System.Runtime.WindowsRuntime
|
||||
[void][System.WindowsRuntimeSystemExtensions, System.Runtime.WindowsRuntime, ContentType=WindowsRuntime]
|
||||
} catch {
|
||||
Write-Output '{"ok":false,"error":"winrt load fail"}'; exit 1
|
||||
}
|
||||
|
||||
$asTaskGeneric = ([System.WindowsRuntimeSystemExtensions].GetMethods() | Where-Object {
|
||||
$_.Name -eq 'AsTask' -and $_.GetParameters().Count -eq 1 -and
|
||||
$_.GetParameters()[0].ParameterType.Name -eq 'IAsyncOperation`1'
|
||||
})[0]
|
||||
|
||||
Function Await($WinRtTask, $ResultType) {
|
||||
$asTask = $asTaskGeneric.MakeGenericMethod($ResultType)
|
||||
$netTask = $asTask.Invoke($null, @($WinRtTask))
|
||||
$netTask.Wait(-1) | Out-Null
|
||||
$netTask.Result
|
||||
}
|
||||
|
||||
function Write-Json($obj) {
|
||||
[Console]::OutputEncoding = [Text.Encoding]::UTF8
|
||||
Write-Output ($obj | ConvertTo-Json -Compress -Depth 5)
|
||||
}
|
||||
|
||||
try {
|
||||
$file = [Windows.Storage.StorageFile, Windows.Storage, ContentType=WindowsRuntime]::GetFileFromPathAsync($path)
|
||||
$storageFile = Await $file ([Windows.Storage.StorageFile])
|
||||
|
||||
$stream = Await ($storageFile.OpenAsync([Windows.Storage.FileAccessMode]::Read)) ([Windows.Storage.Streams.IRandomAccessStream])
|
||||
|
||||
$decoder = Await ([Windows.Graphics.Imaging.BitmapDecoder, Windows.Graphics.Imaging, ContentType=WindowsRuntime]::CreateAsync($stream)) ([Windows.Graphics.Imaging.BitmapDecoder])
|
||||
$bitmap = Await ($decoder.GetSoftwareBitmapAsync()) ([Windows.Graphics.Imaging.SoftwareBitmap])
|
||||
|
||||
$engine = $null
|
||||
try {
|
||||
$langObj = New-Object Windows.Globalization.Language $Lang
|
||||
$engine = [Windows.Media.Ocr.OcrEngine, Windows.Media.Ocr, ContentType=WindowsRuntime]::TryCreateFromLanguage($langObj)
|
||||
} catch { $engine = $null }
|
||||
if ($engine -eq $null) {
|
||||
$engine = [Windows.Media.Ocr.OcrEngine, Windows.Media.Ocr, ContentType=WindowsRuntime]::TryCreateFromUserProfileLanguages()
|
||||
}
|
||||
if ($engine -eq $null) {
|
||||
Write-Json @{ ok = $false; error = 'no OCR engine (need language pack?)' }
|
||||
exit 1
|
||||
}
|
||||
|
||||
$result = Await ($engine.RecognizeAsync($bitmap)) ([Windows.Media.Ocr.OcrResult])
|
||||
|
||||
$items = @()
|
||||
foreach ($line in $result.Lines) {
|
||||
if ($line.Words.Count -eq 0) { continue }
|
||||
$t = ($line.Words | ForEach-Object { $_.Text }) -join " "
|
||||
$minX = [math]::Floor($line.Words[0].BoundingRect.X)
|
||||
$minY = [math]::Floor($line.Words[0].BoundingRect.Y)
|
||||
$maxX = [math]::Ceiling($line.Words[0].BoundingRect.X + $line.Words[0].BoundingRect.Width)
|
||||
$maxY = [math]::Ceiling($line.Words[0].BoundingRect.Y + $line.Words[0].BoundingRect.Height)
|
||||
foreach ($word in $line.Words) {
|
||||
$r = $word.BoundingRect
|
||||
if ($r.X -lt $minX) { $minX = [math]::Floor($r.X) }
|
||||
if ($r.Y -lt $minY) { $minY = [math]::Floor($r.Y) }
|
||||
if (($r.X + $r.Width) -gt $maxX) { $maxX = [math]::Ceiling($r.X + $r.Width) }
|
||||
if (($r.Y + $r.Height) -gt $maxY) { $maxY = [math]::Ceiling($r.Y + $r.Height) }
|
||||
}
|
||||
$items += [PSCustomObject]@{
|
||||
text = $t
|
||||
x = $minX; y = $minY; w = ($maxX - $minX); h = ($maxY - $minY)
|
||||
}
|
||||
}
|
||||
Write-Json @{ ok = $true; items = $items }
|
||||
} catch {
|
||||
Write-Json @{ ok = $false; error = $_.Exception.Message }
|
||||
exit 1
|
||||
}
|
||||
Reference in New Issue
Block a user