无进展兜底: 40 分钟没有「开始执行/结束执行」就结束任务(I-18)

Browse Source
按 logs-raw 查清两次 5 小时多的空转, 都是上游循环没有上限、不报错、也不再打里程碑日志:
- 11-21 14:01 起 5h20m: 游戏改版后驱动菜单的镜牢入口 home/mirror_dungeons_assets.png 最高 0.41
  (11-18 之前每轮 1.00), 大厅 <-> 驱动来回点 2500 多轮; 上游 V1.6.2 c0e4926 换的正是这张图
- 11-17 00:45 起 5h40m: 全员阵亡后 Mirror.re_start() 的 while True, 画面 5.5 小时一帧没变

install_progress_watchdog: AALC logger 上挂 handler 记最后一条「开始执行/结束执行」,
任务主线程的 match_template / take_screenshot 里检查, 超时抛 TaskAbort(退出码 4, finally 照常写回配置),
记 [ERROR] 和 decisions progress/stall。LCP_PROGRESS_STALL_MIN 可调, 0 关闭。
20 份原始日志回放: 17 轮正常运行最长空档约 20 分钟、不会误停; 三次卡住都在最后一条之后 40 分钟停下。

单测: 779 passed(新增 test_progress_watchdog 10 例)。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
yws
2026-11-22 01:53:50 +00:00
co-authored by Claude Opus 5.5
parent 0fd5b66d80
commit 08f86dc932
6 changed files with 266 additions and 7 deletions
+91
View File
@@ -2115,6 +2115,97 @@ def install_image_error_guard(log=print, clock=time.monotonic,
log('[lc_mcp] patch: 截图识别连续出错 %d 秒(至少 %d 次)即结束任务' % (span, limit)) log('[lc_mcp] patch: 截图识别连续出错 %d 秒(至少 %d 次)即结束任务' % (span, limit))
return original return original
# ---------------------------------------------------------------- 长时间没有进展就结束
# AALC 的流程函数都套着 begin_and_finish_time_log, 进出各打一条「开始执行 X」「结束执行 X」(INFO)。
# 认不出画面时它会在某个没有上限的循环里一直转, 不报错也不再打这两种日志
# (feedback/runs/2026-11-15_2026-11-21/logs-raw):
# - 11-21 14:01 起 5 小时 20 分: 游戏界面改版后驱动菜单里的镜牢入口(home/mirror_dungeons_assets.png)
# 最高只匹配到 0.41, 大厅 ↔ 驱动来回点了 2500 多轮(上游 V1.6.2 换了这张图);
# - 11-17 00:45 起 5 小时 40 分: 全员阵亡后 Mirror.re_start() 的 while True, 画面 5.5 小时一帧没变;
# - 11-16 04:25 起 3 小时 30 分: 窗口没了后的识图出错风暴(现在截图熔断 30 秒就停)。
# 正常运行里两条之间最长的是一场战斗, 20 个 run 里最长 1184 秒(约 20 分钟); 默认 40 分钟没有新的一条就结束任务。
# 检查放在任务主线程每帧都会走的 match_template / take_screenshot 里, 抛 TaskAbort, finally 照常写回配置。
PROGRESS_STALL_ENV = 'LCP_PROGRESS_STALL_MIN' # 分钟; 0 = 不装
PROGRESS_STALL_MINUTES = 40
PROGRESS_MARKERS = ('开始执行', '结束执行')
class _ProgressHandler(logging.Handler):
"""AALC 日志里每出现一条「开始执行 / 结束执行」就记下时间。"""
def __init__(self, state, clock):
super().__init__(logging.INFO)
self.state = state
self.clock = clock
def emit(self, record):
try:
message = record.getMessage()
except Exception:
return
if message.startswith(PROGRESS_MARKERS):
self.state['last'], self.state['what'] = self.clock(), message.split(' 耗时', 1)[0]
def install_progress_watchdog(log=print, env=None, clock=time.monotonic):
"""长时间没有「开始执行 / 结束执行」就结束任务。返回挂在 AALC logger 上的 handler, 未安装返回 None。"""
env = os.environ if env is None else env
setting = (env.get(PROGRESS_STALL_ENV) or '').strip()
try:
minutes = float(setting) if setting else PROGRESS_STALL_MINUTES
except ValueError:
log('[lc_mcp] %s=%r 不是数字, 按默认 %d 分钟' % (PROGRESS_STALL_ENV, setting, PROGRESS_STALL_MINUTES))
minutes = PROGRESS_STALL_MINUTES
if minutes <= 0:
log('[lc_mcp] 无进展兜底已关闭(%s=%s)' % (PROGRESS_STALL_ENV, setting))
return None
try:
from utils.image_utils import ImageUtils
from module.automation import auto
except Exception as e:
log('[lc_mcp] 无进展兜底跳过: 无法导入 %s' % e)
return None
real_match = getattr(ImageUtils, 'match_template', None)
real_shot = getattr(auto, 'take_screenshot', None)
if real_match is None or real_shot is None:
log('[lc_mcp] 无进展兜底跳过: 上游没有 ImageUtils.match_template / auto.take_screenshot')
return None
owner = threading.get_ident()
limit = minutes * 60
state = {'last': clock(), 'what': '(任务开始后还没有)', 'fired': False}
def check():
if threading.get_ident() != owner:
return # 重试监控线程里抛只会打死那条线程
idle = clock() - state['last']
if idle < limit:
return
if not state['fired']:
state['fired'] = True
log('[ERROR] [lc_mcp] %d 分钟没有新的「开始执行/结束执行」(最后一条: %s), 判定卡死, 结束任务'
% (idle // 60, state['what']))
decisions.record('progress', None, 'stall', minutes=int(idle // 60), last=state['what'])
raise TaskAbort('%d 分钟没有进展(最后一条: %s), AALC 可能在认不出的画面上空转'
% (idle // 60, state['what']))
def match_template(*args, **kwargs):
check()
return real_match(*args, **kwargs)
def take_screenshot(*args, **kwargs):
check()
return real_shot(*args, **kwargs)
handler = _ProgressHandler(state, clock)
logging.getLogger('AALC').addHandler(handler)
ImageUtils.match_template = staticmethod(match_template)
auto.take_screenshot = take_screenshot
log('[lc_mcp] patch: %g 分钟没有「开始执行/结束执行」就结束任务' % minutes)
return handler
def install_gift_confirm_guard(log=print, clock=time.monotonic): def install_gift_confirm_guard(log=print, clock=time.monotonic):
"""「获得E.G.O饰品」确认点了不关时换落点, 再不行就结束任务。返回原 click_element, 未安装返回 None。 """「获得E.G.O饰品」确认点了不关时换落点, 再不行就结束任务。返回原 click_element, 未安装返回 None。
+3 -2
View File
@@ -14,8 +14,8 @@ from .aalc_engine import bootstrap
from .aalc_patches import (TaskAbort, WindowGuard, driver_click_tasks, headless_hdr_warning, from .aalc_patches import (TaskAbort, WindowGuard, driver_click_tasks, headless_hdr_warning,
install_back_button_result, install_default_battle_team, install_event_choices, install_back_button_result, install_default_battle_team, install_event_choices,
install_exact_team_match, install_gift_confirm_guard, install_gift_preference, install_exact_team_match, install_gift_confirm_guard, install_gift_preference,
install_image_error_guard, install_key_hold, install_rabbit_hole, install_image_error_guard, install_key_hold, install_progress_watchdog,
install_robust_set_foreground, install_shop_audit, install_shop_by_name, install_rabbit_hole, install_robust_set_foreground, install_shop_audit, install_shop_by_name,
install_system_team_selection, install_team_screen_guard, install_system_team_selection, install_team_screen_guard,
install_theme_pack_audit, install_theme_pack_refresh_click, install_theme_pack_systems, install_theme_pack_audit, install_theme_pack_refresh_click, install_theme_pack_systems,
install_typed_keyboard_road, load_rabbit_hole, mirror_progress, stop_driver_click) install_typed_keyboard_road, load_rabbit_hole, mirror_progress, stop_driver_click)
@@ -146,6 +146,7 @@ def _run(task, params):
decisions.open() # TaskRunner 给的 .decisions.jsonl; 直接运行时不写 decisions.open() # TaskRunner 给的 .decisions.jsonl; 直接运行时不写
install_exact_team_match(auto, log) install_exact_team_match(auto, log)
install_robust_set_foreground(screen, log=log) install_robust_set_foreground(screen, log=log)
install_progress_watchdog(log=log) # 40 分钟没有「开始执行/结束执行」就结束(11-17/11-21 各空转 5 小时多)
if not cfg.simulator: if not cfg.simulator:
install_key_hold(log=log) # 前台按键带扫描码、各模式按住再松开(ISSUES I-05) install_key_hold(log=log) # 前台按键带扫描码、各模式按住再松开(ISSUES I-05)
# 日常本(经验/纽本)不翻队伍列表: 队伍是自定义名字时按名字找必然失败, # 日常本(经验/纽本)不翻队伍列表: 队伍是自定义名字时按名字找必然失败,