Work2Skill: Capturing Human GUI Experience for Reusable Agent Skill Induction
We thus introduce Work2Skill, a framework that captures human GUI experience for reusable agent skill induction.
ProofPaper ↗
Key points
- Computer-use agents can perform increasingly complex GUI tasks, yet unfamiliar software and workflows requiring human domain knowledge remain challenging.
- Human demonstrations convey procedural knowledge through the same GUI interaction space, but reuse requires interpreting actions in context and separating workflow rules from task-specific details.
- WorkRecorder provides the framework's infrastructure for collecting this experience, aligning screen recordings, mouse and keyboard events, and available accessibility-tree information.
- We then propose an agentic skill-induction method that inspects evidence on demand to produce reusable skills capturing workflow rules, step dependencies, and completion checks.
Sources (1)
- [1]Work2Skill: Capturing Human GUI Experience for Reusable Agent Skill InductionarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 04:32 AM
We thus introduce Work2Skill, a framework that captures human GUI experience for reusable agent skill induction.
Computer-use agents can perform increasingly complex GUI tasks, yet unfamiliar software and workflows requiring human domain knowledge remain challenging.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 8, 2026openai/codex rust-v0.162.0: 0.162.0
- Oct 8, 2026SoftwareGym: Scaling Computer-Use Agent Training with Diverse Synthesized Software Environments
- Oct 8, 2026Microsoft Joins the Local AI Push
- Oct 8, 2026Can LLMs Fix It Without Code? Toward Automated Verification of No-Code Bug Fixes
- Oct 8, 2026Understanding Jev, the new model everyone is talking about
- Oct 7, 2026[AINews] Claude Haiku 5.5 — better than GPT-6 Luna at the same pricing