AnalysisOpinion / analysisAgents & Tool Use · Multimodal Models · Applications1 source · Oct 11, 2026

What ai is best to operate a system from screen video [p]

I have working programmatic control over a system, I can send inputs reliably and capture the screen as visual feedback.

Proof1 community thread

Key points

  • The control/feedback loop is already done.
  • I need something that can learn from visual information, remember what it has seen and been taught, and get better at operating the system over time.
  • recommendations? the AI I have tried so far it's memory is not good enough and it often doesn't consult the visual information before making decisions which is very frustrating.

Sources (1)

Extractive summary: sentences quoted from the sources.

Related