Structured pre-generation elicitation versus single-shot prompting in AI-assisted enterprise decision-making: a randomised online experiment
Generative AI speeds, and mostly improves, professional work, but there is concern that users who delegate both the production and the evaluation of an answer may accept weak output and engage less with the underlying reasoning (cognitive surrender).
ProofPaper ↗
Key points
- Interventions proposed so far, such as unassisted practice or slowing adoption, sit outside the working task.
- We tested a different approach: an interactive metacognitive scaffolding layer (Cognistance, a prototype developed at the Oxford Centre for Impact Research (OCIR) that asks users to clarify context, choose a strategic direction and explain their reasoning before the AI generates a deliverable).
- Structured elicitation before generation improved the rated quality and task relevance of AI-assisted strategy documents at modest cost in time.
- Delayed retention, error detection and effects in live organisations are the priorities for the next stage of research.
Sources (1)
- [1]Structured pre-generation elicitation versus single-shot prompting in AI-assisted enterprise decision-making: a randomised online experimentarXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 7, 07:39 AM
Generative AI speeds, and mostly improves, professional work, but there is concern that users who delegate both the production and the evaluation of an answer may accept weak output and engage less with the underlying reasoning (cognitive surrender).
Interventions proposed so far, such as unassisted practice or slowing adoption, sit outside the working task.
Extractive summary: sentences quoted from the sources.