Cognitive Shadow: capturing judgement policies for real-time decision support

Formally capturing expert knowledge is a major challenge in the field of decision support. Policy capturing is a judgment analysis method that infers a decision policy based on past examples. A \"judgmental bootstrapping\" effect can be observed when a model actually outperforms the human decision maker on which the model was based (typically because the model is not prone to fatigue or distraction). A prototype policy capturing tool called the Cognitive Shadow has been developed to automatically learn an operator's decision pattern and provide warnings in real-time when a mismatch is detected between the user's decision and the inferred decision policy. This project aims at investigating critical human factors and cognitive engineering issues for implementing the Cognitive Shadow capability in the context of naval air defence. The specific objectives are to 1) assess factors influencing operator trust in the cognitive assistant, 2) assess impacts on workload,

behavior and performance measures, and 3) understand the different impacts of two possible modes of operation: a shadowing mode where the operator receives an alert only when a mismatch with the decision model is detected, versus a recommendation mode where the operator receives information about the model's expected decision. A series of human-in-the-loop experiments will be performed in two phases in order to systematically investigate these issues. In Phase 1 the Simulated Combat Control System testbed is used as a functional simulation of naval air defence practiced aboard the Canadian Navy's Halifax class frigates. Phase 1 will involve two laboratory experiments involving implementations of the Cognitive Shadow hypothesized to differentially impact workload, trust, performance and competency preservation. Phase 2 will involve using the Cognitive Shadow with domain experts in the context of a real-world combat management system. This project will help Thales Research and Technology Canada take a leap forward in maturing the new Cognitive Shadow

technology, and Defence Research and Development Canada in leveraging the technological advances for the Royal Canadian Navy.