Medical Informatics · 1990 · 19 citations · 14 references
NursingKnowledge ManagementClinical Information SystemsHealth Information SystemsHealth Systems EvaluationExpert SystemsHealth Services EvaluationHealth System PerformanceKnowledge Base DesignKnowledge-based DesignSystem ImplementationClinical Decision Support System ImplementationClinical Knowledge ManagementEvaluation Of KnowledgeHealthcare Systems EvaluationHealthcare Systems ImplementationKnowledge Base EvaluationKnowledge-based Expert SystemsMedical Expert Systems
After the early experiments in artificial intelligence a methodology is emerging around advanced systems for the management of medical knowledge. The stress is moving away from the implementation of prototypes to the evaluation. It is possible to adapt and to apply this to field evaluation techniques already developed in similar contexts of knowledge management (books, drugs, epidemiology, consultants, etc.). The time is ready for a further step: to envisage a methodology for the design of real systems that cope with the 'knowledge environment' of the user. Every stage of the evaluation process is re-examined here, and considered as a framework to define goals and criteria about a step of design: (1) the impact of the system on the progress of health care provision (priorities, cost-benefit analysis, share of tasks among different media); (2) effectiveness in the end-user's environment and long-term effects on his behaviour (changes in people's role and responsibilities, improvements in the quality of data, acceptance of the system); (3) the intrinsic efficiency of the system apart from the operational context (correctness of the knowledge base, appropriateness of the reasoning). The need to differentiate the test sample into three classes (obvious, typical, atypical) is emphasized, discussing the influence on both evaluation and design. In particular the difficulty of having 'gold standards' on atypical cases, due to the disagreement among the experts, leads to the definition of two alternative attitudes: the 'standardization mode' and the 'brain-storming mode'.
14
Frederick Hayes‐Roth, D. A. Waterman, Douglas B. Lenat · 1983 · 1.7K citations
Validating Expert System Performance
Robert M. O’Keefe, Eric P. Smith · IEEE Expert · 1987 · 359 citations
Computer-Assisted Medical Decision-Making
Wendy Lutz, Homer R. Warner · Biometrics · 1982 · 111 citations
Evaluating medical expert systems
Henry P. Lundsgaarde · Social Science & Medicine · 1987 · 92 citations