EVALUATION EXERCISE
The system which you are currently running is based on the opinions
of 41 veterinary experts, converted into a suitable format to parameterise
a Bayesian
Belief Network. You might not necessarily agree with all
of the opinions expressed by the experts whom we contacted: certainly,
not all of the experts agreed with each other! We therefore have
to test whether or not the information which was collected can be used
in a belief network. It is important to test whether or not CaDDiS is successfully
generating sensible, accurate and useful diagnoses. We would be grateful
if you would help us to evaluate whether the system is successful in meeting
these objectives.
There are two ways to approach the evaluation of a system such as CaDDiS.
Explore the capabilities of the system, adding and removing clinical
signs as you choose. At each stage, check the beliefs calculated by the
system to see whether they match your analysis of which diseases are most
likely to be implicated in causing these signs. If at any point you feel
that CaDDiS has presented inaccurate information, please note which clinical
signs are contained in the Observed / Absent Signs window (this can easily be
done by screen-dumping the CaDDiS window to a file or directly to a printer.
To this list of signs, please attach a note of what you believe the diagnosis
should be, and specifically highlight those areas of the CaDDiS diagnosis
with which you disagree.
Alternatively, use your field experience to define a set of plausible
observed clinical signs (typically with a particular causal agent in mind).
Enter these signs into CaDDiS, and compare the results with your mental
picture of sensible diagnoses. This type of information is invaluable to
us, whether or not you agree with the CaDDiS findings. For every scenario
which you evaluate, please print off a list of observed clinical signs,
tell us what your diagnosis would have been, and comment on the accuracy
or otherwise of the CaDDiS diagnoses.
To reiterate, please send us this information, whether or not you agree
with CaDDiS, whenever you have specifically chosen a set of clinical signs
as being typical of some situation.
Please send your evaluation materials to Mr. Crawford Revie, Department
of Information Science, 26 Richmond Street, Glasgow, G1 1XH.