Information Evaluation

Chapter 9

An Architecture for the Evolution of Trust: Definition and Impact of the Necessary Dimensions of Opinion Making

9.1. Introduction

The evaluation of algorithm performance, computational techniques and methods is full of scores of trust.¹ These scores generally rate the method rather than its output. That is to say, while it is acknowledged that automated processing yields uncertainty and imprecision, classic measurements usually characterize the way in which the information is constructed, rather than qualifying the degree of trust that should be invested in that piece of information. Although it is generally sufficient to know that an extraction algorithm is effective, say, eight times out of ten, or that its production is in line with expectations in the same proportions, once we start dealing with sensitive data, at the root of potentially tragic decisions, we may wish to measure how much to trust the information item rather than have uniform doubt about its construction. Such a measurement should vary with the contents of the information itself, instead of on the basis of the tools used to produce it. Furthermore, if the elaboration method for this trust indicator is explained, legible or even adaptable, the user can then learn to grasp the system and use its indications in the rest of his intervention. To reach such a goal, it is necessary to introduce a distinction between rating – the expressed degree of trust – and evaluation – the method used to evaluate trust. ...