Results across disciplines showed that Wikipedia scored higher in this sample in terms of provision of references in humanities-based articles, but no differences were apparent in terms of the other criteria, as was also the case with articles in mathematics, physics and life sciences. There was a similar result for articles in social sciences, but with higher scores on style/ readability for the other encyclopaedias. In medical science articles, Wikipedia scored significantly higher on accuracy, references and overall judgment, but there were no differences on the other criteria.
Qualitative results for this sample showed similar findings, but also revealed the importance to reviewers of articles possessing a sense of cohesiveness and structure. Although many Wikipedia articles in the sample were commented on favourably, they were criticised in some cases for lacking cohesiveness and for internal inconsistencies and repetition. Reviewers were particularly approving of articles that presented an engaging and coherent introduction to a topic, rather than excessive amounts of information.
The same differences seen in the quantitative analysis were evident in the qualitative with respect to different languages. In terms of different disciplines, small differences in terms of favoured quality criteria were evident, such as an emphasis on the notion of conciseness in the science-based article reviews.
6. Discussion
In many respects, the methodological approach had proved productive and workable on the small scale of the present study. But it was recognised that there were difficulties (even on this small scale) in terms of identifying appropriate articles, recruiting a sufficient range of reviewers, and anonymising articles which, if the study were to be carried out on a far larger scale, would possibly prove hard to surmount. Therefore, it is recommended that the viability of a larger study of this kind in the future should be considered cautiously, and that consideration might be given instead to carrying out a series of more compact studies of this kind over time.
It is also recommended that more research might be carried out on what is reasonable and appropriate to expect of online encyclopaedia content. It was clear from this study that, while many academics spoke in positive terms about a high proportion of articles reviewed from all encyclopaedias, it was not the case that they were inclined to regard these as being citable in academic publications alongside peer-reviewed journals and published books. We recommend that more research is done on how users interpret and make sense of content from online encyclopaedias in general and from Wikipedia in particular.
Overall, the Wikipedia articles in this very small sample, investigated as part of a pilot study only in this instance, fared well in comparison with articles from other encyclopaedias. While no generalisations can be made from this outcome, these findings do help to point researchers in future studies towards investigation of the unique qualities of Wikipedia, as a source of knowledge that was shown in the small number of instances studied here at least to be capable of producing articles that were markedly up to date and well referenced.
7
