The term „bibliometrics“ refers to the quantitative evaluation of scientific publications using statistical methods. It was coined by Alan Pritchard as early as 1969. Pritchard defined bibliometrics as:
..the application of mathematics and statistical methods to books and other media of communication.
One of the essential tasks of bibliometric analysis is to provide information on the publication performance of a person, a scientific group or an institution. It also provides information on the impact of the publication within the professional public, the integration in the scientific landscape, and the international perception in comparison with others.
For individual scientists, the impact as well as the visibility of their research in form of publications play a major role in enhancing their reputation. Recent studies, such as the one by Jim Ottaviani, have shown that the citation frequency of freely accessible scientific content is much higher than that of paid publications.
According to Mr. Ottaviani’s study, articles that are published as a secondary publication and thus made freely accessible are cited up to 19 percent more often than the commercial counterpart. More and more open access journals have a journal impact factor that is often higher than that of comparable paid journals.
In addition to the journal impact factor, the number of downloads of open access documents is a measure for the visibility of the scientific publications provided.
Citation databases such as Web of Science have recognized the constantly increasing tendency of open access journals, so that the number of open access publications in this database is rising continuously.
Since 2014, Web of Science has enabled the selection of open access publications in its “Refine Menu” (bottom left column “refine results”). According to Thomson Reuters, these are open access journals from the Directory of Open Accesss Journals (DOAJ).
Furthermore, Forschungszentrum Jülich offers the possibility of more searches for open access journals via their publication portal JOIN2. For example, the intersection of journals indexed in DOAJ and in the Journal Citation Reports (JCR) of Web of Science can be queried.
It is often argued against bibliometric indicators, e.g. the Hirsch Index (h-index) or the impact factor, that complex systems cannot be represented by a single indicator on principle. Different indicators in themselves have only very limited significance. Therefore, the combination of several indicators is absolutely necessary for the evaluation of research performance. The quantitative statements thus obtained are a valuable orientation in the evaluation of research performance, but do not state anything definitive about the quality of research.
Furthermore, the following issues should be taken into account when creating bibliometric analyses:
Document Types Included
On the one hand, publication databases for bibliometric analyses do not represent the entire extent of scientific publications, on the other hand not all document types are recorded and included.
Counting the Citations
Another source of error is the fact that works cited several times in a paper are listed only once in the „Citation Index“. This can lead to a distortion of results as no information on the intensity of the used works is provided. That way, it can happen that a work is used as basis in the published paper and therefore is automatically mentioned more often while another work is mentioned only once. The result for both is a single listing in the „Citation Index“. Thus the “Citation Index” only records the fact of the corresponding information transfer, but not its specification.
Citations as Measure for Quality
Each citation indicates an impact on the citing authors, be it that citation results are used by them directly, or be it that they only evaluate the cited publication and classify it into the course of research. In any case, a publication receives attention through the citation. A high number of citations, however, is not a statement about the quality of the work.
Discipline-specific Publication Behavior
Every scientific discipline has its own habit of publication and citation. For example, you find on average five to ten literature references in papers from the mathematical discipline, 20 to 30 literature references in chemistry and physics, and 50 to 60 references in papers on molecular biology. Due to this, it is always difficult to compare different interdisciplinary fields.
With the constantly growing number of electronic publications, new metrics are also increasingly developing based on usage figures as an important criterion of evaluation. Unlike the classic citation-based metrics, usage statistics reflect the current scientific significance of a digital document in the scientific community. Article-related metrics such as downloads, views or mentions of scientific publications in social networks and on reference management platforms are recorded.