Email this article
Publications in the Institute of Scientific Information’s (ISI, currently Thomson Reuters) Web of Science (WoS) and Elsevier’s Scopus databases were utilized to collect data about Wikipedia research and citations to Wikipedia. The growth of publications on Wikipedia research, the most active researchers, their associated institutions, academic fields and their geographic distribution are treated in this paper. The impact and influence of Wikipedia were identified, utilizing cited work found in (WoS) and Scopus. Additionally, leading authors, affiliated institutions, countries, academic fields, and publications that frequently cite Wikipedia are identified.Contents
Introduction
Brief introduction to Wikipedia
Literature review
Data collection
Data analysis and discussion
Summary and conclusions
No one denies that Wikipedia is now a highly used, albeit controversial, information source. Wikipedia has become increasingly an important tool for “fact–checking” (Kniffel, 2008) as well as a topic of research because of its convenient access on the Web, its coverage, and the nature of large–scale collaborative work, among other reasons. According to WorldCat (24 August 2010), Wikipedia has been a topic of more than 50 theses and dissertations worldwide and has been a subject of more than 200 monographic publications.
The purpose of this study is to explore the extent of Wikipedia’s presence in scholarly publications in Web of Science (WoS) and Elsevier’s Scopus databases. The Institute for Scientific Information (ISI), publisher of WoS, asserts that it contains the world’s leading citations from multidisciplinary coverage of over 10,000 high–impact journals in the sciences, social sciences, and arts and humanities, as well as international proceedings coverage for over 120,000 conferences. WoS covers Science Citation Index Expanded indexing over 6,650 major journals, Social Science Citation Index containing over 1,950 journals and Arts and Humanities Citation Index for 1,160 of the world’s leading arts and humanities journals [1]. Scopus states that it contains 18,000 titles from more than 5,000 international publishers, including 16,500 peer–reviewed journals in addition to about 1,200 open access journals, 600 trade publications, 2,350 book series, and 3.6 million conference papers among others [2]. Some differences in WoS and Scopus databases should be noted. The scope and types of publications included in WoS and Scopus differ and this should be taken into account in understanding the search results and interpretations. It is clear that WoS covers journals more selectively while Scopus covers a much higher numbers of conference papers. A recent study on the journal title overlap between WoS and Scopus databases reported that about 45 percent of titles in Scopus are not covered in WoS, while 16 percent of titles in WoS are not covered in Scopus (Gavel and Iselid, 2008).
Brief introduction to Wikipedia
Wikipedia: About page defines Wikipedia as a multilingual, Web–based, free–content encyclopedia project based on an openly editable model. Anyone can contribute and edit the Wikipedia articles. Users can contribute anonymously, or under a pseudonym, or with their real identity. The page history view (revision history or edit history) includes a list of the page’s previous revisions, including date and time, the user name (or IP address) and edit history. However, Cohen (2009) reported that the English Wikipedia added an imposing layer of editorial reviews on articles about living people declared no longer available in the openly editable mode. Since its inception in 2001, Wikipedia has published 17,000,000 articles. There are currently 91,000 active contributors, and Wikipedia is now available in 270 languages. The English Wikipedia alone includes more than three million articles, 23 million pages, more than 446 million edits and is attracting 79 million visitors monthly on the Internet, as of January 2011. Wikipedia was founded as an offshoot of Nupedia, founded by Jimmy Wales and officially launched on 15 January 2001.
Among approximately three million articles in the English Wikipedia, there are about 3,194 (about 0.1 percent) featured articles. Featured articles represent the best articles which, according to Wikipedia’s featured list criteria, have undergone a thorough review process by Wikipedia’s editors to meet the highest standards for usefulness, completeness, accuracy, neutrality and style. A featured article has a small bronze star icon on the top right corner of the article’s page. Citing a study conducted by researchers at the Carnegie Mellon University and Palo Alto Research Center, the Wikipedia’s site lists the most frequently covered topics — Culture and the arts (30 percent), Biographies and persons (15 percent), Geography and places (14 percent), Society and social sciences (12 percent), History and events (11 percent), among others. Spoerri (2007) examined the popularity of topics in Wikipedia and found the most popular Wikipedia pages were related to entertainment and sexuality. Popular pages appeared to be related to search engines, especially Google. The site reports that the growth of the English Wikipedia in terms of new articles and contributors reached a plateau in early 2007. Landgraf (2009) also reported a reduction in Wikipedia’s growth. Kopytoff (2011) reported on the celebration of Wikipedia’s tenth anniversary and mentioned plans to increase the number of foreign language articles by opening an office in India, then possibly Egypt and Brazil. Plans also include the recruitment of a wider range of contributors — more women, elderly, and, to add more graphical content, museum experts.
Reviews in the library and information science literature indicated that Wikipedia itself has increasingly become a subject of research from diverse academic disciplines due to its exceptional scale and utility (Medelyan, et al., 2009). The concept of “information quality (IQ),” incorporating collaboration, evolving debates, and process as assurance, was studied using Wikipedia as an example (Stvilia, et al., 2008). A Wikipedia entry, Wikipedia: Academic studies of Wikipedia (http://en.wikipedia.org/wiki/Academic_studies_about_Wikipedia) reports a partial list of academic writings about Wikipedia reported in journal articles, and conference proceedings among other formats and the Academic studies about Wikipedia page includes some Wikipedia research in peer–reviewed publications.
A question about Wikipedia’s quality and reliability as an information source has been one of the most frequently investigated research topics. In an evaluation of Wikipedia as a reference source, applying the classic reference evaluation criteria — purpose, authority, scope, audience, cost, and format — Danny P. Wallace and Connie Van Fleet (2005) concluded that Katz’s criteria for reference sources do not stand up well to Wikipedia. A comparison of Wikipedia and other encyclopedias in historical entries revealed that Wikipedia’s accuracy was 80 percent compared with 95–96 percent accuracy in other sources (Rector, 2008). A special report by the prestigious weekly journal Nature (Giles, 2005) raised commentary from Encyclopedia Britannica. Nature’s investigation, based on 42 science entries, found that both Wikipedia and Britannica contained numerous errors, but the difference in accuracy was not great. The average inaccuracy rate in Britannica was about three per article while Wikipedia contained about four. The number of edits, collaborators, and edit patterns were studied in relation to article quality. Wilkinson and Huberman (2007) compared the number of edits and contributors to the 1,211 “featured” articles to the same number of other articles to test the correlation between number of edits and article quality. They concluded that Wikipedia article quality appeared to increase on average as the number of collaborators and number of edits increase. Revising patterns — the total number of editors, the number of edits, and the number of major and minor edits — in a sample of two groups of articles were studied to determine their relationship to article quality (Poderi, 2009). The study reported that not every contribution had the same weight and major edits were not necessarily contributing to article quality. The role of main editors differed in the two groups of article. The articles in the group with a high presence of main editors tended to become featured articles more easily. Other aspects of quality such as Wikipedia’s biased coverage and lack of cited sources were identified as “Wikipedia risks” (Black, 2008). Nielsen (2007) examined about 30,368 outbound links in Wikipedia’s science entries. Although the number of linked citations to scholarly literature was small compared to the number of citations found in scientific journals, Wikipedia showed a slight tendency to cite articles in high–impact ISI journals. For example, the largest number of citations were to Nature, Science and the New England Journal of Medicine in the sample studied.
Coverage of philosophers in the twentieth century listed in Wikipedia and in two other widely used online resources was compared for data regarding their birth date, gender, national and disciplinary backgrounds. This study found that Wikipedia contained more entries for living and ‘minor’ philosophers than traditional resources (Elvebakk, 2008). The semantic coverage of the English Wikipedia was studied and represented in terms of baseline statistics for articles, subject categories, and the top 10 authors (Holloway, et al., 2007).
Use of Wikipedia is on the rise. While some university professors have banned using Wikipedia as a research source (Cohen, 2007), use of Wikipedia was promoted using epistemic values. Fallis (2008) argued that there were good epistemic consequences of using Wikipedia as a source of information by illustrating some empirical examples. Epistemic values such as power, speed, immediate availability, wiki technology, the wisdom of crowds, and Wikipedia policies were noted as outweighing the deficiencies in the reliability of Wikipedia. Despite controversies, use of Wikipedia by academic communities has been expanding. More positive responses to Wikipedia have been reported from academic libraries. For example, libraries at the University of Washington, University of North Texas, and Wake Forest University, among others, have decided to participate in Wikipedia by editing, adding links, or writing new articles (Lally and Dunford, 2007; Pressley and McCallum, 2008). Lim’s (2009) survey on college students’ use of Wikipedia also showed that students use it as a source for quick fact–checking and for finding background information. Student’s perceptions regarding information utility and their positive emotions toward Wikipedia were related to their usage level. Use of Wikipedia in college class room has been reported. One of Wikipedia’s recent projects, Public Policy Initiative (http://outreach.wikimedia.org/wiki/Public_Policy_Initiative), became a teaching resource in some universities [3]. For example, five universities — Georgetown, George Washington, Harvard, Indiana University and Syracuse — were invited to work on editing articles on the policy–related entries in Wikipedia to improve the article quality.
Citation counts in scholarly publications have been frequently used as an important tool — to assess the relative scholarly impact of research, diffusion of new research ideas, to study journals, individual researchers, and to identify maps of scholarly communication across scientific specialties and so on (Meho and Sugimoto, 2009). Cronin and Shaw (2007) used bibliometric tools to identify Kling’s intellectual impact and network using his publications, his cited works, and acknowledgment data. Others studied citing behaviors and motivations of citers besides scientific impact (Bornmann and Daniel, 2008). For the citation counts, ISI databases (such as WoS), Scopus and Google Scholar are the most often used tools. ISI’s three citation databases were the only comprehensive citation data source until Elsevier’s Scopus and Google Scholar were launched in 2004. In a paper comparing the citation counts provide by WoS, Scopus and Google Scholar for articles from the Journal of the American Society for Information Science and Technology, Bauer and Bakkalbasi (2005) conclude that Google Scholarr likely retrieves traditional journal articles which are also possibly covered by WoS and Scopus in addition to unique citations. However, the coverage of scholarly publications was the least in Google Scholar.
The visibility of Wikipedia in scholarly communications was examined based on the following questions:
- How many times Wikipedia has been a topic of research in scholarly publications covered in WoS and Scopus databases?
- Who are the contributors most often engaged in doing research about Wikipedia?
- What are these authors’ institutional affiliations?
- Which publications have published studies on Wikipedia most often?
- Which academic fields are engaged in studying Wikipedia most frequently?
- How often Wikipedia has been cited in the scholarly publications covered in WoS and Scopus databases?
- Who cites Wikipedia most often?
- Which publications cite Wikipedia most frequently?
- Which academic fields cite Wikipedia most often?
- Authors from which institutions most frequently cite Wikipedia in their publications?
Two types of data were collected to examine the visibility of Wikipedia in scholarly publications. The presence of Wikipedia in scholarly publications was assumed if a study’s major topics include “Wikipedia,” or “Wikipedia” has been used in their references. A search in WoS using Wikipedia in the topic OR title field was conducted in January 2011 to find the number of records for which a publication’s topic is Wikipedia. A truncated search was used to match any variations and to achieve a more comprehensive search result. In the same way a search in Scopus in the title, abstract, or keyword fields was conduced. There were 291 records in WoS and 1,455 in Scopus with topics including Wikipedia. Scopus allows a search conducted beyond its own databases by providing Web searching options. Yet, this research was limited to Scopus alone as it includes only peer–reviewed publications.
The search result displays typical citation information including author(s), title (document), source title, its volume and number designation, pagination (if available), and publication year. From the search in WoS, all search results were selected to display the list of publications with a main topic on Wikipedia and to refine the result using ISI analysis tools. These analysis tools allow the search results to be sorted by ranked order for a selected field (e.g., author, institutional name, country, etc.). For example, Brendan Luyt and Oded Nov have published most frequently on Wikipedia in scholarly publications covered by WoS. A search result in Scopus displays ranked lists of each field, for example, by source title, author name, publication year, affiliation, subject area, document type, etc. Advanced search features in Scopus were utilized for more precise and comprehensive searching. For example, a search combined with the field “affilcountry” (United States or US) displays publication output by researchers affiliated with institutions located in the United States. An “affilorg” (Hong Kong) brings additional research output by researchers affiliated with institutions in Hong Kong. An advanced search combined with “subjarea” (comp) shows the number of documents categorized as computer science.
To examine aspects of Wikipedia’s impact, a search for “cited work = Wikipedia*” was conducted in WoS. In a “Cited Reference Search”, all references in the WoS databases that cite Wikipedia were retrieved. The search result listed cited author(s), cited work (Wikipedia), year (if available), and the number of times cited for a specific article. There were 340 records citing Wikipedia in WoS. Once the search is executed, all entries which cite Wikipedia are selected, then the search is finished. One should note that the number of citing articles on the “Cited Reference Search” page and the number listed in the “Times Cited” count on the results page after finishing the search might differ depending on the scope of one’s institution’s subscriptions to various databases within WoS. The “Times Cited” count on the results page are counted from all the databases in WoS: Science Citation Index Expanded, Social Science Citation Index, Arts and Humanities Citation Index, Conference Proceedings Citation Index–Science, and Conference Proceedings Citation Index–Social Sciences and Humanities. For example, if an institution has a subscription to the Science Citation Index Expanded and Social Science Citation Index, Arts and Humanities Citation Index but not the Conference Proceedings Citation Index, the number of citing articles on the “Cited Reference Search” page may be smaller. In addition, the result may be influenced by one’s subscription periods. If an institution has access to a limited time period such as from 2005 to the present, the result would probably be smaller.
Larry Dossey and Brendan Luyt cited Wikipedia most often in their scholarly publications as noted in WoS. In a similar way, a search for “refsrctitle = wikipedia*” was conducted in Scopus for publications with the source title (Wikipedia) in references. There were 3,339 records citing Wikipedia as source titles in references in Scopus. All search results were downloaded into an MS Excel file for data analysis.
Country productivity
There were, as of January 2011, a total of 1,746 publications in WoS and Scopus for the period 2002 to 2010, which contained research about Wikipedia. The number should be taken with caution due to overlapping coverage of publications between WoS and Scopus as noted earlier. Furthermore, these numbers may change as the coverage of publications in the WoS and Scopus databases is updated.
To achieve a more precise measurement of research production by country, Hong Kong was searched separately and added to China’s production for data analysis. China’s production included three additional publications from Hong Kong in WoS. Likewise, for the United Kingdom, additional searches were conducted in WoS for England, Scotland, Wales, and Northern Ireland. Four publications from Scotland were added to the United Kingdom’s total. In a similar manner, an additional country search for Hong Kong in Scopus added 18 more publications to China. The country name, United Kingdom, was used consistently in Scopus for all publications affiliated with that nation. Table 1 lists the most productive countries in Wikipedia research. The most productive countries were the U.S. and the United Kingdom in WoS and the U.S. and Germany in Scopus. The next most productive countries were China, France, the United Kingdom, Japan, Italy and the Netherlands in Scopus. The U.S. is far stronger in producing research on Wikipedia than any other country, accounting for about 22 percent of the publications in Scopus and about 37 percent in WoS.
Table 1: Research production on Wikipedia by country. Country Number of publications in WoS
(percent)Country Number of publications in Scopus
(percent)United States 107 (36.8) United States 315 (21.6) United Kingdom 25 (8.6) Germany 137 (9.4) Germany 22 (7.6) China
(including Hong Kong)99 (6.8) Canada 13 (4.4) France 69 (4.7) Australia 12 (4.2) United Kingdom 65 (4.5) China
(including Hong Kong)12 (4.2) Japan 64 (4.4) France 11 (3.8) Italy 57 (3.9) Italy 9 (3.1) Netherlands 57 (3.9) Spain 9 (3.1) Australia 55 (3.8) Netherlands 9 (3.1) Spain 50 (3.4) Singapore 9 (3.1) Canada 42 (2.9)
Author productivity
Analysis of author productivity, based on the number of publications included in WoS and Scopus, indicated that a small numbers of authors created a number of publications. There were 291 publications with a total of 701 authors in WoS and 1,455 publications in Scopus with a total of 3,940 principal and collaborative authors with a research topic including Wikipedia. Multiple authorship was the norm. For example, one publication about Wikipedia research in Scopus was coauthored by 37 individuals.
Two individuals wrote 13 papers, another two researchers contributed 12 publications, and six published 10 items about Wikipedia. Altogether, 123 individuals wrote more than four publications on Wikipedia. The most highly productive 15 individuals, their affiliated institutions, countries, and the number of their publications are listed in Table 2. Individual researchers who developed the most frequent publications were affiliated with institutions located in Europe and Asian countries. Jaap Kamps at the University of Amsterdam (http://staff.science.uva.nl/~kamps/) and Gerhard Weikum of the Max–Planck–Institut für Informatik (http://www.mpi-inf.mpg.de/~weikum/) each wrote 13 articles dealing with, in some fashion, Wikipedia.
Table 2: Most highly productive authors in research on Wikipedia, based on Scopus. Name Affiliation Country Number of publications Kamps, J. University of Amsterdam Netherlands 13 Weikum, G. Max–Planck–Institut für Informatik Germany 13 Geva, S. Queensland University of Technology Australia 12 Nakayama, K. Osaka University Japan 12 Koolen, M. University of Amsterdam Netherlands 10 Hara, T. Osaka University Japan 10 Kittur, A. Carnegie Mellon University United States 10 Ortega, F. Universidad Rey Juan Carlos Spain 10 Nishio, S. Osaka University Japan 10 Sun, A. Nanyang Technological University Singapore 10 Demartini, G. L3S Research Center Germany 8 Jijkoun, V. University of Amsterdam Netherlands 8 Milne, D. University of Waikato New Zealand 8 Trotman, A. University of Otago New Zealand 8 Witten, I.H. University of Waikato New Zealand 8
Affiliated institution productivity
The majority of researchers on Wikipedia were affiliated with universities. The most productive 15 institutions are listed below in ranked order, in Table 3. Individual researchers affiliated with the University of Amsterdam, Nanyang Technological University and the Max–Planck–Institut für Informatik were the most productive in doing research on Wikipedia. These 15 affiliated institutions contributed 230 publications which were about 13 percent of the total publications in WoS and Scopus. Researchers affiliated with the Carnegie Mellon University and Indiana University were most active in research on Wikipedia in the United States.
Table 3: Most highly productive institutions on Wikipedia. Institution Number of papers University of Amsterdam 31 Nanyang Technological University 23 Max–Planck–Institut für Informatik 19 Queensland University of Technology 17 Carnegie Mellon University 17 University of Tokyo 16 Indiana University 15 University of Illinois at Urbana–Champaign 12 Hewlett–Packard Laboratories 12 Osaka University 12 Shanghai Jiao Tong University 12 University of Washington 11 IBM Thomas J. Watson Research Center 11 Georgia Institute of Technology 11 Microsoft Research 11
Academic fields which are most active in Wikipedia research
Table 4 describes 10 academic fields which are most active in Wikipedia research according to WoS and Scopus. Academic fields in this study were defined by these databases respectively. Scopus categorizes its content into 27 subject areas and WoS includes 251 subject areas. Certainly, computer science was the most productive. About 42 percent of Wikipedia research is produced from many areas in the computer science fields and about 26 percent from information and library science in WoS databases while in Scopus about 72 percent of research originates from the computer science category as defined by Scopus. The fields of mathematics, social sciences, and engineering are also highly productive. In Scopus an exceeding small portion of publications, about one percent of Wikipedia research output, derives from the arts and humanities. Note that a publication may be categorized in more than one subject category and thus the total number of publications may include duplication.
Table 4: Academic fields most active in Wikipedia research. Academic fields
(WoS)Number of publications
(percentage)Academic fields
(Scopus)Number of publications
(percentage)Information science, Library science 74
(25.6)Computer science 1,052
(72.3)Computer science, Information systems 73
(25.2)Mathematics 341
(23.4)Computer science, Artificial intelligence 24
(8.3)Social sciences 260
(17.9)Engineering, Electrical and electronic 19
(6.6)Engineering 199
(13.7)Communications 16
(5.5)Biochemistry, Genetics and molecular biology 109
(7.5)Computer science, Theory and methods 13
(4.5)Decision sciences 99
(6.8)Education and education research 13
(4.5)Business, Management, Accounting 85
(5.8)Management 13
(4.5)Medicine 44
(3.0)Computer science, Hardware and architecture 12
(4.2)Agriculture and Biological sciences, Arts 16
(1.1)Multidisciplinary sciences 10
(3.5)Arts and humanities 15
(1.0)Physics and Astronomy 15
(1.0)
The leading publications reporting research on Wikipedia
Table 5 rank orders the 11 most productive publications on Wikipedia research in both WoS and Scopus. it appears that more research about Wikipedia has been published in conference papers and proceedings than in journal articles. As conference titles tend to vary frequently, more comprehensive searches for conference publications were conducted. Series such as Lecture Notes in Computer Science including the subseries Lecture Notes in Artificial Intelligence; Lecture Notes in Bioinformatics; Proceedings of the International Conference on Information and Knowledge Management; and, International Symposium on Wikis (with slightly variant titles) are the leading outlets for Wikipedia research. Lecture Notes in Computer Science is a major series; a WorldCat search retrieves more than 100,000 items. The International Symposium on Wikis’ Web site reports that it focuses on research and practice about wikis and open collaboration. Thus it appears to be a very appropriate venue for Wikipedia research. First Monday is also highly regarded in publishing Wikipedia research. Because of the coverage differences between WoS and Scopus, Wikipedia research is most often reported in journals in WoS and conference proceedings in Scopus. However, it is noteworthy that the Journal of American Society for Information Science and Technology (JASIST) appears on both lists. The top 11 publications produced about 20 percent of the Wikipedia research in WoS compared to about 37 percent in Scopus. It is interesting that Wikipedia research appears to be concentrated in a small number of publications as recorded in Scopus while scattered among a larger number in WoS.
Table 5: Leading serials publishing Wikipedia research. Publication as reported in WoS Number Publication as reported in Scopus Number Journal of American Society for Information Science and Technology 14 Lecture Notes in Computer Science, including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics 292 Online Information Review 6 Proceedings of International Conference on Information and Knowledge Management 61 Journal of Computer–mediated Communication 5 International Symposium on Wikis 53 Journal of Web Semantics 5 International ACM SIGIR Conference on Research and Development in Information Retrieval 29 BMC Bioinformatics 4 First Monday 26 Computers in Human Behavior 4 AAAI Workshop Technical Report 14 Electronic Library 4 Journal of American Society for Information Science and Technology 14 Information Systems 4 Proceedings of AAAI National Conference on Artificial Intelligence 13 Nature 4 Proceedings of ACM Conference on Human Factors in Computing Systems 12 Information Retrieval 4 Proceedings of ACM Conference on Computer Supported Cooperative Work 12 New Media & Society 4 International Conference on Knowledge Discovery and Data Mining 11
Impact of Wikipedia
Citations to Wikipedia in scholarly publications were examined to test Wikipedia’s impact on scholarly communication. This effort attempted to identify those who cite Wikipedia most often, their affiliated institutions, associated fields, and geographic distribution.
Wikipedia was cited 3,679 times in the WoS and Scopus databases. The 11 researchers who cited Wikipedia most frequently in their scholarly publications were from eight countries. Saou–Wen Su, affiliated with the Lite–On Technology Corporation in Taiwan, cited Wikipedia in eight publications; Gerhard Weikum of the Max–Planck–Institut für Informatik cited Wikipedia in in seven publications. Table 6 lists individual researchers who cited Wikipedia most frequently in their papers as recorded by Scopus and WoS.
Table 6: Citation of Wikipedia by specific researchers. Name Country Number Su, Saou–Wen Taiwan 8 Weikum, Gerhard Germany 7 Boukerche, Azzedine Canada 6 Ortega, Felipe Spain 6 Ren, Y. United States 6 Ros, L. France 5 Hijazi, H. France 5 González–Barahona, J.M. Spain 5 Milne, David New Zealand 5 Witten, Ian H. New Zealand 5 Wong, K.L. Malaysia 5
Citations to Wikipedia by affiliated institutions
As illustrated below, authors affiliated with institutions in the U.S. appear to cite Wikipedia more often in their scholarly publications than authors in any other country. Researchers affiliated with Carnegie Mellon University, Georgia Institute of Technology, and Indiana University were most active in citing Wikipedia. The most highly citing affiliated institutions are rank ordered in Table 7. International researchers affiliated with universities in Asian countries — Nanyang Technological University, University of Hong Kong, Tsinghua University, and Chinese University of Hong Kong — cited Wikipedia most frequently. Nanyang Technological University, Carnegie Mellon University, Indiana University, and Tsinghua University were also listed among the 15 institutions which are most productive in Wikipedia research as well.
Table 7: Institutions whose researchers cite Wikipedia most frequently. Institution Number of citations Carnegie Mellon University 23 Georgia Institute of Technology 19 Indiana University 17 Institute of Electrical and Electronics Engineers (IEEE) 16 Nanyang Technological University 15 University of Hong Kong 15 Purdue University 15 New York University 15 Tsinghua University 15 Chinese University of Hong Kong 14 Arizona State University 14 University of California, Berkeley 14 University of California, Los Angeles 14
Citations to Wikipedia by country
Table 8 lists the number of citations to Wikipedia by country. Researchers from the U.S., China, United Kingdom, Germany and Canada most frequently cite Wikipedia according to Scopus while the U.S., United Kingdom, Canada, and Germany cite the most in the WoS database. For the United Kingdom’s total, an additional four citations from Scotland were added. Likewise, a combined search with “affilcountry” (Hong Kong) brought an additional 42 citations by researchers affiliated with institutions in Hong Kong in Scopus which were added into China. Scholars in the U.S., Germany, United Kingdom, China, and France were most active in generating research on Wikipedia while researchers affiliated in the U.S., United Kingdom, Germany, and China cited Wikipedia most often. American scholars are strong in both Wikipedia research and citing Wikipedia in their publications. However, a closer look reveals that U.S. scholars are more likely to cite Wikipedia than to actually produce research on Wikipedia itself. American scholars account for about 37 percent of published research on Wikipedia in WoS and 22 percent in Scopus whereas they produce 43 percent of the citations to Wikipedia in WoS and 27 percent in Scopus.
Table 8: Citations to Wikipedia by country. Country Numbers cited in WoS
(percent)Country Numbers cited in Scopus
(percent)United States 146
(43)United States 908
(27)United Kingdom
(including Scotland)19
(5.6)China
(including Hong Kong)212
(6.3)Canada 18
(5.3)United Kingdom 196
(5.8)Germany 14
(4.1)Germany 158
(4.7)Australia 11
(3.2)Canada 138
(4.1)Singapore 9
(2.7)Australia 116
(3.5)China 8
(2.4)France 78
(2.3)Taiwan 8
(2.4)Japan 75
(2.2)Austria 7
(2.1)Italy 63
(1.9)France 7
(2.1)Netherlands 62
(1.9)Netherlands 7
(2.1)Spain 46
(1.4)
Scholarly publications citing Wikipedia most often
The publications in WoS and Scopus which most cite Wikipedia were identified and are rank ordered in Table 9. Among the 22 publications that produced the most research about Wikipedia, four — namely Lecture Notes in Computer Science (with subseries), Proceedings of the International Symposium on Wikis, First Monday and Journal of the American Society for Information Science and Technology — also cited Wikipedia most frequently. Interestingly, in WoS the 10 most frequently citing publications contain about 12 percent of the total citations to Wikipedia while the 11 publications most active in producing Wikipedia research comprise about 20 percent of the publications about it. Likewise, in Scopus the 10 most highly citing publications contain only 12 percent of the relevant citations whereas the top 11 publications on Wikipedia research contain about 37 percent of pertinent publications. Wikipedia research is highly concentrated in a relatively few publications whereas citations to Wikipedia are scattered among a larger number of diverse publications in both WoS and Scopus. Thus, Wikipedia’s impact on scholarly communications appears to be stronger through citations to it rather than through publications about it.
Table 9: Publications in WoS and Scopus which cite Wikipedia most often. Publications in WoS Number of publications
(percent)Publications in Scopus Number of publications
(percent)Lecture Notes in Computer Science 11
(3.2)Lecture Notes in Computer Science, including the subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics 202
(6)Journal of American Society for Information Science and Technology 6
(1.8)Proceedings pf the International Symposium on Wikis 33
(1)Publications of the Modern Language Association of America (PMLA) 5
(1.5)Proceedings of SPIE (International Society for Optical Engineering) 30
(<1)Computers & Security 4
(1.2)Proceedings of ACM International Conference Series 28
(<1)Explore: The Journal of Science and Healing 4
(1.2)Conference Proceedings of the American Society for Engineering Education (ASEE) 26
(<1)AAA — Arbeiten aus Anglistik und Amerikanistik 3
(0.9)First Monday 21
(<1)Journal of Universal Computer Science 3
(0.9)Proceedings of the International Conference on Information and Knowledge Management 20
(<1)Athletic Therapy Today 2
(0.6)Journal of the American Society for Information Science and Technology 15
(<1)Biochemistry and Molecular Biology Education 2
(0.6)Communications in Computer and Information Science 14
(<1)Clinical Orthopedics and Related Research 2
(0.6)Proceedings of the ACM Conference on Human Factors in Computing Systems 13
(<1)
Academic fields citing Wikipedia most often
Table 10 displays the 12 academic fields which cite Wikipedia the most often as noted in WoS and Scopus. About 16 percent of the citations to Wikipedia originate from computer science fields, about 10 percent from information and library science, about six percent from literature, and about four percent from communications and engineering in WoS. In Scopus, about 42 percent of citations come from computer science, 24 percent from engineering, and another 21 percent from the social sciences. The computer science field displays both the highest proportion of Wikipedia research and citations to Wikipedia. The fields of engineering (24 percent), and medicine (14 percent) are quite active in citing Wikipedia in their publications. In contrast, 14 percent of Wikipedia research derives from engineering, and three percent from medicine. Mathematicians contribute a larger proportion of the Wikipedia research (23 percent) than the citations to it (11 percent). The proportions are nearly equal for social scientists who produce 18 percent of the Wikipedia research and 21 percent of the citations. In the arts and humanities the proportion of citations to Wikipedia (about four percent) is also greater than the proportion of research publications about Wikipedia (about one percent). Remember that a publication may be assigned to more than one subject category so citation counts by fields may include duplicates.
Table 10: Academic fields citing Wikipedia most frequently. Academic fields identified in WoS Number of citations
(percent)Academic fields identified in Scopus Number of citations
(percent)Information science and Library science 34
(9.9)Computer science 1,419
(42.5)Computer science, Information systems 27
(7.9)Engineering 797
(23.8)Literature 19
(5.5)Social sciences 711
(21.3)Computer science, theory and methods 17
(4.9)Medicine 483
(14.5)Communications 13
(3.8)Mathematics 366
(10.9)Engineering, electrical and electronic 13
(3.8)Biochemistry, Genetics and Molecular biology 183
(5.5)Computer science, software engineering 11
(3.2)Arts and Humanities 149
(4.5)Education and Education research 11
(3.2)Business, Management and Accounting 139
(4.2)Law 9
(2.6)Physics and Astronomy 139
(4.2)Humanities, Multidisciplinary 8
(2.3)Material science 109
(3.3)Language and Linguistics 8
(2.3)Decision science 102
(3.1)Languages 8
(2.3)Nursing 83
(2.5)
Wikipedia’s increasing visibility in scholarly communications
Scholarly research about Wikipedia apparently first appeared in the 3 June 2002 issue of First Monday, in paper entitled “Open source intelligence” by Felix Stalder and Jesse Hirsh [4] as well as in a 2002 article in Online entitled “Péter’s picks and pans review on Wikipedia” by Péter Jacsó [5]. Table 11 summaries the pertinent data about Wikipedia in WoS and Scopus from 2002 to 2010. As the Table 11 illustrates, research about and citations to Wikipedia in scholarly publications have steadily increased over time since its launch in 2001. Although citations to Wikipedia in WoS peaked in 2007, there is substantial evidence in citation patterns to demonstrate the significant impact of Wikipedia on scholarly communication over the past decade, corresponding to its increased use as an information resource.
Table 11: Research about Wikipedia and citations to Wikipedia, by year. Year Number of research publications identified in Scopus Number of citations identified in Scopus Number of research publications identified in WoS Number of citations identified in WoS 2002 2 0 2 1 2003 0 4 0 0 2004 3 39 0 10 2005 19 97 7 24 2006 80 303 22 70 2007 209 491 33 81 2008 340 592 65 57 2009 390 880 76 48 2010 412 933 86 49
Tables 12 and 13 present data about the types of publications that, respectively, write about and cite Wikipedia. Table 12 shows that in Scopus research about Wikipedia has been published predominantly in conference papers (63 percent), articles (26 percent), and review papers (three percent) among other formats, while in WoS it has been published more frequently in articles (65 percent), proceeding papers (nine percent), and editorial materials (seven percent). However, Wikipedia tends to be more highly cited in journal articles as shown in Table 13: 30 percent in Scopus and 70 percent in WoS. Only seven percent of the citations in WoS were to conference papers, contrasted to 31 percent in Scopus. In summary, the visibility of Wikipedia research is more prominent in conference and proceedings papers while citations to Wikipedia are more prevalent in journal articles.
Table 12: Types of publications publishing research about Wikipedia. Document type identified in WoS Number
(percent)Document type identified in Scopus Number
(percent)Articles 188
(65.0)Conference papers 921
(63.2)Proceedings papers 25
(8.7)Articles 385
(26.4)Editorial material 21
(7.2)Reviews 47
(3.2)Book reviews 15
(5.2)Conference reviews 46
(3.1)Letters 11
(3.8)News items 10
(3.4)Other 21
(7.2)Other 56
(3.8)
Table 13: Types of publications citing Wikipedia. Document type identified in WoS Number
(percent)Document type identified in Scopus Number
(percent)Articles 239
(70.3)Conference papers 1,046
(31.3)Editorial material 39
(11.5)Articles 995
(29.8)Proceedings papers 24
(7.1)Reviews 258
(7.7)Reviews 19
(5.5)Editorial material 54
(1.6)Book reviews 12
(3.5)Short surveys 21
(<1)Letters 6
(1.7)Notes 17
(<1)
Since Wikipedia was launched in 2001, the number of research publications about Wikipedia and citations to Wikipedia has increased steadily. There were a total of 1,746 publications included in WoS and Scopus for the years 2002 to 2010.
Research about Wikipedia has been published most frequently by individual researchers who are affiliated with academic institutions in Europe and Asian countries — Netherlands, Germany, Australia and Japan. However, the largest proportion of research on Wikipedia has been contributed by scholars in academic institutions in the U.S. (about 37 percent in WoS and 22 percent in Scopus), followed by scholars from Germany, United Kingdom, and China. Researchers in universities are the major contributors to Wikipedia research. The University of Amsterdam in the Netherlands and the Max–Planck–Institut für Informatik in Germany were the most active in producing research on Wikipedia. Analysis by discipline shows that the most frequent contributors to Wikipedia research are computer scientists, information scientists, and mathematicians. For example, the Lecture Notes in Computer Science (with subseries) and Proceedings of the International Symposium on Wikis (with variant titles) have published more Wikipedia research than any other publications. Conference publications and journal articles are the major venues for reporting research on Wikipedia.
Wikipedia’s citation rates in scholarly publications have been consistently increasing. It was cited 3,679 times in the WoS and Scopus databases during the last nine years. Academic institutions are not only the major producers of Wikipedia research but also the major consumers that cite Wikipedia most often. The rate of citing was highest among scholars from the US, United Kingdom, Germany, and China. Wikipedia has been cited in more than 30 countries and by 306 institutions worldwide in WoS alone. Authors affiliated with academic institutes in the U.S. appear to cite Wikipedia most frequently. American scholars tended to cite Wikipedia to a greater extent than they published research about it. Researchers affiliated with Carnegie Mellon University, Georgia Institute of Technology, and Indiana University were the most active in citing Wikipedia in their publications. Scholars in the fields of computer science, information science and social sciences are the most active in citing Wikipedia. Interestingly, researchers in engineering and medicine cite more often than do research on Wikipedia, while researchers in mathematics more often write about Wikipedia than cite it. Arts and humanities also give more citations to Wikipedia than conduct research about it. Wikipedia research is most likely to be published in conference and proceedings papers, then journal articles along with other formats. However, citations to Wikipedia were more often found in journal articles followed by conference papers and then editorial materials. A few publications contain a high portion of Wikipedia research while citations were scattered in a wider range of publications. The breath of Wikipedia’s impact has stretched to authors in many fields and professional areas.
Reported numbers regarding the writing about and citing of Wikipedia should be taken carefully as they reflect only a snapshot provided by several databases. Since this research is based only on WoS and Scopus, publications included in these databases are mostly in English. Finally, book reviews, editorial material, letters, and news items (which constitute a significant portion of publications about Wikipedia in WoS) are not strictly speaking “research,” but they nevertheless are indicative of Wikipedia’s impact on the scholarly communication.
This research adds to our understanding of Wikipedia’s role in scholarship and reflects scholarly regard in some sense for a highly controversial yet well used resource on the Internet. This bibliometric study demonstrates Wikipedia’s visibility in the scholarly communication process — productivity of scholars, affiliated institutions, academic fields, and the geographic distribution of affiliated institutions, and the type of publications. The influence of Wikipedia on the scholarly community as indicated by citations was identified in the course of this research. Hence this paper sheds some light on trends regarding Wikipedia’s place in formal scholarship and demonstrates its growing visibility.
Recent involvement by higher education communities in Wikipedia implies Wikipedia’s potential to become not only a reliable resource but also a learning and teaching tool for students. Wikipedia’s plans to include more women and elderly as well as expanding international offices will bring balance and wholeness in content. As demonstrated in this study, active research on Wikipedia and citations to Wikipedia testifies to Wikipedia position as a rich resource. The increasing scholarly attention to Wikipedia suggests a growing acceptance of its credibility as a valid information resource.
This study is only a small step in demonstrating the visibility of Wikipedia in scholarly communication. Identifying major topics covered in scholarly publications about Wikipedia may be addressed in future research. Other issues — such as examining gender differences, co–author networks in Wikipedia research, and motivations for citing Wikipedia — could add further details on the utility of Wikipedia in scholarship.
About the author
Taemin Kim Park, Ph.D., is an Associate Librarian of Indiana University Libraries and Adjunct Faculty in the School of Library and Information Science at Indiana University, Bloomington.
Acknowledgements
This research was conducted during the author’s research leave which was partially supported by Indiana University Libraries.
Notes
1. Web of Science (WoS) Web site, at http://thomsonreuters.com/products_services/science/science_products/a-z/web_of_science/, accessed 24 August 2010.
2. Scopus Web site, at http://www.info.sciverse.com/scopus/, accessed 24 August 2010.
3. Steve Hinnefeld, 2010. “SPEA seminar producing policy articles for Wikipedia,&lrquo; at http://homepages.indiana.edu/web/page/normal/15707.html, accessed 21 July 2011.
4. Felix Stalder and Jesse Hirsh, 2002. “Open source intelligence,” First Monday, volume 7, number 6, at http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/961/882, accessed 21 July 2011.
5. Péter Jacsó, 2002. “Péter’s picks & pans: Pans: Wikipedia,” Online, volume 26, number 2, pp. 81–82.
References
Kathleen Bauer and Nisa Bakkalbasi, 2005. “An examination of citation counts in a new scholarly communication environment,” D–Lib Magazine, volume 11, number 9, at http://www.dlib.org/dlib/september05/bauer/09bauer.html,accessed 25 April 2011.
Erik W. Black, 2008. “Wikipedia and academic peer review: Wikipedia as recognized medium for scholarly publication?” Online Information Review, volume 32, number 1, pp. 73–88.
Lutz Bornmann and Hans–Dieter Daniel, 2008. “What do citation counts measure? A review of studies on citing behavior,” Journal of Documentation, volume 64, number 1. pp. 45–80.
Noam Cohen, 2009. “New rules in wiki world,” New York Times (25 August), at http://query.nytimes.com/gst/fullpage.html?res=9501E5DC1430F936A1575BC0A96F9C8B63, accessed 20 August 2010.
Noam Cohen, 2007. “A history department bans citing Wikipedia as a research source,” New York Times (21 February), at http://www.nytimes.com/2007/02/21/education/21wikipedia.html, accessed 20 August 2010.
Blaise Cronin and Debora Shaw, 2007. “Peers and spheres of influence: Situating Rob Kling,” Information Society, volume 23, number 4, pp. 221–233.
Beate Elvebakk, 2008. “Philosophy democratized? A comparison between Wikipedia and two other Web–based philosophy resources,” First Monday, volume 13, number 2, at http://www.uic.edu/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/2091/1938, accessed 30 August 2010.
Don Fallis, 2008. “Toward an epistemology of Wikipedia,” Journal of the American Society for Information Science and Technology, volume 59, number 10, pp. 1,662–1,674.
Ylva Gavel and Lars Iselid, 2008. “Web of Science and Scopus: A journal title overlap study,” Online Information Review, volume 32, number 1, pp. 8–21.
Jim Giles, 2005. “Internet encyclopaedias go head to head,” Nature, volume 438, number 7070 (15 December), pp. 900–901, and at http://www.nature.com/nature/journal/v438/n7070/full/438900a.html, accessed 20 August 2010.
Steve Hinnefeld, 2010. “SPEA seminar producing policy articles for Wikipedia,&lrquo; at http://homepages.indiana.edu/web/page/normal/15707.html, accessed 21 July 2011.
Todd Holloway, Miran Bozicevic, and Katy Börner, 2007. “Analyzing and visualizing the semantic coverage of Wikipedia and its authors,” Complexity, volume 12, number 3, pp. 30–40.
Péter Jacsó, 2002. “Péter’s picks & pans: Pans: Wikipedia,” Online, volume 26, number 2, pp. 81–82.
Leonard Kniffel, 2008. “Authority and Wikipedia,” American Libraries, volume 39, number 7, p. 4.
Verne G. Kopytoff, 2011. “BITS; Celebrating 10 years of Wikipedia,” New York Times (17 January), at http://query.nytimes.com/gst/fullpage.html?res=9407E1D6133CF934A25752C0A9679D8B63&scp=2&sq=&st=nyt, accessed 30 January 2011.
Ann M. Lally and Carolyn E. Dunford, 2007. “Using Wikipedia to extent digital collections,” D–Lib Magazine, volume 13, numbers 5–6, at http://www.dlib.org/dlib/may07/lally/05lally.html, accessed 24 October 2010.
Greg Landgraf, 2009. “Wikipedia growth slows,” American Libraries, volume 40, number 11, p. 27.
Sook Lim, 2009. “How and why do college students use Wikipedia?” Journal of the American Society for Information Science and Technology, volume 60, number 11, pp. 2,189–2,202.
Olena Medelyan, David Milne, Catherine Legg, and Ian H. Witten, 2009. “Mining meaning from Wikipedia,” International Journal of Human–Computer Studies, volume 67, number 9, pp. 716–754.
Lokman I. Meho and Cassidy R. Sugimoto, 2009. “Assessing the scholarly impact of information studies: A tale of two citation databases — Scopus and Web of Science,” Journal of The American Society for Information Science and Technology, volume 60, number 12, pp. 2,499–2,508.
Finn Årup Nielsen, 2007. “Scientific citations in Wikipedia,” First Monday, volume 12, number 8, at http://www.uic.edu/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/1997/1872, accessed 30 August 2010.
Lauren Pressley and Carolyn J. McCallum, 2008, “Putting the library in Wikipedia,” Online, volume 32, number 5, pp. 39–42.
Lucy Holman Rector, 2008, “Comparison of Wikipedia and other encyclopedias for accuracy, breath, and depth in historical articles,” Reference Services Reviews, volume 36, number 1, pp. 7–22.
Scopus, 2010. Scopus, at http://www.scopus.com, accessed 26 October 2010.
Anselm Spoerri, 2007. “What is popular on Wikipedia and why?” First Monday, volume 12, number 4, at http://www.uic.edu/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/1765/1645, accessed 22 February 2011.
Felix Stalder and Jesse Hirsh, 2002. “Open source intelligence,” First Monday, volume 7, number 6, at http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/961/882, accessed 21 July 2011.
Besiki Stvilia, Michael B. Twidale, Linda C. Smith, and Les Gasser, 2008. “Information quality work organization in Wikipedia,” Journal of the American Society for Information Science and Technology, volume 59, number 6, pp. 983–1,001.
Danny P. Wallace and Connie Van Fleet, 2005. “The democratization of information? Wikipedia as a reference source,” Reference & User Services Quarterly, volume 45, number 2, pp. 100–103.
Web of Science, 2010. Web of Science, at http://thomsonreuters.com/products_services/science/science_products/a-z/web_of_science/, accessed 24 August 2010.
Dennis M. Wilkinson and Bernado A. Huberman, 2007. “Assessing the value of cooperation in Wikipedia,” First Monday, volume 12, number 4, at http://www.uic.edu/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/1763/1643, accessed 20 August 2010.
Editorial history
Received 16 March 2011; revised 29 April 2011; accepted 22 July 2011.
Copyright © 2011, First Monday.
Copyright © 2011, Taemin Kim Park.The visibility of Wikipedia in scholarly publications
by Taemin Kim Park.
First Monday, Volume 16, Number 8 - 1 August 2011
http://www.firstmonday.dk/ojs/index.php/fm/article/view/3492/3031
A Great Cities Initiative of the University of Illinois at Chicago University Library.
© First Monday, 1995-2013.