Google Scholar - Limitations and Criticism

Limitations and Criticism

Access to publications — Use of Google Scholar does not automatically grant searchers access to publications available through subscription sites. The full texts of articles in Google Scholar are not necessarily available freely to all searchers, though searchers with access through an institution such as a research laboratory or university may be able to access select articles freely. Some articles found through Google Scholar are hosted by sites that allow searchers to subscribe or purchase the full text of their articles.

Quality — Some searchers consider Google Scholar of comparable quality and utility to commercial databases. The reviews recognize that its "cited by" feature in particular poses serious competition to Scopus and ISI Web of Knowledge, although, in a study limited to the biomedical field, the citation information found in Google Scholar have found to be sometimes inadequate, and less often updated. Another important issue is that the relative coverage of Google Scholar varies by discipline compared to other general databases.

Secrecy about coverage — A significant problem with Google Scholar is the secrecy about its coverage. Some publishers do not allow it to crawl their journals. Elsevier journals were not included before mid-2007, when Elsevier began to make most of its ScienceDirect content available to Google Scholar and Google's web search. As of February 2008 the absentees still included the most recent years of the American Chemical Society journals. Google Scholar does not publish a list of scientific journals crawled, and the frequency of its updates is unknown. It is therefore impossible to know how current or exhaustive searches are in Google Scholar. Nonetheless, it allows easy access to published articles without the difficulties encountered in some of the most expensive commercial databases.

Matthew effect — Google Scholar puts high weight on citation counts in its ranking algorithm and therefore is being criticised for strengthening the Matthew effect; as highly cited papers appear in top positions they gain more citations while new papers hardly appear in top positions and therefore get less attention by the users of Google Scholar and hence fewer citations.

Incorrect field detection — Google Scholar has problems identifying publications on the arXiv preprint server correctly. Interpunctuation characters in titles produce wrong search results, and authors are assigned to wrong papers, which leads to erroneous additional search results. Some search results are even given without any comprehensible reason.

Vulnerability to spam — Google Scholar is vulnerable to spam. Researchers from the University of California, Berkeley and Otto-von-Guericke University Magdeburg demonstrated that citation counts on Google Scholar can be manipulated and complete non-sense articles created with SCIgen were indexed from Google Scholar. They concluded that citation counts from Google Scholar should only be used with care especially when used to calculate performance metrics such as the h-index or impact factor. Although Google Scholar does not compute the h-index itself, several downstream web sites use its data. The practicality of manipulating h-index calculators by spoofing Google Scholar was demonstrated in 2010 by Cyril Labbe from Joseph Fourier University, who managed to rank "Ike Antkare" ahead of Albert Einstein by means of a large set of SCIgen-produced documents citing each other (effectively an academic link farm).

Google Scholar is also not able to shepardize case law, as Westlaw and Lexis can.