The Primo ScholarRank Technology: Bringing the Most Relevant Results to the Top of the List

By Tamar Sadeh, Director of Marketing

In today’s world, users’ expectations for a quick and easy search process, combined with an information landscape as large and complex as that covered by the Primo Central Index, render sophisticated relevance-ranking algorithms crucial to the success of the discovery process. In addition to the traditional assessment of the degree to which a retrieved item matches a user’s query, relevance-ranking algorithms need to take into account factors that relate to the academic significance of the retrieved item and to the context of the query: who submitted the query and what information need led the user to submit that query.

In March 2011, Ex Libris initiated a relevance-ranking project to enrich and optimize the original Primo® relevance-ranking algorithms. The algorithms that have thus far resulted from this project constitute the Ex Libris ScholarRank™ technology.

The project team includes members of the Ex Libris research and development staff and information-retrieval specialists. In addition, input from researchers who are located all over the world and work in various disciplines has helped the team establish metrics for the evaluation of the improvements that are made to the algorithms. In-depth information about the relevance-ranking project is available in a white paper, which you can obtain from your account manager.

What Is Primo ScholarRank?

Although relevance-ranking algorithms are not new in the context of information retrieval (IR) systems, the Ex Libris R&D team realized early on in the development of the Primo discovery and delivery solution that for optimal application to scholarly data, traditional IR algorithms would have to be adjusted and enhanced considerably. The current Primo relevance-ranking project is equipping the algorithms with new capabilities, which take into account a user’s background and information needs as well as the global scholarly significance of materials. The latter aspect is expressed as a measure of various factors, such as the number of citations that a publication has generated and usage information that reflects scholars’ interest in the publication. Along with these enhancements, the project team is adding a self-learning mechanism that feeds data back to the algorithms and helps the system constantly improve the order of search results over time. Together, all these features constitute the ScholarRank relevance-ranking technology; some of the features are already deployed by the Primo solution, and others will be implemented in 2012.

To determine the position of an item on a result list, ScholarRank is designed to take into account the following three elements:

• The degree to which the item matches the query
• A score representing the item’s scholarly value (referred to as the ScholarRank value score)
• Information about the user and the user’s research need at the specific point in time

The match between a query and an item is calculated according to IR methods that have been adapted to the structure of the specific type of information (metadata, abstract, or full text). Not only do the proximity and order of the query terms in a result record have an impact on the ranking, but the field in which the query terms appear also has an effect; for example, if the terms appear in an item’s title, the item is likely to be more relevant to the user than an item for which the query words appear only in the full text. Furthermore, specific types of materials are typically more likely to satisfy user needs; for example, when all else is equal, a journal article is ranked higher than a newspaper article and a recent publication is ranked higher than an older one.

The ScholarRank value score represents an evaluation of an item’s academic significance regardless of the degree to which the item matches the query. To calculate the value score, the Primo ScholarRank technology relies on usage metrics derived from the bX article recommender database and other data, such as the item’s citation information.

The Primo ScholarRank technology also considers certain characteristics of a user to provide personalized ranking. Applying information about the user’s area of research, ScholarRank boosts materials related to the user’s discipline when the topic that is inferred from the query is ambiguous. Information about the user’s academic degree enables ScholarRank to boost materials that would be considered appropriate for that level; for instance, for a query submitted by a researcher who holds a Ph.D., in‑depth items would be among the highest ranked.

Finally, a user’s specific information need (a particular item or materials on a particular subject) is factored into the relevance-ranking equation. By analyzing a query, the Primo ScholarRank technology “infers” the user’s need and adapts to the type of search (a known-item search, narrow-topic search, broad-topic search, or author-related search). For example, in a broad-topic search, reference materials or review articles are likely to be more relevant to the user than an article dealing with a specific aspect of the subject matter.

Looking Ahead

Awareness of the huge impact of relevance ranking on the success of the discovery process has brought the ScholarRank technology to the forefront of research at Ex Libris.

The goal of the work invested in the Primo relevance-ranking algorithms is to enable academic users to find the exact scholarly materials that they need—and find them quickly. By shortening users’ discovery time, Primo improves their productivity, draws more traffic to the library site, and helps achieve optimal use of library collections. As a result, Primo enables libraries to better serve their community and their institution’s mission and to gain the prominence that they deserve in the provision of scholarly information.

The research and development work on the ScholarRank technology is an ongoing effort and will continue to introduce enhancements. Additional methods of personalizing relevance ranking will be added to the algorithms, as well as more features drawn from relationships between researchers, authors, and scholarly materials.