Relevance can be system- or user-based

There are two classes of relevance: 1. Objective or system-based relevance and 2. Subjective or user-based relevance.

System relevance: Logical relevance, the relationship between the query and the results. It is defined by “how well the topic of the information retrieved matches the topic of the request. A document is objectively relevant to a request if it deals with the topic of the request (Harter, 1992, p. 602). This requires you to have a firm grasp of what the topic of something is. You can measure this by identifying the number of features in common between the query and the information objects, and give each resulting information object a decimal value between 0 and 1.

User relevance is concerned with the aboutness and appropriateness of a retrieved information object.

• Topicality: The degree to which an information object is about the topic of the query. This is traditionally measured by an assessor, who makes a judgement about how topically related a query and and information object are. Because it is done by trained assessors, it’s subjective, but not entirely contextual to an individual.
• Pertinence: The relationship between the intrinsic information need represented by the query and the information objects retrieved. This can be measured by user interviews, approximated by expert assessment. It is generally limited by the quality of the information need expression.
• Situational: The utility or usefulness of the information objects for the user’s task at hand. Measured by user interviews. In practice, situational and pertinence relevance are hard to distinguish.

People often express information needs in imprecise ways. “Can glasses be fixed?” “books about Washington.” Disambiguating the query is necessary to push the relevance of content we can show them down from just logical relevance to more meaningful measures.

References

Borlund, P. (2003). The concept of relevance in IR. Journal of the American Society for Information Science and Technology, 54(10), 913–925. https://doi.org/10.1002/asi.10286