Citation metrics and evaluation of journals and conferences

Abstract

Citation analysis aims at evaluating the published scientific manuscripts, their authors and the publication venues (journals/conferences). There are several popular metrics for measuring the impact of the journals, the Impact Factor (IF) being the most popular. Similarly, the <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:mi mathvariant="normal">H</mml:mi> </mml:mrow> </mml:math> -index is a popular metric for evaluating and ranking conferences. We have presented a review of metrics for citation analysis, categorised according to their applicability for evaluating journals and conferences. The citation metrics may also be categorised as popularity measuring and prestige measuring. Prestige measuring indicators like SCImago Journal Rank (SJR) and Eigenfactor have already gained popularity for evaluating journals. We discuss their role in evaluating the conferences. Indeed, some conferences have already started mentioning their prestige score in terms of the SJR of their conference proceedings. We also propose a Normalised Immediacy Index ( <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:mi>I</mml:mi> <mml:msub> <mml:mrow> <mml:mi>I</mml:mi> </mml:mrow> <mml:mrow> <mml:mi>norm</mml:mi> </mml:mrow> </mml:msub> </mml:mrow> </mml:math> ), a variant of the Immediacy Index ( <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:mi>II</mml:mi> </mml:mrow> </mml:math> ), to measure the immediate relevance of articles published in a journal/conference. It is shown that the proposed metric can be used for immediacy relevance comparison irrespective of the publication schedule of the articles. Spearman correlation was run to determine the relationship between the values of the proposed <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:mi>I</mml:mi> <mml:msub> <mml:mrow> <mml:mi>I</mml:mi> </mml:mrow> <mml:mrow> <mml:mi>norm</mml:mi> </mml:mrow> </mml:msub> </mml:mrow> </mml:math> and traditional metrics ( <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:mi>H</mml:mi> </mml:mrow> </mml:math> -index for conferences and IF for journals). A strong, positive monotonic correlation was observed between <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:mi>I</mml:mi> <mml:msub> <mml:mrow> <mml:mi>I</mml:mi> </mml:mrow> <mml:mrow> <mml:mi>norm</mml:mi> </mml:mrow> </mml:msub> </mml:mrow> </mml:math> and H -index ( <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:msub> <mml:mrow> <mml:mi>r</mml:mi> </mml:mrow> <mml:mrow> <mml:mi>s</mml:mi> </mml:mrow> </mml:msub> </mml:mrow> </mml:math> = .67, <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:mi>n</mml:mi> </mml:mrow> </mml:math> = 17, <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:mi>p</mml:mi> </mml:mrow> </mml:math> &lt; .01) for conferences and between <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:mi>I</mml:mi> <mml:msub> <mml:mrow> <mml:mi>I</mml:mi> </mml:mrow> <mml:mrow> <mml:mi>norm</mml:mi> </mml:mrow> </mml:msub> </mml:mrow> </mml:math> and IF ( <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:msub> <mml:mrow> <mml:mi>r</mml:mi> </mml:mrow> <mml:mrow> <mml:mi>s</mml:mi> </mml:mrow> </mml:msub> </mml:mrow> </mml:math> = .65, <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:mi>n</mml:mi> </mml:mrow> </mml:math> = 20, <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" overflow="scroll"> <mml:mrow> <mml:mi>p</mml:mi> </mml:mrow> </mml:math> &lt; .01) for journals.