15.1.2 Evaluation measuers

• precision($BE,9gN((B), recall($B:F8=N((B)
  
  $$\begin{eqnarray*}precison ($BE,9gN((B) &=& \frac{$B8!:w7k2L$N(B document $BCf$N@52r?t(B} {.. ...=& \frac{$B8!:w7k2L(B document $BCf$N@52r?t(B} {$BA4(B document $B$N@52r?t(B} \end{eqnarray*}$$
  
  
  • query $B$KBP$7$F(B,$B$9$Y$F$N(B document $B$r@52r$H=PNO$9$k(B system $\rightarrow$ recall 100%
  • query $B$KBP$7$F(B $B@5$7$$(B document $B$?$@(B1$B$D$rEz$($k(B system $\rightarrow$ precison 100%
  $B0lHLE*$K(B trade off $B$N4X78(B
• CUTOFF
  • N $B0L(B $B$N(B precison
    
    $$\begin{eqnarray*}precison(N) = \frac{N$B$h$j>e0L$N@52r?t(B}{N} \end{eqnarray*}$$
    
    
  • N $B$r(B cutoff$B$H$$$&(B (5,10,20,100)
  ($BNc(B Table 15.2)
• Uninterpolated average precision
  
  $$\begin{eqnarray*}S=$B8!:w7k2L$N$&$A@52r(B document $B$N=g0L$r$$$
  
  ($BNc(B Figure 15.2)
  ranking 3 $B$N(B Uninterpolated average precison $B$O(B?
  
  $$\begin{eqnarray*}Avg\{1/2(d1), 2/3(d2), 3/6(d3), 4/7(d5), 5/8(d4)\} = 0.57256 \end{eqnarray*}$$
  
  
• Interpolated average precison (11-point)
  • $B0lHLE*$JI>2A4p=`(B, recall $B$r9MN8(B
  • recall 0% 10% 20% $\cdots$ 100% (11 points) $B$K$*$1$k(B precision $B$NJ?6QCM(B
  • recall $B$r(B $\alpha$ $B$H$$$&CM$K8GDj$7$F(B $B$=$N;~$N(B precison $\beta$ $B$r5a$a$k$3$H$O0lHLE*$K:$Fq(B
    • $B@52r$,(B 2$B$D$7$+L5$$>l9g$O(B, recall $B$O(B 0,50,100% $B$N(B3 $BDL$j$7$+$J$$(B
      $B@52r$,==J,$J8D?t$"$l$P2DG=(B
    • precison $B$,C1D48:>/$7$J$$>l9g(B(Figure 15.2$B>e(B),
      $B$3$l$^$G$N:GBgCM$r(B precison $B$H$9$k(B
  • interpolate (Figure 15.2$B2<(B)
• precision-recall curves
  • $B0lHLE*$K1&2<$,$j(B
  • trade off $B$N4X78(B
• F measures
  
  
  $$\begin{eqnarray*}F = \frac{1}{\alpha\frac{1}{P}+(1-\alpha)\frac{1}{R}} \ \ (P=precision, R= recall, 0 \leq \alpha \leq 1) \end{eqnarray*}$$
  
  
• $B$3$l$i$NCM$O(B, $B8!:w(Bsystem $B$N@-G=I>2A$KMQ$$$i$l$k(B,
  $B$7$+$7(B,$B6vA3(B, system $B$K0MB8$7$?(B query $BEy$,860x$G(B, $B@dBPE*$JI>2A4p=`$H$J$j$&$k$H$O8B$i$J$$(B $\rightarrow$ t$B8!Dj$J$I$rMQ$$$k(B
• $BM>CL(B
  TREC $B$N(B tool $B$K$3$l$i$NI>2ACM$r7W;;$9$k(B program $B$,$"$j$^$9(B