next up previous
Next: Evaluation: Up: First Experiment Previous: First Experiment

Presentation of the task and evaluation:

In order to evaluate the impact of some parameters used by ALLiS, we conducted some experiments. The task here is the extraction of non-overlapping Noun Phrases (hereafter NP). This problem can be viewed as a problem of categorization using three categories: B-NP, I-NP and O. I-NP means that the element belongs to an NP, a B-NP is used when the current element belongs to an NP and the preceding belongs to another NP. We refer the reader to [Ramshaw and Marcus(1995)] for more information. Here is an example of such a segmentation with these three categories:
InO earlyI-NP tradingI-NP inO HongI-NP KongI-NP MondayB-NP ,O goldI-NP wasO quotedO atO $I-NP 366.50I-NP anB-NP ounceI-NP .O
In order to evaluate the system, we use two measures: precision and recall. The F1-score presented in [van Rijsbergen(1979)] offers a simple way to combine both (in our evaluation, we give the same weight to recall and precision: $\beta=1$).
$R= \frac{Number of correct proposed patterns}{Number of correct patterns}$ $P= \frac{Number of correct proposed patterns}{Number of proposed patterns}$

F$\beta$= $\frac{(\beta\raisebox{+3pt}{\small 2}+1)*R*P}{\beta\raisebox{+3pt}{\small 2}*R+P}$


next up previous
Next: Evaluation: Up: First Experiment Previous: First Experiment
Hammerton J. 2002-03-13