The effect of refinement:

We can immediately see the positive effect of the refinement mechanism. The last two rows provide the F1-score of the system for different parameters values without and with the use of exception rules. Refinement improves all results, the F1-score gaining between 1 and 2.5%, which is a very good increase for this task. More interestingly, refinement is especially efficient for $\theta=0.50$ since the gain is +2.49 and then provides the best F1-score, outperforming all the other results without refinement. Results deteriorate for higher values of $\theta$ (0.90). This may be due to the fact that $\theta$ is too close or greater than the noise level. One other interesting result is the flattening of the F1-scores. Without refinement rules, the variation of F1-score is much significant and more correlated to $\theta$. The use of refinement lessens these variations. This result meets our expectation. We can then conclude that refinement is very helpful since it improves results uniformly and gives $\theta$ less importance. This is an interesting issue since such a threshold is a major parameter of inductive systems and has a significant impact on results.

In traditional top-down induction systems, a high value of $\theta$ provides a high precision, but can provide a poor recall. Here, precision and recall are always close one another, even for ``extreme'' values such as 0.5 (P:90.72, R:90.69) or 0.9 (P:88.67, R:89.20). Without refinement the difference between precision and recall increases: P:87.50, R:88.94 for $\theta=0.5$. To increase the size of the context (3 elements) marginally improves results.

The value $\theta$=0.75 provides the best tradeoff between accuracy and the number of rules (and also the best result without using refinement). When $\theta$ is low (for example 0.5), exceptions of exceptions appear and even exceptions of exceptions of exceptions. We have never observed a recursion level greater than 3. Of course, the lower the value of $\theta$, the higher the number of exceptions. When $\theta=0.90$ exceptions represent 16% of the rules, when $\theta=0.50$, they represent 60%.