Note:
CVP(Critical Value Pruning) is also called
Chi-Square Pruning(test) in many materials.

The following is a contingency table[1]:

$$H_0:\frac{X_{ij}}{n}=\frac{N_i{N}_j}{n^2}$$
$$H_1:\frac{X_{ij}}{n}=\frac{N_i{N}_j}{n^2}$$
$N_{ij}=X_{ij}$
when
$$\sum _{i=1}^{i=r}\sum _{j=1}^{j=s}\frac{(N_{ij}?\frac{N_i?N_j}{n}{)}^2}{\frac{N_i?N_j}{n}}<{\chi }_{[(r?1)(s?1)],\alpha }^2=criticalvalue$$
then $H_0$ is accepted and the decision tree is pruned.
among which $\alpha$ can be set as 0.05,etc.
$n$ is the total length(means counts,quantity) of your datasets.
The relationships between contingency table and Decision Tree are listed in the following table:

Split node with Attribute $f$class $1$class $2$$\dots$class $s$

branch $1$	$n_{L1}$	$n_{L2}$	$\dots$	$n_L$
branch $2$	$n_{R1}$	$n_{R2}$	$\dots$	$n_R$
$?$	$?$	$?$	$?$	$?$
branch $r$	$n_1$	$n_2$	$\dots$	$n$

Now let’s use the above table to learn the following lecture PPT[2].

in the above picture,some parameters are explained in the table in front of it.

Let’s go on…

In the above ppt,note that CVP can both be used in pre-pruning and post-pruning stages.
According to the growth stage ,we know “l(fā)ess than critical value” happend before current sub-tree is pruned.
So,we can infer from the above lecture PDF that the post-pruning will have the same experience.

How to understand the above pruning criterion?

------------------------------------------
In above table,
different branches
=different value level of current decision node of decision tree
（also called split node,one split node owns one Attribute of datasets）

when $H_0$ is accepted,then
$\frac{X_{ij}}{N_{i?}}\approx \frac{N_{?j}}{n}$
which means:

the probability of “items belongs to class j” in each $i_{th}(i\in [1,r])$ branch
=the probability of “items belongs to class j” in all datasets
=>Merging(prune) these branches into one leaf will Not make the probability of “items belongs to class j”vary too much,which means that accuracy will not vary too much after being pruned

In conclusion,when Chi-Square Statistics do Not reach the Critical Value,branches(different value levels of split Attribute of Decision Tree)
will not contribute too much for increasing the accuracy,then these branches can be pruned.

The above conclusion can be used directly when we implement our CVP(Critical Value Pruning) algorithm with python.

We can also learn from above analysis that CVP is targeted at simplifying your decision tree while Not losing accuracy too much.

Reference:
[1]http://www.maths.manchester.ac.uk/~saralees/pslect8.pdf
[2]https://www.docin.com/p1-2336928230.html

總結(jié)

以上是生活随笔為你收集整理的CVP(Critical Value Pruning)illustration with clear principle in details的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。