神经网络的收敛标准有最优值吗？
作一個5分類的三層網(wǎng)絡，分類9*9的圖片，收斂標準從0.5到6e-8，共47個收斂標準，每個收斂標準收斂199次，共收斂了47*199次。取平均值統(tǒng)計平均性能pave。觀察pave是如何隨著收斂標準改變的。
?
得到的表格
| f2[0] | f2[1] | f2[2] | f2[3] | f2[4] | 迭代次數(shù)n | 平均準確率p-ave | δ | 耗時ms/次 | 耗時ms/199次 | 耗時 min/199 | 最大值p-max | 平均值標準差 |
| 0.4854799 | 0.1858553 | 0.272492 | 0.2602853 | 0.2563134 | 1084.7739 | 0.6110079 | 0.5 | 70.015075 | 13949 | 0.2324833 | 0.8237011 | 0.2530133 |
| 0.5326435 | 0.12101 | 0.1954744 | 0.1860563 | 0.1582058 | 1643.7136 | 0.778164 | 0.4 | 78.79397 | 15680 | 0.2613333 | 0.8505546 | 0.0234568 |
| 0.6958971 | 0.0446005 | 0.1856664 | 0.1821425 | 0.1503321 | 1914.8794 | 0.8280271 | 0.3 | 83.59799 | 16654 | 0.2775667 | 0.8669002 | 0.0191745 |
| 0.7952586 | 0.0322357 | 0.1595491 | 0.1555259 | 0.1254544 | 2235.7487 | 0.8500412 | 0.2 | 89.135678 | 17738 | 0.2956333 | 0.8898618 | 0.0184869 |
| 0.3876615 | 0.0156595 | 0.0816177 | 0.0503286 | 0.5872436 | 3160.2261 | 0.8985676 | 0.1 | 104.77387 | 20850 | 0.3475 | 0.9208017 | 0.0088857 |
| 0.0611409 | 0.0047451 | 0.0363387 | 0.0046498 | 0.9119949 | 6747.4322 | 0.9403243 | 0.01 | 166.92462 | 33218 | 0.5536333 | 0.9453201 | 0.0018007 |
| 0.3821803 | 4.17E-04 | 0.0911098 | 0.0609155 | 0.4675613 | 28730.467 | 0.9551943 | 0.001 | 550.80905 | 109626 | 1.8271 | 0.9620549 | 0.004127 |
| 0.3118726 | 3.98E-04 | 0.1211546 | 0.0457933 | 0.5227548 | 31145.402 | 0.9568312 | 9.00E-04 | 587.94975 | 117033 | 1.95055 | 0.9624441 | 0.0036683 |
| 0.2867465 | 3.69E-04 | 0.1261143 | 0.0858857 | 0.5026527 | 34822.437 | 0.9582198 | 8.00E-04 | 644.41709 | 128254 | 2.1375667 | 0.963417 | 0.0029477 |
| 0.3319181 | 3.20E-04 | 0.1361159 | 0.0406462 | 0.4925607 | 39842.181 | 0.9591331 | 7.00E-04 | 738.93467 | 147048 | 2.4508 | 0.96439 | 0.002471 |
| 0.3067792 | 2.99E-04 | 0.1159715 | 0.0656649 | 0.5126319 | 45219.985 | 0.9600444 | 6.00E-04 | 814.98995 | 162229 | 2.7038167 | 0.9657521 | 0.0024014 |
| 0.3268212 | 2.69E-04 | 0.1209193 | 0.0756559 | 0.4774668 | 53172.774 | 0.9615122 | 5.00E-04 | 951.00503 | 189281 | 3.1546833 | 0.9675034 | 0.0029077 |
| 0.251422 | 0.0052492 | 0.1811337 | 0.0605513 | 0.5025688 | 63857.302 | 0.9637104 | 4.00E-04 | 1132.3015 | 225328 | 3.7554667 | 0.9696439 | 0.0029834 |
| 0.2413307 | 0.0102346 | 0.2061994 | 0.0353675 | 0.5075742 | 80150.191 | 0.9662254 | 3.00E-04 | 1408.1156 | 280231 | 4.6705167 | 0.9747033 | 0.00291 |
| 0.1759618 | 1.33E-04 | 0.3669061 | 0.1257354 | 0.3317159 | 113404.32 | 0.9705993 | 2.00E-04 | 1971.5276 | 392334 | 6.5389 | 0.9766492 | 0.0027735 |
| 0.0301947 | 6.71E-05 | 0.5176071 | 0.1508071 | 0.3015366 | 178704.5 | 0.9755657 | 1.00E-04 | 3079.9296 | 612906 | 10.2151 | 0.9807356 | 0.0020734 |
| 0.0352193 | 6.16E-05 | 0.5728769 | 0.1558284 | 0.2362122 | 189645.84 | 0.9762081 | 9.00E-05 | 3289.3618 | 654583 | 10.909717 | 0.9801518 | 0.0017323 |
| 0.0301881 | 5.37E-05 | 0.5326786 | 0.1708962 | 0.2663572 | 199240.38 | 0.9767577 | 8.00E-05 | 3124.5779 | 621799 | 10.363317 | 0.980541 | 0.0016933 |
| 0.0201314 | 4.96E-05 | 0.5176013 | 0.1909908 | 0.2713782 | 213000.2 | 0.9770862 | 7.00E-05 | 3870.0804 | 770149 | 12.835817 | 0.9820977 | 0.0017903 |
| 0.0201276 | 4.13E-05 | 0.5377 | 0.1608375 | 0.2814242 | 232538.81 | 0.9775898 | 6.00E-05 | 4183.5427 | 832531 | 13.875517 | 0.9819031 | 0.0019222 |
| 0.0251499 | 3.54E-05 | 0.4321747 | 0.2914793 | 0.2512723 | 261837.84 | 0.9784054 | 5.00E-05 | 4671.7035 | 929680 | 15.494667 | 0.9824869 | 0.0017131 |
| 0.020118 | 0.0050529 | 0.4673476 | 0.2864494 | 0.2211226 | 293045.84 | 0.97917 | 4.00E-05 | 4496.6231 | 894833 | 14.913883 | 0.982876 | 0.0017905 |
| 0.0150892 | 2.13E-05 | 0.5125703 | 0.206044 | 0.266342 | 341946.9 | 0.9802202 | 3.00E-05 | 5834.7286 | 1161121 | 19.352017 | 0.9836544 | 0.0016258 |
| 8.67E-06 | 0.0050394 | 0.467342 | 0.2964908 | 0.2311632 | 408835.39 | 0.9810798 | 2.00E-05 | 6995.1508 | 1392042 | 23.2007 | 0.9846274 | 0.0016328 |
| 4.22E-06 | 7.36E-06 | 0.2663364 | 0.4673396 | 0.2663348 | 576604.23 | 0.9824663 | 1.00E-05 | 9819.1457 | 1954028 | 32.567133 | 0.9856003 | 0.0014906 |
| 0.005029 | 6.68E-06 | 0.3567873 | 0.3718623 | 0.266335 | 590518.09 | 0.9824595 | 9.00E-06 | 10070.03 | 2003949 | 33.39915 | 0.9861841 | 0.0016341 |
| 3.66E-06 | 6.13E-06 | 0.361812 | 0.4221129 | 0.2160835 | 609672.87 | 0.9826296 | 8.00E-06 | 10578.678 | 2105158 | 35.085967 | 0.9854057 | 0.0014217 |
| 0.0100532 | 5.28E-06 | 0.3517614 | 0.4321629 | 0.206033 | 649084.43 | 0.9828379 | 7.00E-06 | 11378.653 | 2264389 | 37.739817 | 0.9863787 | 0.0014723 |
| 2.51E-06 | 4.55E-06 | 0.3919619 | 0.402012 | 0.2060328 | 688141.75 | 0.9829699 | 6.00E-06 | 11112.593 | 2211411 | 36.85685 | 0.9863787 | 0.0013737 |
| 0.0050274 | 3.74E-06 | 0.366836 | 0.3919614 | 0.2361828 | 739405.14 | 0.983187 | 5.00E-06 | 13885.628 | 2763247 | 46.054117 | 0.9863787 | 0.0013701 |
| 0.0050268 | 2.85E-06 | 0.3165845 | 0.4221118 | 0.2562828 | 814895.43 | 0.9833523 | 4.00E-06 | 12441.407 | 2475855 | 41.26425 | 0.9875462 | 0.0013463 |
| 0.0100515 | 2.26E-06 | 0.3115589 | 0.4221114 | 0.2562826 | 898370.47 | 0.983451 | 3.00E-06 | 15068.241 | 2998588 | 49.976467 | 0.9867679 | 0.0012352 |
| 0.0100511 | 0.0050265 | 0.3065335 | 0.4422116 | 0.2361817 | 1018724.4 | 0.9834647 | 2.00E-06 | 17163.116 | 3415468 | 56.924467 | 0.9865733 | 0.0014891 |
| 0.0100506 | 6.75E-07 | 0.2462316 | 0.4422113 | 0.3015079 | 1244358.3 | 0.9834852 | 1.00E-06 | 20607.568 | 4100909 | 68.348483 | 0.9865733 | 0.001324 |
| 0.0100506 | 6.44E-07 | 0.3216084 | 0.4522616 | 0.2160808 | 1291161.7 | 0.9834999 | 9.00E-07 | 21746.186 | 4327501 | 72.125017 | 0.987157 | 0.0014341 |
| 0.0050255 | 5.71E-07 | 0.2512566 | 0.432161 | 0.311558 | 1360011.7 | 0.9835537 | 8.00E-07 | 22496.085 | 4476721 | 74.612017 | 0.9867679 | 0.0014877 |
| 0.0050254 | 4.83E-07 | 0.2412064 | 0.432161 | 0.3216083 | 1402576.1 | 0.9834344 | 7.00E-07 | 24104.799 | 4796856 | 79.9476 | 0.9865733 | 0.0014402 |
| 0.0100505 | 4.13E-07 | 0.3015078 | 0.4371861 | 0.2512565 | 1430741.6 | 0.9834246 | 6.00E-07 | 24127.779 | 4801430 | 80.023833 | 0.987157 | 0.0014815 |
| 0.0050253 | 0.0100506 | 0.3366836 | 0.3718594 | 0.2763821 | 1539160.9 | 0.9834148 | 5.00E-07 | 26017.894 | 5177575 | 86.292917 | 0.9867679 | 0.0013904 |
| 0.0100504 | 2.66E-07 | 0.2964825 | 0.4170855 | 0.2763821 | 1618461.8 | 0.983409 | 4.00E-07 | 27559.96 | 5484442 | 91.407367 | 0.9867679 | 0.001473 |
| 0.0100504 | 2.00E-07 | 0.2864323 | 0.4020101 | 0.3015076 | 1702883.1 | 0.9833728 | 3.00E-07 | 29320.739 | 5834845 | 97.247417 | 0.9867679 | 0.0013969 |
| 0.0150755 | 1.47E-07 | 0.2663317 | 0.4221106 | 0.2964825 | 1843820.1 | 0.9831616 | 2.00E-07 | 31024.427 | 6173862 | 102.8977 | 0.9865733 | 0.0013638 |
| 0.0050252 | 0.0050252 | 0.2763819 | 0.3919598 | 0.3216081 | 2188509.1 | 0.9828672 | 1.00E-07 | 37078.025 | 7378528 | 122.97547 | 0.98813 | 0.0015414 |
| 0.0150754 | 0.0100503 | 0.2914573 | 0.3517588 | 0.3316583 | 2266508.3 | 0.9827519 | 9.00E-08 | 38511.402 | 7663774 | 127.72957 | 0.9875462 | 0.0015187 |
| 0.0201005 | 5.58E-08 | 0.2964824 | 0.3567839 | 0.3266332 | 2273906.5 | 0.9828144 | 8.00E-08 | 38618.528 | 7685090 | 128.08483 | 0.9861841 | 0.0014832 |
| 0.0100503 | 4.94E-08 | 0.2713568 | 0.3969849 | 0.3216081 | 2310469.9 | 0.9827157 | 7.00E-08 | 39082.382 | 7777409 | 129.62348 | 0.9867679 | 0.0014673 |
| 0.0150754 | 0.0100503 | 0.2512563 | 0.3668342 | 0.3567839 | 2424313.6 | 0.9827822 | 6.00E-08 | 42822.03 | 8521589 | 142.02648 | 0.9863787 | 0.0015439 |
?
?
這張圖是δ=5e-5到δ=6e-8的pave曲線，相當直觀當δ=8e-7時網(wǎng)絡達到峰值，峰值是0.983554。
這張圖是δ=1e-5到δ=6e-8的pave曲線，表明對三層網(wǎng)絡收斂標準是有最優(yōu)值的，超過最優(yōu)值以后網(wǎng)絡性能是下降的。
《估算卷積核數(shù)量的近似方法》實驗表明卷積核數(shù)量是有最優(yōu)值的
《平均分辨準確率對網(wǎng)絡隱藏層節(jié)點數(shù)的非線性變化關(guān)系03》實驗表明隱藏層節(jié)點數(shù)是有最優(yōu)值的。
因為有最優(yōu)值的存在，如果將網(wǎng)絡的收斂標準設(shè)定為某一迭代次數(shù)，則將迭代次數(shù)調(diào)大并不必然導致網(wǎng)絡的性能改善。如將收斂標準設(shè)為6e-8,平均性能只有峰值δ=8e-7的0.999216.但迭代次數(shù)卻是峰值的1.78倍，耗時是峰值的1.9倍，也就是用了1.9倍的時間卻換來性能下降萬分之8.或者也可以解釋成導致網(wǎng)絡性能下降的一個原因恰恰是迭代次數(shù)過多了。
?
總結(jié)
以上是生活随笔為你收集整理的神经网络的收敛标准有最优值吗？的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。
- 上一篇： 地暖机耗电量高不高？我看朋友家的挺暖的。
- 下一篇： 猜测：引力与空间映射