第一次浪潮：Rationalism

? ? ?“The approaches, based on the belief that knowledge of language in the human mind is fixed in advance by generic? ? ? ?inheritance”?

?? ?例證：特定領域的專家系統(exper system)

?? ?缺點：過分依賴于特定領域的知識工程(paradigm of expert knowledge engineering)，泛化性極差

第二次浪潮：Empiricism

?? ?數據驅動下的shallow machine learning 和 statistical learning 大行其道

?? ?“The key algorithms and methods for machine learning include EM(expectation-maximization), Bayesian networks, support vector machines, decision trees, and, for neural networks, backpropagation algorithm.”（欠缺得補）

?? ?缺點：不夠deep，“shallow layers”導致的模型解空間大小有限。

第三次浪潮：Deep learning

?? ?現實NLPer的寫照：“given an NLP task, apply standard sequence models based on (bidirectional) LSTMs, add attention mechanisms if information required in the task needs to flow from another source,and then train the full models in an end-to-end manner.”

?? ?局限：“have no reasoning and explaining capabilities”、句子間的關系推理

NLP未來的方向

?? ?1、Neural-Symbolic Integration（集成網絡和符號主義），“人話” 就是追求神經網絡可解釋性

?? ?2、Structure, Memory, and Knowledge，認可lstm的基礎上，追求更好的Memory model

?? ?3、Unsupervised and Generative Deep Learning

?? ?4、Multimodal and Multitask Deep Learning，跨領域的學習

?? ?5、Meta-learning，“The goal of meta-learning is to learn how to learn new tasks faster by reusing previous experience”

總結

以上是生活随笔為你收集整理的慢慢读《Deep Learning In Natural Language Processing》（一）的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。