神经网络
153
Network 类。在开始写 Network 类时,首先我们要写一个初始化类,它的入参是
Language 类的数组。其次,因为我们不想手动实现所有的神经网络代码,所以使用一
个名为
PyBrain
Python
库。我们的主要目标是得到一个能够接收数据、进行训练的
神经网络。
但是现在我们要做一个重要的决定:使用哪个神经网络工具库?有很多可以使用的
,而且我们也可以自己动手编写一个神经网络库。目前最好的神经网络库可能是
theanets
,它能够与
NumPy
很好地集成,可以用于深度学习、自动编码,而不仅仅是
单纯的前馈网络。我们将使用
theanets
库,而读者也可以使用诸如
PyBrain
之类的库。
But now we have an important decision to make: which neural net library should we
use? There are many out there, and of course we could build our own. Probably the
best one out there right now is theanets, which integrates nicely with NumPy and can
actually be utilized for deep learning, autoencoding, and much more than just
straight feed-forward networks. We will use that, although you could use other libra‐
ries like PyBrain.
import numpy as np
import theanets
from tokenizer import Tokenizer
class Network(object):
def __init__(self, languages, error=0.005):
self._trainer = None
self._net = None
self.languages = languages
self.error = error
self.inputs = set()
for language in languages:
self.inputs.update(language.characters)
self.inputs = sorted(self.inputs)
def _build_trainer(self):
inputs = []
desired_outputs = []
for language_index, language in enumerate(self.languages):
for vector in language.vectors:
inputs.append(self._code(vector))
desired_outputs.append(language_index)
inputs = np.array(inputs, dtype=np.float32)
desired_outputs = np.array(desired_outputs, dtype=np.int32)
self._trainer = (inputs, desired_outputs)
def _code(self, vector):
result = np.zeros(len(self.inputs))
for char, freq in vector.iteritems():
if char in self.inputs:
result[self.inputs.index(char)] = float(freq)
return result
def _build_ann(self):
hidden_neurons = 2 * (len(self.inputs) + len(self.languages)) / 3
self._net = theanets.Classifier([len(self.inputs),
{'size': hidden_neurons, 'activation': 'tanh'},
len(self.languages)])
Now that we have the proper inputs and the proper outputs, the model is set up and
we should be able to run the whole crossvalidation.py. But, of course, there is an error
because we cannot run new data against the network. To address this, we need to
build a function called
#run. At this point, we have something that works and looks
like this:
Using a Neural Network to Classify a Language | 153
现在我们已经有了正确的输入层和输出层,建立了模型,并且已经有条件来运行整个
crossvalidation.py
但是还欠缺了一点,因为还不能用新的数据来运行这个神经网络。
为了解决这个问题,需要构建一个名为 #run 的函数。完成这个函数之后,我们的神
经网络就可以工作了,具体代码如下所示:
class Network:
def train(self):
self._build_trainer()
self._build_ann()
self._net.train(self._trainer, learning_rate=0.01)
def predict(self, sentence):
if self._net is None or self._trainer is None:
raise Exception('Must train first')
vectors, characters = Tokenizer.tokenize(sentence)
if len(vectors) == 0:
return None
input = np.array(self._code(vectors[0]),
ndmin=2,
dtype=np.float32)
result = self._net.predict(input)
return self.languages[result[0]]
Tuning the Neural Network
At this point theres quite a few optimizations we could make. Also you could play
around with different hidden-layer activation functions like
tanh, softmax, or various
others.
I’ll leave further tuning to you as an exercise in playing around with what works and
what does not. You can try many different activation functions, as well as internal
rates of decay or errors. The takeaway here is that with an initial test to base accuracy
against, you can try many different avenues.
Precision and Recall for Neural Networks
Going a step further, when we deploy this neural network code to a production envi‐
ronment, we need to close the information loop by introducing a precision and recall
metric to track over time. This metric will be calculated from user input.
We can measure precision and recall by asking in our user interface whether our pre‐
diction was correct. From this text, we can capture the blurb and the correct classifi‐
cation, and feed that back into our model the next time we train.
To learn more about monitoring precision and recall, see Chapter 9.
What we need to monitor the performance of this neural network in production is a
metric for how many times a classification was run, as well as how many times it was
wrong.
Wrap-Up of Example
The neural networks algorithm is a fun way of mapping information and learning
through iterations, and it works well for our case of mapping sentences to languages.
154 | Chapter 8: Neural Networks
调优神经网络
对这个神经网络还可以做出很多优化,而且还可以尝试使用不同的隐藏层激励函数,
tanhsoftmax 或其他一些函数。
我把进一步的调整留给读者作为练习,请尝试一下哪些方法可以工作,哪些不行。可
以尝试许多不同的激励函数,也可以尝试调整内部衰减率或者误差率。由于已经做了
最初测试,就可以尝试许多不同的途径,把得到的正确率跟最初测试的结果相比较。
神经网络的精度和召回
如果我们更进一步,把这个神经网络的代码部署到生产环境中,则需要引入一个精确
率和召回率度量,随着时间持续跟踪结果,这样就完成了信息的闭环。这个度量需要
根据用户的输入来确定。
我们可以通过在用户界面中询问测量结果是否正确,来测量精确率和召回率。从查询
结果中,我们可以提取出简要概况和正确的分类,并在下次训练时将其反馈到我们的
模型。
要了解更多有关监测精确率和召回率的信息,请参见第
9
章。

Get Python 机器学习实践:测试驱动的开发方法 now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.