Machine Learning with Swift

Building a neural layer in Swift

A fully-connected layer is easy to implement, because it can be expressed as two operations:

• A matrix multiplication between weights matrix W and input vector x.
• A point wise application of activation function f :

Figure 8.5: One layer in detail

In many frameworks, the two operations are separated so that matrix multiplication happens in the fully-connected layer and activation happens in the next nonlinearity layer. This is handy because in this way we can easily replace the weighted sum with convolution. ...