自定义层
========
深度学习成功背后的一个因素是神经网络的灵活性：
我们可以用创造性的方式组合不同的层，从而设计出适用于各种任务的架构。
例如，研究人员发明了专门用于处理图像、文本、序列数据和执行动态规划的层。
有时我们会遇到或要自己发明一个现在在深度学习框架中还不存在的层。
在这些情况下，必须构建自定义层。本节将展示如何构建自定义层。
不带参数的层
------------
首先，我们构造一个没有任何参数的自定义层。 回忆一下在
:numref:`sec_model_construction`\ 对块的介绍， 这应该看起来很眼熟。
下面的\ ``CenteredLayer``\ 类要从其输入中减去均值。
要构建它，我们只需继承基础层类并实现前向传播功能。
.. raw:: html
.. raw:: html
.. raw:: latex
\diilbookstyleinputcell
.. code:: python
from mindspore import Parameter, nn, ops
class CenteredLayer(nn.Cell):
def __init__(self):
super().__init__()
def construct(self, X):
return X - X.mean()
.. raw:: html
.. raw:: html
.. raw:: latex
\diilbookstyleinputcell
.. code:: python
import torch
import torch.nn.functional as F
from torch import nn
class CenteredLayer(nn.Module):
def __init__(self):
super().__init__()
def forward(self, X):
return X - X.mean()
.. raw:: html
.. raw:: html
.. raw:: html
.. raw:: latex
\diilbookstyleinputcell
.. code:: python
layer = CenteredLayer()
layer(ops.Tensor([1., 2, 3, 4, 5]))
.. raw:: latex
\diilbookstyleoutputcell
.. parsed-literal::
:class: output
Tensor(shape=[5], dtype=Float32, value= [-2.00000000e+00, -1.00000000e+00, 0.00000000e+00, 1.00000000e+00, 2.00000000e+00])
.. raw:: html
.. raw:: html
.. raw:: latex
\diilbookstyleinputcell
.. code:: python
layer = CenteredLayer()
layer(torch.FloatTensor([1, 2, 3, 4, 5]))
.. raw:: latex
\diilbookstyleoutputcell
.. parsed-literal::
:class: output
tensor([-2., -1., 0., 1., 2.])
.. raw:: html
.. raw:: html
.. raw:: html
.. raw:: latex
\diilbookstyleinputcell
.. code:: python
net = nn.SequentialCell([nn.Dense(8, 128), CenteredLayer()])
.. raw:: html
.. raw:: html
.. raw:: latex
\diilbookstyleinputcell
.. code:: python
net = nn.Sequential(nn.Linear(8, 128), CenteredLayer())
.. raw:: html
.. raw:: html
.. raw:: html
.. raw:: latex
\diilbookstyleinputcell
.. code:: python
Y = net(ops.rand((4, 8)))
Y.mean()
.. raw:: latex
\diilbookstyleoutputcell
.. parsed-literal::
:class: output
Tensor(shape=[], dtype=Float32, value= -2.32831e-09)
.. raw:: html
.. raw:: html
.. raw:: latex
\diilbookstyleinputcell
.. code:: python
Y = net(torch.rand(4, 8))
Y.mean()
.. raw:: latex
\diilbookstyleoutputcell
.. parsed-literal::
:class: output
tensor(-5.5879e-09, grad_fn=)
.. raw:: html
.. raw:: html
.. raw:: html
.. raw:: latex
\diilbookstyleinputcell
.. code:: python
class MyLinear(nn.Cell):
def __init__(self, in_units, units):
super().__init__()
self.weight = Parameter(ops.randn((in_units, units)))
self.bias = Parameter(ops.randn((units,)))
def construct(self, X):
linear = ops.matmul(X, self.weight) + self.bias
return ops.relu(linear)
.. raw:: html
.. raw:: html
.. raw:: latex
\diilbookstyleinputcell
.. code:: python
class MyLinear(nn.Module):
def __init__(self, in_units, units):
super().__init__()
self.weight = nn.Parameter(torch.randn(in_units, units))
self.bias = nn.Parameter(torch.randn(units,))
def forward(self, X):
linear = torch.matmul(X, self.weight.data) + self.bias.data
return F.relu(linear)
.. raw:: html
.. raw:: html
.. raw:: html
.. raw:: latex
\diilbookstyleinputcell
.. code:: python
linear = MyLinear(5, 3)
linear.weight.value()
.. raw:: latex
\diilbookstyleoutputcell
.. parsed-literal::
:class: output
Tensor(shape=[5, 3], dtype=Float32, value=
[[-8.37819934e-01, -1.42728090e-01, 6.98715270e-01],
[ 6.33553147e-01, 4.66425151e-01, -9.23575342e-01],
[ 7.80555904e-02, 1.27004075e+00, -9.80480433e-01],
[ 7.85815343e-02, 1.37643766e+00, -4.39103961e-01],
[ 1.91481805e+00, -3.83728445e-01, 1.56912458e+00]])
.. raw:: html
.. raw:: html
.. raw:: latex
\diilbookstyleinputcell
.. code:: python
linear = MyLinear(5, 3)
linear.weight
.. raw:: latex
\diilbookstyleoutputcell
.. parsed-literal::
:class: output
Parameter containing:
tensor([[ 0.1146, 0.2678, -2.7968],
[-0.7714, 0.3853, 0.3882],
[ 0.3864, 0.2533, 0.2811],
[ 0.8532, 0.2337, 0.3863],
[ 1.8150, -0.7465, 0.5738]], requires_grad=True)
.. raw:: html
.. raw:: html
.. raw:: html
.. raw:: latex
\diilbookstyleinputcell
.. code:: python
linear(ops.rand((2, 5)))
.. raw:: latex
\diilbookstyleoutputcell
.. parsed-literal::
:class: output
Tensor(shape=[2, 3], dtype=Float32, value=
[[ 1.47905600e+00, 3.60304499e+00, 0.00000000e+00],
[ 2.11579418e+00, 3.30277443e+00, 0.00000000e+00]])
.. raw:: html
.. raw:: html
.. raw:: latex
\diilbookstyleinputcell
.. code:: python
linear(torch.rand(2, 5))
.. raw:: latex
\diilbookstyleoutputcell
.. parsed-literal::
:class: output
tensor([[2.4316, 0.0000, 0.0000],
[2.4526, 0.0000, 0.0000]])
.. raw:: html
.. raw:: html
.. raw:: html
.. raw:: latex
\diilbookstyleinputcell
.. code:: python
net = nn.SequentialCell([MyLinear(64, 8), MyLinear(8, 1)])
net(ops.rand((2, 64)))
.. raw:: latex
\diilbookstyleoutputcell
.. parsed-literal::
:class: output
Tensor(shape=[2, 1], dtype=Float32, value=
[[ 0.00000000e+00],
[ 0.00000000e+00]])
.. raw:: html
.. raw:: html
.. raw:: latex
\diilbookstyleinputcell
.. code:: python
net = nn.Sequential(MyLinear(64, 8), MyLinear(8, 1))
net(torch.rand(2, 64))
.. raw:: latex
\diilbookstyleoutputcell
.. parsed-literal::
:class: output
tensor([[0.8125],
[9.8983]])
.. raw:: html
.. raw:: html
.. raw:: html
`讨论 `__
.. raw:: html
.. raw:: html
`讨论 `__
.. raw:: html
.. raw:: html