簡述

簡單講講關于torch.autograd內容（因為我也有點菜）

文章目錄

• • 簡述
  • 簡單講講
  • Tensor
  • • 介紹Tensor
    • 創建Tensor
    • 獲取Tensor數據規模
    • 將tensor轉成其他數據類型
    • 改變Tensor形狀
    • Tensor的切片操作
    • Tensor的比較
    • Tensor的數據篩選
    • Tensor其他常用函數
    • • index_select(input, dim, index)函數
      • masked_select(input, mask)
      • nonzero(input)操作
      • gather 根據index，在dim上選取數據，輸出size與index一直
    • clamp截斷
  • 后記

簡單講講

• 創建數據

>>> import torch >>> from torch.autograd import Variable >>> x = Variable(torch.ones(2, 2), requires_grad=True) >>> x tensor([[1., 1.],[1., 1.]], requires_grad=True)

• Variable中有三個東西，就是下面的三個：data，grad，grad_fn

>>> x.data tensor([[1., 1.],[1., 1.]]) >>> x.grad >>> x.grad_fn

• 這時候下面兩個其實是NoneType的

>>> x.grad_fn.data Traceback (most recent call last):File "<stdin>", line 1, in <module> AttributeError: 'NoneType' object has no attribute 'data' >>> x.grad_fn.data Traceback (most recent call last):File "<stdin>", line 1, in <module> AttributeError: 'NoneType' object has no attribute 'data'

• 嘗試做反向傳播

>>> x.backward() Traceback (most recent call last):File "<stdin>", line 1, in <module>File "C:\Users\sean\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\tensor.py", line 93, in backwardtorch.autograd.backward(self, gradient, retain_graph, create_graph)File "C:\Users\sean\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\autograd\__init__.py", line 84, in backwardgrad_tensors = _make_grads(tensors, grad_tensors)File "C:\Users\sean\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\autograd\__init__.py", line 28, in _make_gradsraise RuntimeError("grad can be implicitly created only for scalar outputs") RuntimeError: grad can be implicitly created only for scalar outputs

意思很簡單，就是說，這里如果是做反向傳播，東西必須是標量的輸出。

• 對x求和

>>> y = x.sum() >>> y tensor(4., grad_fn=<SumBackward0>)

• 關于y做反向傳播
  有反向傳播的函數。

>>> y.backward() >>> y tensor(4., grad_fn=<SumBackward0>) >>> y.grad >>> y.grad_fn <SumBackward0 object at 0x0000026BADBFF278> >>> a = y.backward() >>> a >>> a == None True

• 這時候再去檢查x

>>> x tensor([[1., 1.],[1., 1.]], requires_grad=True) >>> x.grad tensor([[2., 2.],[2., 2.]]) >>> x.grad_fn

x的梯度不再是None
這里的話，其實x的梯度本來應該全是1的，但是做了兩次之后，有了累積。

• 再做一次y的反向傳播，驗證猜想

>>> y.backward() >>> x.grad tensor([[3., 3.],[3., 3.]])

驗證正確。所以這告訴我們每次計算前都需要把梯度歸零。這里不歸零，是因為有些算法需要用到這個操作。

• 梯度清0

>>> x.grad.data.zero_() tensor([[0., 0.],[0., 0.]]) >>> x tensor([[1., 1.],[1., 1.]], requires_grad=True) >>> x.grad tensor([[0., 0.],[0., 0.]])

• 再算一次y的反向傳播

>>> y.backward() >>> x.grad tensor([[1., 1.],[1., 1.]])

• 關于之前的求梯度結果全是1的解釋

$$y=\sum X_i$$

$y$關于$X_i$來求導，得到的結果都是1。所以每一個數值都是1。

• 驗證無關a的數值

由于我們看到下面的數值當中，a的數值都變成了2。但是 求導的結果任然是這個。

>>> a = Variable(torch.ones(2,2)*2, requires_grad=True) >>> a tensor([[2., 2.],[2., 2.]], requires_grad=True) >>> y = a.sum() >>> a tensor([[2., 2.],[2., 2.]], requires_grad=True) >>> y tensor(8., grad_fn=<SumBackward0>) >>> y.backward() >>> a.grad tensor([[1., 1.],[1., 1.]])

• 驗證和算法有關

$$y=\sum \cos X_i$$
關于X_i求導有

$$\frac{dy}{d{X}_i}=\sin X_i$$

>>> a = Variable(torch.ones(2,2)*2, requires_grad=True) >>> a tensor([[2., 2.],[2., 2.]], requires_grad=True) >>> a.grad >>> b = a.cos() >>> b tensor([[-0.4161, -0.4161],[-0.4161, -0.4161]], grad_fn=<CosBackward>) >>> c = b.sum() >>> b.grad >>> c.backward() >>> b.grad >>> a.grad tensor([[-0.9093, -0.9093],[-0.9093, -0.9093]])

通過數值來驗算一下：

>>> import numpy as np >>> np.sin(2) 0.9092974268256817 >>>

發現，保留4位有效數值的結果是一樣的。

Tensor

介紹Tensor

Tensor，又名張量。從工程角度不妨認為是一個數組。
和numpy中的ndarray類似，但是tensor支持GPU加速

–部分摘取于《深度學習框架Pytorch入門與實踐》

創建Tensor

只有使用torch.Tensor(*size)的方式創建的數組，才是創建數組的時候并不會分配內存空間，只有當使用到的時候，才會分配。

• 直接Tensor創建

>>> a = torch.Tensor(1) >>> a tensor([5.6052e-45])

發現數值其實是接近0的（準確說，在計算機里這個跟0也沒太多區別了，一般來說）

• 直接Tensor創建（二）

這個是將另外的類似于數組的結構轉變成Tensor

>>> a = torch.Tensor((2, 3)) >>> a tensor([2., 3.])

類似的操作

>>> a = torch.Tensor((2, 3, 4, 5)) >>> a tensor([2., 3., 4., 5.]) >>> a = torch.Tensor([2, 3, 4, 5]) >>> a tensor([2., 3., 4., 5.])

不得不說，這其實跟numpy的array構造法類似的

>>> a = torch.Tensor(np.array([2, 3, 4, 5])) >>> a tensor([2., 3., 4., 5.])

• 直接Tensor創建（三）

用size的維度來做輸入

>>> a = torch.Tensor(1,2) >>> a tensor([[0.0000, 0.0000]]) >>> a = torch.Tensor(3,2) >>> a tensor([[0.0000, 0.0000],[0.0000, 0.0000],[0.0000, 0.0000]])

• 構造全1的張量

>>> a = torch.ones(3,2) >>> a tensor([[1., 1.],[1., 1.],[1., 1.]]) >>> a = torch.ones((3,2)) >>> a tensor([[1., 1.],[1., 1.],[1., 1.]]) >>> a = torch.ones([3,2]) >>> a tensor([[1., 1.],[1., 1.],[1., 1.]]) >>> a = torch.ones(np.array((3,2))) Traceback (most recent call last):File "<stdin>", line 1, in <module> TypeError: ones(): argument 'size' (position 1) must be tuple of ints, not numpy.ndarray >>>

除了numpy不能使用的，其他的上面提到的，都可以作為size

• 構造全0的張量

跟全1的一模一樣。

>>> a = torch.zeros(3,2) >>> a tensor([[0., 0.],[0., 0.],[0., 0.]]) >>> a = torch.zeros((3,2)) >>> a tensor([[0., 0.],[0., 0.],[0., 0.]]) >>> a = torch.zeros([3,2]) >>> a tensor([[0., 0.],[0., 0.],[0., 0.]]) >>> a = torch.zeros(np.array([3,2])) Traceback (most recent call last):File "<stdin>", line 1, in <module> TypeError: zeros(): argument 'size' (position 1) must be tuple of ints, not numpy.ndarray >>> a = torch.zeros(1) >>> a tensor([0.]) >>> a = torch.zeros(2) >>> a tensor([0., 0.]) >>>

• 對角元全是1，其他都是0的矩陣
  只能是方陣

>>> a = torch.eye(2) >>> a tensor([[1., 0.],[0., 1.]]) >>> a = torch.eye(1) >>> a tensor([[1.]]) >>> a = torch.eye(3) >>> a tensor([[1., 0., 0.],[0., 1., 0.],[0., 0., 1.]]) >>> a = torch.eye(4) >>> a tensor([[1., 0., 0., 0.],[0., 1., 0., 0.],[0., 0., 1., 0.],[0., 0., 0., 1.]]) >>> a = torch.eye((1,2)) Traceback (most recent call last):File "<stdin>", line 1, in <module> TypeError: eye(): argument 'n' (position 1) must be int, not tuple

• arange(s,e, step)
  • step：步長

>>> a = torch.arange(0,1,10) >>> 1 >>> a = torch.arange(0,10,1) >>> a tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

• linspace(s, e, steps)
  • steps：步數
  • steps：大于等于2

>>> a = torch.linspace(0,1,10) >>> a tensor([0.0000, 0.1111, 0.2222, 0.3333, 0.4444, 0.5556, 0.6667, 0.7778, 0.8889,1.0000]) >>> a = torch.linspace(0,10,1) Traceback (most recent call last):File "<stdin>", line 1, in <module> RuntimeError: invalid argument 3: invalid number of points at d:\build\pytorch\pytorch-0.4.1\aten\src\th\generic\thtensormath.cpp:4408 >>> a = torch.linspace(0,10,2) >>> a tensor([ 0., 10.])

• rand(*size) 均勻分布 (類似的有randn標準正態分布)

>>> a = torch.rand(1) >>> a tensor([0.5391]) >>> a = torch.rand(1) >>> a tensor([0.7884]) >>> a = torch.rand(1,2) >>> a tensor([[0.6491, 0.8738]]) >>> a = torch.rand(2,1) >>> a tensor([[0.1860],[0.4988]]) >>> a = torch.rand(2) >>> a tensor([0.9654, 0.6263]) >>> a = torch.randn(1) >>> a tensor([-1.2497]) >>> a = torch.randn(1) >>> a tensor([-2.0466]) >>> a = torch.randn(1) >>> a tensor([0.4620]) >>> a = torch.randn(2) >>> a tensor([0.3337, 0.5260]) >>> a = torch.randn(1,2) >>> a tensor([[-0.0197, 0.1966]]) >>> a = torch.randn(2,1) >>> a tensor([[-0.3927],[-0.4279]])

• normal(mean, std)正態分布 / (uniform(s, e))均勻分布

下面這個寫的normal()還行
https://blog.csdn.net/qq_34690929/article/details/80029974

• 固定mean，但不固定std

>>> torch.normal(mean=0,std=torch.rand(5),out=a) tensor([-0.1707, 0.6314, 0.2634, -0.2100, -1.3774]) >>> torch.normal(mean=0,std=torch.rand(5)) tensor([-0.1613, -0.0957, 0.2675, 0.3178, -0.6548])

• 固定std，但不固定mean

>>> torch.normal(mean=torch.rand(5), std=1) tensor([-0.7600, 1.1998, 0.9734, 3.5803, 0.8375]) >>> torch.normal(mean=torch.rand(5), std=1,out=a) tensor([ 0.5822, 2.9761, -0.3904, 0.7321, 1.2330])

• 兩個都不固定

>>> torch.normal(mean=torch.rand(5), std=torch.linspace(1,2,5),out=a) tensor([-0.2679, 0.1669, 3.7573, -1.9038, -0.2729]) >>> torch.normal(mean=torch.rand(5), std=torch.linspace(1,2,5)) tensor([ 0.7096, 1.0136, -2.5004, -0.2733, 3.6389])

• 如果想要兩個都固定，只能重復來做
  • 下面的構造函數只有三個。

>>> torch.normal(mean=0,std=1,out=a) Traceback (most recent call last):File "<stdin>", line 1, in <module> TypeError: normal() received an invalid combination of arguments - got (mean=int, std=int, out=Tensor, ), but expected one of:* (Tensor mean, Tensor std, torch.Generator generator, Tensor out)* (Tensor mean, float std, torch.Generator generator, Tensor out)* (float mean, Tensor std, torch.Generator generator, Tensor out)

比如：
下面的就是標準正態分布。

>>> torch.normal(mean=torch.ones(5), std=torch.ones(5)+1) tensor([ 0.6148, -3.3352, 3.0508, 0.4238, 1.4016])

uniform就不一樣了。必須另外操作

我懷疑是改了版本了，我看有些書都是沒這么寫的。。感覺蠻奇怪的。

但是在pytorch的官方文檔中有這樣的解釋：

Fills self tensor with numbers sampled from the continuous uniform distribution:
用采樣自下面的連續均勻分布的數字來填充self tensor 。

所以，雖然文檔中沒有寫具體的實例代碼，也能知道如何操作（如下：）

>>> torch.Tensor(10).uniform_(0,1) tensor([0.3394, 0.8679, 0.3344, 0.6168, 0.6465, 0.7618, 0.3902, 0.5174, 0.0205, 0.6731])

• randperm(m)隨機排列

生成0到m的隨機排列

>>> torch.randperm(10) tensor([1, 7, 4, 8, 9, 3, 0, 6, 2, 5])

獲取Tensor數據規模

• 其實只有下面的兩種方法

>>> a tensor([-0.2679, 0.1669, 3.7573, -1.9038, -0.2729]) >>> a.size() torch.Size([5]) >>> a.shape torch.Size([5]) >>> a tensor([[-0.2679],[ 0.1669],[ 3.7573],[-1.9038],[-0.2729]]) >>> a.size() torch.Size([5, 1]) >>> a.shape torch.Size([5, 1])

將tensor轉成其他數據類型

• 轉成list

很吃驚的是，居然精度有這么高！！

>>> a tensor([-0.2679, 0.1669, 3.7573, -1.9038, -0.2729]) >>> a.tolist() [-0.2678675651550293, 0.16691356897354126, 3.757300853729248, -1.9038498401641846, -0.27292633056640625]

• 轉成numpy

>>> a tensor([-0.2679, 0.1669, 3.7573, -1.9038, -0.2729]) >>> a.numpy() array([-0.26786757, 0.16691357, 3.7573009 , -1.9038498 , -0.27292633],dtype=float32)

改變Tensor形狀

先創建一個數據

>>> a tensor([[-0.0752, 0.0072, 1.7456, 1.2480, 0.7506, -1.2426],[ 0.1641, -0.6519, -0.9540, -1.0443, -0.8130, 1.0243],[ 1.2052, -1.0993, 2.5415, 0.9572, -0.9123, 0.6194],[ 1.4059, 1.1456, -0.1732, -1.0271, -0.0565, -0.6258],[ 0.7262, -2.5908, 0.5556, 0.6691, -0.0912, 2.1089],[ 1.5669, -0.6453, 0.8954, 0.4817, -0.6550, 0.9734]]) >>> a.size() torch.Size([6, 6])

• reshape操作

>>> a.reshape(2, -1) tensor([[-0.0752, 0.0072, 1.7456, 1.2480, 0.7506, -1.2426, 0.1641, -0.6519,-0.9540, -1.0443, -0.8130, 1.0243, 1.2052, -1.0993, 2.5415, 0.9572,-0.9123, 0.6194],[ 1.4059, 1.1456, -0.1732, -1.0271, -0.0565, -0.6258, 0.7262, -2.5908,0.5556, 0.6691, -0.0912, 2.1089, 1.5669, -0.6453, 0.8954, 0.4817,-0.6550, 0.9734]]) >>> a.reshape(2, -1).shape torch.Size([2, 18])

可以看出，其實是扁平壓縮的。應該說，這個是類似于C++存放數組的方式，將高維數據轉成一維數據來處理。之后，獲取的時候，只需要用這種方式來進行獲取就好了。

但是注意，replace不會覆蓋原來的版本

>>> a tensor([[-0.0752, 0.0072, 1.7456, 1.2480, 0.7506, -1.2426],[ 0.1641, -0.6519, -0.9540, -1.0443, -0.8130, 1.0243],[ 1.2052, -1.0993, 2.5415, 0.9572, -0.9123, 0.6194],[ 1.4059, 1.1456, -0.1732, -1.0271, -0.0565, -0.6258],[ 0.7262, -2.5908, 0.5556, 0.6691, -0.0912, 2.1089],[ 1.5669, -0.6453, 0.8954, 0.4817, -0.6550, 0.9734]]) >>> a.shape torch.Size([6, 6])

• view方法

>>> a.view(2, -1) tensor([[-0.0752, 0.0072, 1.7456, 1.2480, 0.7506, -1.2426, 0.1641, -0.6519,-0.9540, -1.0443, -0.8130, 1.0243, 1.2052, -1.0993, 2.5415, 0.9572,-0.9123, 0.6194],[ 1.4059, 1.1456, -0.1732, -1.0271, -0.0565, -0.6258, 0.7262, -2.5908,0.5556, 0.6691, -0.0912, 2.1089, 1.5669, -0.6453, 0.8954, 0.4817,-0.6550, 0.9734]]) >>> a.reshape(2,-1) tensor([[-0.0752, 0.0072, 1.7456, 1.2480, 0.7506, -1.2426, 0.1641, -0.6519,-0.9540, -1.0443, -0.8130, 1.0243, 1.2052, -1.0993, 2.5415, 0.9572,-0.9123, 0.6194],[ 1.4059, 1.1456, -0.1732, -1.0271, -0.0565, -0.6258, 0.7262, -2.5908,0.5556, 0.6691, -0.0912, 2.1089, 1.5669, -0.6453, 0.8954, 0.4817,-0.6550, 0.9734]])

但是a也不會覆蓋掉

• resize_方法

>>> a.resize_(2,18) tensor([[-0.0752, 0.0072, 1.7456, 1.2480, 0.7506, -1.2426, 0.1641, -0.6519,-0.9540, -1.0443, -0.8130, 1.0243, 1.2052, -1.0993, 2.5415, 0.9572,-0.9123, 0.6194],[ 1.4059, 1.1456, -0.1732, -1.0271, -0.0565, -0.6258, 0.7262, -2.5908,0.5556, 0.6691, -0.0912, 2.1089, 1.5669, -0.6453, 0.8954, 0.4817,-0.6550, 0.9734]]) >>> a tensor([[-0.0752, 0.0072, 1.7456, 1.2480, 0.7506, -1.2426, 0.1641, -0.6519,-0.9540, -1.0443, -0.8130, 1.0243, 1.2052, -1.0993, 2.5415, 0.9572,-0.9123, 0.6194],[ 1.4059, 1.1456, -0.1732, -1.0271, -0.0565, -0.6258, 0.7262, -2.5908,0.5556, 0.6691, -0.0912, 2.1089, 1.5669, -0.6453, 0.8954, 0.4817,-0.6550, 0.9734]])

這次就會發生改變了。
但是注意，這次不允許使用負數（來進行默認計算）

>>> a.resize_(2,-1) Traceback (most recent call last):File "<stdin>", line 1, in <module> RuntimeError: sizes must be non-negative

再變回去，內容也不會發生變化

>>> a.resize_(6,6) tensor([[-0.0752, 0.0072, 1.7456, 1.2480, 0.7506, -1.2426],[ 0.1641, -0.6519, -0.9540, -1.0443, -0.8130, 1.0243],[ 1.2052, -1.0993, 2.5415, 0.9572, -0.9123, 0.6194],[ 1.4059, 1.1456, -0.1732, -1.0271, -0.0565, -0.6258],[ 0.7262, -2.5908, 0.5556, 0.6691, -0.0912, 2.1089],[ 1.5669, -0.6453, 0.8954, 0.4817, -0.6550, 0.9734]])

• unsqueeze()方法
  • 壓縮
  • 其實就是擴充一個維度
  • dim這個參數很關鍵。（從0開始，由于一開始沒有dim=1，使用后多出這個維度來）
  • 不發生覆蓋

>>> a = torch.rand(10) >>> a tensor([0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,0.4718]) >>> a.unsqueeze(dim=1) tensor([[0.5478],[0.4366],[0.2502],[0.5778],[0.7834],[0.8406],[0.3881],[0.8908],[0.0255],[0.4718]]) >>> a tensor([0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,0.4718])

但是如果維度只有1的話，我們dim=2就會出問題

>>> a.unsqueeze(dim=2) Traceback (most recent call last):File "<stdin>", line 1, in <module> RuntimeError: Dimension out of range (expected to be in range of [-2, 1], but got 2)

但是這個報錯很有趣，所以，我都試了下

>>> a tensor([0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,0.4718]) >>> a.unsqueeze(dim=0) tensor([[0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,0.4718]]) >>> a.unsqueeze(dim=-1) tensor([[0.5478],[0.4366],[0.2502],[0.5778],[0.7834],[0.8406],[0.3881],[0.8908],[0.0255],[0.4718]]) >>> a.unsqueeze(dim=-2) tensor([[0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,0.4718]])

• squeeze()反向操作

>>> a tensor([0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,0.4718]) >>> a.squeeze() tensor([0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,0.4718])

把數據做一下簡單的調整

>>> a tensor([[0.5478],[0.4366],[0.2502],[0.5778],[0.7834],[0.8406],[0.3881],[0.8908],[0.0255],[0.4718]]) >>> a.squeeze() tensor([0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,0.4718])

先壓縮一下：

>>> a = a.unsqueeze(dim=1) >>> a tensor([[[0.5478]],[[0.4366]],[[0.2502]],[[0.5778]],[[0.7834]],[[0.8406]],[[0.3881]],[[0.8908]],[[0.0255]],[[0.4718]]]) >>> a.shape torch.Size([10, 1, 1]) >>> a.squeeze() tensor([0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,0.4718])

直接就壓縮到最簡單的版本了。
但是，通過下面的代碼結果可以看出來，其實不是壓縮到最簡單的版本。估計只是考慮了中間只有1的操作

>>> a.resize_(2,5) tensor([[0.5478, 0.4366, 0.2502, 0.5778, 0.7834],[0.8406, 0.3881, 0.8908, 0.0255, 0.4718]]) >>> a tensor([[0.5478, 0.4366, 0.2502, 0.5778, 0.7834],[0.8406, 0.3881, 0.8908, 0.0255, 0.4718]]) >>> a.squeeze() tensor([[0.5478, 0.4366, 0.2502, 0.5778, 0.7834],[0.8406, 0.3881, 0.8908, 0.0255, 0.4718]])

只是把中間的維度為1的給全去掉了。

>>> a.resize_(2, 1, 1, 5, 1, 1) tensor([[[[[[0.5478]],[[0.4366]],[[0.2502]],[[0.5778]],[[0.7834]]]]],[[[[[0.8406]],[[0.3881]],[[0.8908]],[[0.0255]],[[0.4718]]]]]]) >>> a.squeeze() tensor([[0.5478, 0.4366, 0.2502, 0.5778, 0.7834],[0.8406, 0.3881, 0.8908, 0.0255, 0.4718]])

Tensor的切片操作

創建數據

>>> a = torch.randn(6,6) >>> a tensor([[-0.9391, 1.4903, -1.4979, 1.4666, 0.1815, -0.6964],[-1.2770, -0.8761, -1.9706, 0.8806, -0.9304, 0.0181],[ 0.1157, 1.3184, 0.2521, -1.5565, -0.8318, 1.0100],[-0.2843, 0.3335, -0.1813, -1.3236, 2.2849, -0.0776],[-0.6731, -0.5142, 0.2758, 0.4677, 2.0181, -1.2722],[ 1.8404, 0.7929, -0.8389, 1.0610, -0.0790, -0.2701]])

• 簡單提取和切片的區別
  • 后面兩個為切片，第一個是直接index提取

>>> a[0] tensor([-0.9391, 1.4903, -1.4979, 1.4666, 0.1815, -0.6964]) >>> a[:1] tensor([[-0.9391, 1.4903, -1.4979, 1.4666, 0.1815, -0.6964]]) >>> a[0,:] tensor([-0.9391, 1.4903, -1.4979, 1.4666, 0.1815, -0.6964])

• 取第一行到第二行（數值從0開始取）
  • 看來默認是取行了。

>>> a[1:3] tensor([[-1.2770, -0.8761, -1.9706, 0.8806, -0.9304, 0.0181],[ 0.1157, 1.3184, 0.2521, -1.5565, -0.8318, 1.0100]])

• 取列的話操作類似（取第一列到第二列）

>>> a[:,1:3] tensor([[ 1.4903, -1.4979],[-0.8761, -1.9706],[ 1.3184, 0.2521],[ 0.3335, -0.1813],[-0.5142, 0.2758],[ 0.7929, -0.8389]])

• 最后一行和最后一列

>>> a[-1] tensor([ 1.8404, 0.7929, -0.8389, 1.0610, -0.0790, -0.2701]) >>> a[:,-1] tensor([-0.6964, 0.0181, 1.0100, -0.0776, -1.2722, -0.2701])

• 取不連續的行列
  • 注意后面的逗號不能省去
  • 列是類似的。

>>> a tensor([[-0.9391, 1.4903, -1.4979, 1.4666, 0.1815, -0.6964],[-1.2770, -0.8761, -1.9706, 0.8806, -0.9304, 0.0181],[ 0.1157, 1.3184, 0.2521, -1.5565, -0.8318, 1.0100],[-0.2843, 0.3335, -0.1813, -1.3236, 2.2849, -0.0776],[-0.6731, -0.5142, 0.2758, 0.4677, 2.0181, -1.2722],[ 1.8404, 0.7929, -0.8389, 1.0610, -0.0790, -0.2701]]) >>> a[(1,3),] tensor([[-1.2770, -0.8761, -1.9706, 0.8806, -0.9304, 0.0181],[-0.2843, 0.3335, -0.1813, -1.3236, 2.2849, -0.0776]])

Tensor的比較

• Tensor > int

>>> a tensor([[-0.9391, 1.4903, -1.4979, 1.4666, 0.1815, -0.6964],[-1.2770, -0.8761, -1.9706, 0.8806, -0.9304, 0.0181],[ 0.1157, 1.3184, 0.2521, -1.5565, -0.8318, 1.0100],[-0.2843, 0.3335, -0.1813, -1.3236, 2.2849, -0.0776],[-0.6731, -0.5142, 0.2758, 0.4677, 2.0181, -1.2722],[ 1.8404, 0.7929, -0.8389, 1.0610, -0.0790, -0.2701]]) >>> a > 0 tensor([[0, 1, 0, 1, 1, 0],[0, 0, 0, 1, 0, 1],[1, 1, 1, 0, 0, 1],[0, 1, 0, 0, 1, 0],[0, 0, 1, 1, 1, 0],[1, 1, 0, 1, 0, 0]], dtype=torch.uint8)

得到的是一個非負整數的矩陣，同等規模。估計是pytorch為了更方便計算，直接用的是uint8類型。

• 顯示的時候是有精度的，所以等號不太管用
  • 如果你還記得上面的tolist的操作的話，就理解了

>>> a tensor([[-0.9391, 1.4903, -1.4979, 1.4666, 0.1815, -0.6964],[-1.2770, -0.8761, -1.9706, 0.8806, -0.9304, 0.0181],[ 0.1157, 1.3184, 0.2521, -1.5565, -0.8318, 1.0100],[-0.2843, 0.3335, -0.1813, -1.3236, 2.2849, -0.0776],[-0.6731, -0.5142, 0.2758, 0.4677, 2.0181, -1.2722],[ 1.8404, 0.7929, -0.8389, 1.0610, -0.0790, -0.2701]]) >>> a == -0.9391 tensor([[0, 0, 0, 0, 0, 0],[0, 0, 0, 0, 0, 0],[0, 0, 0, 0, 0, 0],[0, 0, 0, 0, 0, 0],[0, 0, 0, 0, 0, 0],[0, 0, 0, 0, 0, 0]], dtype=torch.uint8)

Tensor的數據篩選

• 比如Tensor大于某個數值的才要之類的操作

>>> a[a>0] tensor([1.4903, 1.4666, 0.1815, 0.8806, 0.0181, 0.1157, 1.3184, 0.2521, 1.0100,0.3335, 2.2849, 0.2758, 0.4677, 2.0181, 1.8404, 0.7929, 1.0610])

這個跟numpy的結果類似

>>> a.numpy()[a.numpy() > 0] array([1.4902998 , 1.4666233 , 0.18153903, 0.8805863 , 0.01809631,0.11573527, 1.3184009 , 0.25213283, 1.0099975 , 0.3334596 ,2.2849436 , 0.2758326 , 0.46773553, 2.0181057 , 1.8403844 ,0.7928698 , 1.0610487 ], dtype=float32)

Tensor其他常用函數

index_select(input, dim, index)函數

在指定的維度上選取

>>> indices = torch.tensor([0,1]) >>> torch.index_select(a, 1, indices) tensor([[-0.9391, 1.4903],[-1.2770, -0.8761],[ 0.1157, 1.3184],[-0.2843, 0.3335],[-0.6731, -0.5142],[ 1.8404, 0.7929]]) >>> a tensor([[-0.9391, 1.4903, -1.4979, 1.4666, 0.1815, -0.6964],[-1.2770, -0.8761, -1.9706, 0.8806, -0.9304, 0.0181],[ 0.1157, 1.3184, 0.2521, -1.5565, -0.8318, 1.0100],[-0.2843, 0.3335, -0.1813, -1.3236, 2.2849, -0.0776],[-0.6731, -0.5142, 0.2758, 0.4677, 2.0181, -1.2722],[ 1.8404, 0.7929, -0.8389, 1.0610, -0.0790, -0.2701]])

記住，這里的tensor不同于Tensor

>>> indices tensor([0, 1]) >>> torch.Tensor([0,1]) tensor([0., 1.])

發現了吧，必須要是正整數。
而且也必須是Tensor

>>> torch.index_select(a, 1, [0,1]) Traceback (most recent call last):File "<stdin>", line 1, in <module> TypeError: index_select(): argument 'index' (position 3) must be Tensor, not list

masked_select(input, mask)

• 效果跟a[a>0]類似。
• 只不過可以使用手動創建的更復雜的masked_select操作。

>>> torch.masked_select(a, a>0) tensor([1.4903, 1.4666, 0.1815, 0.8806, 0.0181, 0.1157, 1.3184, 0.2521, 1.0100,0.3335, 2.2849, 0.2758, 0.4677, 2.0181, 1.8404, 0.7929, 1.0610]) >>> a[a>0] tensor([1.4903, 1.4666, 0.1815, 0.8806, 0.0181, 0.1157, 1.3184, 0.2521, 1.0100,0.3335, 2.2849, 0.2758, 0.4677, 2.0181, 1.8404, 0.7929, 1.0610])

nonzero(input)操作

• 返回非0點坐標

>>> a tensor([[1., 0., 0., 0., 0.],[0., 1., 0., 0., 0.],[0., 0., 1., 0., 0.],[0., 0., 0., 1., 0.],[0., 0., 0., 0., 1.]]) >>> torch.nonzero(a) tensor([[0, 0],[1, 1],[2, 2],[3, 3],[4, 4]])

gather 根據index，在dim上選取數據，輸出size與index一直

• 其實就是一個多重映射

>>> t = torch.tensor([[1,2],[3,4]]) >>> torch.gather(t, 1, torch.tensor([[0,0],[1,0]])) tensor([[1, 1],[4, 3]]) >>> torch.gather(t, 0, torch.tensor([[0,0],[1,0]])) tensor([[1, 2],[3, 2]])

• 用下面的例子來區分dim=1和dim=0的區別

>>> t tensor([[1, 2],[3, 4]]) >>> torch.gather(t, 0, torch.tensor([[0,1],[1,0]])) tensor([[1, 4],[3, 2]]) >>> torch.gather(t, 1, torch.tensor([[0,1],[1,0]])) tensor([[1, 2],[4, 3]])

看到有這么一段公式
3D-Tensor下的公式。（n維都是類似的）

out[i][j][k] = input[index[i][j][k]][j][k] # if dim == 0 out[i][j][k] = input[i][index[i][j][k]][k] # if dim == 1 out[i][j][k] = input[i][j][index[i][j][k]] # if dim == 2

clamp截斷

這個函數就是用來做截斷的

>>> t tensor([[1, 2],[3, 4]]) >>> torch.clamp(t, 2, 3) tensor([[2, 2],[3, 3]])

后記

大概就這么多吧？以后再遇到再補充吧？大家有想法也可以提出來呢~

總結

以上是生活随笔為你收集整理的pytorch基础操作学习笔记（autograd，Tensor）的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。