pytorch学习 基本组成、基本练习

1. 基本结构

这里写图片描述
pytorch主要分为以下几个模块来训练模型:

  1. tensor:tensor为基本结构,可以直接创建,从list创建以及由numpy数组得到,torch还提供一套运算以及shape变换方式。
  2. Variable:自动求导机制,利用Variable包装tensor后,便可以使用其求导的功能了,有点像个装饰器。
  3. nn:nn模块是整个pytorch的核心,自己设计的Net(),继承nn.Model后可以提取模型参数,进行前向forward()运算(自己设计),以及后向运算(自动),nn提供基本网络结构单元,例如nn.Linear(),nn.Conv2d()等,还提供基本损失函数nn.CrossEntropyLoss等。
  4. torch.optim:该模块提供自动求导更新参数等功能,用它封装模型参数nn.parameter()后,loss求导后,可以用.step来更新整个参数。
  5. torch.utils.data.DataSet:该模块提供加载数据初始化的方式,完善好getitem和len的接口后,便可以利用DataLoader多进程加载数据。

2.基本练习

1.Tensor学习

import torch

1.1 tensorf创建

1.1 list 转 tensor

V_data = [1,2,3,4]
V = torch.tensor(V_data)
TypeErrorTraceback (most recent call last)

<ipython-input-4-9481b0be1435> in <module>()
----> 1 V = torch.tensor(V_data)


TypeError: 'module' object is not callable
V = torch.Tensor(V_data)
print V
 1
 2
 3
 4
[torch.FloatTensor of size 4]

1.2 通过tensor自定义创建tensor

1.2.1 torch.rand

?torch.rand

rand创建的是随机均匀分布的,值的范围为0到1

torch.rand(2,3) 
 0.7315  0.7298  0.0671
 0.7685  0.0843  0.5733
[torch.FloatTensor of size 2x3]

1.2.2 torch.randn

?torch.randn

torch.randn为正太分布,均值为0,方差为1

torch.randn(1,2)
-0.7939 -0.7320
[torch.FloatTensor of size 1x2]

1.2 tensor数学运算

1.2.1 加法

torch.Tensor([1,2,3]) + torch.Tensor([4,5,6])
 5
 7
 9
[torch.FloatTensor of size 3]

1.2.2 tensor拼接

维度解读

k0·k1·k2·k3·k4·k5

如上,最左边的为第0维,从左到右依次递增,拼接第几维就是那一维度相加,其余维度不变(也是约束条件)

?torch.cat
torch.cat(torch.Tensor([1,2,3]),torch.Tensor([4,5,6]),0)
TypeErrorTraceback (most recent call last)

<ipython-input-20-367eddc16eb7> in <module>()
----> 1 torch.cat(torch.Tensor([1,2,3]),torch.Tensor([4,5,6]),0)


TypeError: cat received an invalid combination of arguments - got (torch.FloatTensor, torch.FloatTensor, int), but expected one of:
 * (sequence[torch.FloatTensor] seq)
 * (sequence[torch.FloatTensor] seq, int dim)
torch.cat([torch.Tensor([1,2,3]),torch.Tensor([4,5,6])],0)
 1
 2
 3
 4
 5
 6
[torch.FloatTensor of size 6]
torch.cat([torch.Tensor([1,2,3]),torch.Tensor([4,5,6])],1)
RuntimeErrorTraceback (most recent call last)

<ipython-input-23-ffc600f25730> in <module>()
----> 1 torch.cat([torch.Tensor([1,2,3]),torch.Tensor([4,5,6])],1)


RuntimeError: dim out of range - got 1 but the tensor is only 1D
torch.Tensor([1,2,3])
 1
 2
 3
[torch.FloatTensor of size 3]
torch.Tensor([[1,2,3]])
 1  2  3
[torch.FloatTensor of size 1x3]
torch.Tensor([[[1,2,3]]])
(0 ,.,.) = 
  1  2  3
[torch.FloatTensor of size 1x1x3]
torch.cat([torch.Tensor([[1,2,3]]),torch.Tensor([[4,5,6]])],0)
 1  2  3
 4  5  6
[torch.FloatTensor of size 2x3]
torch.cat([torch.Tensor([[1,2,3]]),torch.Tensor([[4,5,6]])],1)
 1  2  3  4  5  6
[torch.FloatTensor of size 1x6]

1.2.3 tensor变形

x = torch.randn(2,3,4)
x
(0 ,.,.) = 
 -0.2756 -0.7231 -0.2633 -0.5982
  1.1090  1.0519  0.7157  0.4336
  1.4142 -0.1507 -0.1062 -1.3992

(1 ,.,.) = 
  0.4155  1.1804 -1.0576 -2.0781
  0.6167 -0.0995 -0.1453  0.2573
 -1.3292 -1.8697 -0.2988 -0.0990
[torch.FloatTensor of size 2x3x4]
x.view(1,12)
RuntimeErrorTraceback (most recent call last)

<ipython-input-33-c96cc26a3a86> in <module>()
----> 1 x.view(1,12)


RuntimeError: invalid argument 2: size '[1 x 12]' is invalid for input of with 24 elements at /pytorch/torch/lib/TH/THStorage.c:41
x.view(12,1)
RuntimeErrorTraceback (most recent call last)

<ipython-input-34-a611982e8650> in <module>()
----> 1 x.view(12,1)


RuntimeError: invalid argument 2: size '[12 x 1]' is invalid for input of with 24 elements at /pytorch/torch/lib/TH/THStorage.c:41
x.view(2,12)
Columns 0 to 9 
-0.2756 -0.7231 -0.2633 -0.5982  1.1090  1.0519  0.7157  0.4336  1.4142 -0.1507
 0.4155  1.1804 -1.0576 -2.0781  0.6167 -0.0995 -0.1453  0.2573 -1.3292 -1.8697

Columns 10 to 11 
-0.1062 -1.3992
-0.2988 -0.0990
[torch.FloatTensor of size 2x12]
x.view(-1)
-0.2756
-0.7231
-0.2633
-0.5982
 1.1090
 1.0519
 0.7157
 0.4336
 1.4142
-0.1507
-0.1062
-1.3992
 0.4155
 1.1804
-1.0576
-2.0781
 0.6167
-0.0995
-0.1453
 0.2573
-1.3292
-1.8697
-0.2988
-0.0990
[torch.FloatTensor of size 24]
x.view(12,-1)
-0.2756 -0.7231
-0.2633 -0.5982
 1.1090  1.0519
 0.7157  0.4336
 1.4142 -0.1507
-0.1062 -1.3992
 0.4155  1.1804
-1.0576 -2.0781
 0.6167 -0.0995
-0.1453  0.2573
-1.3292 -1.8697
-0.2988 -0.0990
[torch.FloatTensor of size 12x2]

1.3 求导机制

import torch.autograd.Variable as Variable
ImportErrorTraceback (most recent call last)

<ipython-input-39-4a31155c778b> in <module>()
----> 1 import torch.autograd.Variable as Variable


ImportError: No module named Variable
import torch.autograd as autograd
x = autograd.Variable(torch.randn(2,3))
y = autograd.Variable(torch.randn(2,3))
x+y
Variable containing:
 0.0876 -0.5458 -0.6579
-0.9776  0.6228  0.7369
[torch.FloatTensor of size 2x3]
z = x+y
z.data
 0.0876 -0.5458 -0.6579
-0.9776  0.6228  0.7369
[torch.FloatTensor of size 2x3]
z
Variable containing:
 0.0876 -0.5458 -0.6579
-0.9776  0.6228  0.7369
[torch.FloatTensor of size 2x3]
z.creator
AttributeErrorTraceback (most recent call last)

<ipython-input-49-a6571e6541ef> in <module>()
----> 1 z.creator


/usr/lib64/python2.7/site-packages/torch/autograd/variable.pyc in __getattr__(self, name)
     63         if name in self._fallthrough_methods:
     64             return getattr(self.data, name)
---> 65         return object.__getattribute__(self, name)
     66 
     67     def __getitem__(self, key):


AttributeError: 'Variable' object has no attribute 'creator'
z1 = autograd.Variable(torch.randn(2,3),requires_grad=True)+autograd.Variable(torch.randn(2,3),requires_grad=True)
z1.data
-1.2218 -0.6063 -1.8739
-1.2457 -2.4399 -0.6936
[torch.FloatTensor of size 2x3]
z1
Variable containing:
-1.2218 -0.6063 -1.8739
-1.2457 -2.4399 -0.6936
[torch.FloatTensor of size 2x3]
z1.creator
AttributeErrorTraceback (most recent call last)

<ipython-input-53-8fec5dcfb2ec> in <module>()
----> 1 z1.creator


/usr/lib64/python2.7/site-packages/torch/autograd/variable.pyc in __getattr__(self, name)
     63         if name in self._fallthrough_methods:
     64             return getattr(self.data, name)
---> 65         return object.__getattribute__(self, name)
     66 
     67     def __getitem__(self, key):


AttributeError: 'Variable' object has no attribute 'creator'
z1.sum()
Variable containing:
-8.0810
[torch.FloatTensor of size 1]
z1.backward
<bound method Variable.backward of Variable containing:
-1.2218 -0.6063 -1.8739
-1.2457 -2.4399 -0.6936
[torch.FloatTensor of size 2x3]
>
z1.backward()
RuntimeErrorTraceback (most recent call last)

<ipython-input-58-19fb924d2b5b> in <module>()
----> 1 z1.backward()


/usr/lib64/python2.7/site-packages/torch/autograd/variable.pyc in backward(self, gradient, retain_graph, create_graph, retain_variables)
    154                 Variable.
    155         """
--> 156         torch.autograd.backward(self, gradient, retain_graph, create_graph, retain_variables)
    157 
    158     def register_hook(self, hook):


/usr/lib64/python2.7/site-packages/torch/autograd/__init__.pyc in backward(variables, grad_variables, retain_graph, create_graph, retain_variables)
     84         grad_variables = list(grad_variables)
     85 
---> 86     grad_variables, create_graph = _make_grads(variables, grad_variables, create_graph)
     87 
     88     if retain_variables is not None:


/usr/lib64/python2.7/site-packages/torch/autograd/__init__.pyc in _make_grads(outputs, grads, user_create_graph)
     32             if out.requires_grad:
     33                 if out.numel() != 1:
---> 34                     raise RuntimeError("grad can be implicitly created only for scalar outputs")
     35                 data = out.data
     36                 new_grads.append(


RuntimeError: grad can be implicitly created only for scalar outputs

!!!求导只能用一个scalar outputs,这个怎么翻译?常量?而不是tensor

# Variable 可以从 Tensor 中创建
x = autograd.Variable( torch.Tensor([1., 2., 3]), requires_grad=True )
# 可以使用.data属性来取出 Variable 中的数据
print(x.data)

# Tensor 支持的运算,Variable 都支持
y = autograd.Variable( torch.Tensor([4., 5., 6]), requires_grad=True )
z = x + y
print(z.data)

# 但是 Variable 的特点是,它在运算中可以自动建立父子节点关系
print(z.creator)
 1
 2
 3
[torch.FloatTensor of size 3]


 5
 7
 9
[torch.FloatTensor of size 3]






AttributeErrorTraceback (most recent call last)

<ipython-input-59-d68a9640cdfd> in <module>()
     10 
     11 # 但是 Variable 的特点是,它在运算中可以自动建立父子节点关系
---> 12 print(z.creator)


/usr/lib64/python2.7/site-packages/torch/autograd/variable.pyc in __getattr__(self, name)
     63         if name in self._fallthrough_methods:
     64             return getattr(self.data, name)
---> 65         return object.__getattribute__(self, name)
     66 
     67     def __getitem__(self, key):


AttributeError: 'Variable' object has no attribute 'creator'
# 让我们把z中的值都加起来(5+7+9)
s = z.sum()
print(s)
print(s.creator)
Variable containing:
 21
[torch.FloatTensor of size 1]






AttributeErrorTraceback (most recent call last)

<ipython-input-60-df2387da0c35> in <module>()
      2 s = z.sum()
      3 print(s)
----> 4 print(s.creator)


/usr/lib64/python2.7/site-packages/torch/autograd/variable.pyc in __getattr__(self, name)
     63         if name in self._fallthrough_methods:
     64             return getattr(self.data, name)
---> 65         return object.__getattribute__(self, name)
     66 
     67     def __getitem__(self, key):


AttributeError: 'Variable' object has no attribute 'creator'
s.backward() 
print(x.grad)
Variable containing:
 1
 1
 1
[torch.FloatTensor of size 3]

1.4 一些参数函数

import torch.nn as nn

1.4.1 f(x)=Ax+b

?nn.Linear
m = nn.Linear(20,30)
input = autograd.Variable(torch.randn(128,64,20))
output=m(input)
print output.size()
torch.Size([128, 64, 30])

1.4.2 tanh或者Relu

import torch.nn.functional as F
data = autograd.Variable(torch.randn(2,3))
data
Variable containing:
-0.0968 -0.5334  0.5023
 1.6757  1.5588  1.4051
[torch.FloatTensor of size 2x3]
F.relu(data)
Variable containing:
 0.0000  0.0000  0.5023
 1.6757  1.5588  1.4051
[torch.FloatTensor of size 2x3]
F.tanh(data)
Variable containing:
-0.0965 -0.4880  0.4639
 0.9323  0.9152  0.8864
[torch.FloatTensor of size 2x3]

1.4.3 softmax

data = autograd.Variable(torch.randn(2,3,4))
data
Variable containing:
(0 ,.,.) = 
  1.0451  0.2925 -1.0171  1.0889
  0.2728 -0.7194 -0.2669 -0.1076
 -1.3558 -0.1216  0.5279 -2.5866

(1 ,.,.) = 
 -0.5489  1.5895  2.1688 -0.2448
 -0.0070  1.4209 -0.6757  0.0670
  0.4146 -0.3724  1.9020  0.5877
[torch.FloatTensor of size 2x3x4]
F.softmax(data)
Variable containing:
(0 ,.,.) = 
  0.8312  0.2147  0.0397  0.7914
  0.5695  0.1052  0.6008  0.4565
  0.1455  0.5624  0.2020  0.0401

(1 ,.,.) = 
  0.1688  0.7853  0.9603  0.2086
  0.4305  0.8948  0.3992  0.5435
  0.8545  0.4376  0.7980  0.9599
[torch.FloatTensor of size 2x3x4]
F.softmax(data).sum()
Variable containing:
 12
[torch.FloatTensor of size 1]
F.softmax(data).data
(0 ,.,.) = 
  0.8312  0.2147  0.0397  0.7914
  0.5695  0.1052  0.6008  0.4565
  0.1455  0.5624  0.2020  0.0401

(1 ,.,.) = 
  0.1688  0.7853  0.9603  0.2086
  0.4305  0.8948  0.3992  0.5435
  0.8545  0.4376  0.7980  0.9599
[torch.FloatTensor of size 2x3x4]
F.softmax(data).data[0]+F.softmax(data).data[1]
 1  1  1  1
 1  1  1  1
 1  1  1  1
[torch.FloatTensor of size 3x4]

说明是对第0维做了softmax,pytorch第零维在左边

更多相关推荐

python语法基本组成单位_Python...

1.基本语法单位标识符是编程语言中允许作为名称的有效字符串集合,用于变量、函数、类等的名称...

继续阅读

python语法基本组成单位_Python...

学习任何一门编程语言,首先要了解这门编程语言的语法要求,掌握好基础语法是学好一门编程语言...

继续阅读

python程序是由什么组成的_pytho...

python程序的基本组成是什么?1.程序中几个基本的概念1.1表达式1.2语句1.3程序(program)1.4函...

继续阅读

python中对象的基本组成

对象  在python中,一切事物都可以视为对象;对象由标识(identity)、类型(type)、值(value)组成...

继续阅读

【PyTorch系列教程一】深度学习...

一、深度学习框架比较    深度学习的基本介绍在这里就不做过多介绍,感兴趣的童鞋可以在网上搜...

继续阅读

Horizon dashboard的基本组成模...

作者:吕嘉伟团队:大云虚机网络  Horizon可分为3层,由上到下分别为Dashboard→PanelGroup→Pan...

继续阅读

网络骨架:Backbone(神经网络基...

利用卷积神经网络处理输入图像,生成深层的特征图,再利用各种算法完成区域生成与损失计算,这...

继续阅读

pytorch学习

这里写自定义目录欢迎使用Markdown编辑器新的改变功能快捷键合理的创建标题,有助于目录的生成...

继续阅读

网络骨架:Backbone(神经网络基...

激活函数层神经网络如果仅仅是由线性的卷积运算堆叠组成,则其无法形成复杂的表达空间,也就很...

继续阅读