July 2018
Intermediate to advanced
310 pages
8h 21m
Chinese
Content preview from 深度学习入门 : 基于Python的理论与实现
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
第 8章 深度学习
252
8.3.4
运算精度的位数缩减
在深度学习的高速化中，除了计算量之外，内存容量、总线带宽等也有
可能成为瓶颈。关于内存容量，需要考虑将大量的权重参数或中间数据放在
内存中。关于总线带宽，当流经
GPU
（或者
CPU
）总线的数据超过某个限制时，
就会成为瓶颈。考虑到这些情况，我们希望尽可能减少流经网络的数据的位数。
计算机中为了表示实数，主要使用64 位或者32 位的浮点数。通过使用
较多的位来表示数字，虽然数值计算时的误差造成的影响变小了，但计算的
处理成本、内存使用量却相应地增加了，还给总线带宽带来了负荷。
关于数值精度（用几位数据表示数值），我们已经知道深度学习并不那么
需要数值精度的位数。这是神经网络的一个重要性质。这个性质是基于神经
网络的健壮性而产生的。这里所说的健壮性是指，比如，即便输入图像附有
一些小的噪声，输出结果也仍然保持不变。可以认为，正是因为有了这个健
壮性，流经网络的数据即便有所“劣化”，对输出结果的影响也较小。
计算机中表示小数时，有 32 位的单精度浮点数和64 位的双精度浮点数
等格式。根据以往的实验结果，在深度学习中，即便是16 位的半精度浮点
数（
half float
），也可以顺利地进行学习
[30]
。实际上，
NVIDIA
的下一代
GPU
框架
Pascal
也支持半精度浮点数的运算，由此可以认为今后半精度浮点数将
被作为标准使用。
NVIDIA的Maxwell GPU虽然支持半精度浮点数的存储（保存数据
的功能），但是运算本身不是用16位进行的。下一代的Pascal ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.