November 2018
Intermediate to advanced
272 pages
6h 51m
Japanese
book
エレガントなSciPy ―Pythonによる科学技術計算
by Juan Nunez-Iglesias, Stéfan van der Walt, Harriet Dashnow, 山崎 邦子, 山崎 康宏
Content preview from エレガントなSciPy ―Pythonによる科学技術計算
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
189
8
章
Toolz
を使って小さなノートパソコン
でビッグデータを処理する方法
グレイシー：ナイフですって？ヤツは身長が 12 フィートもあるのよ！
ジャック：7 フィートだよ。大丈夫だって、なんとかできると思うよ。
―
ジャック・バートン『ゴーストハンターズ』
ストリーミングは、
SciPy
自体の機能ではありませんが、科学の世界でよく見られるような大
規模なデータセットを効率よく処理できるようにする手法です。
Python
には、ストリーミング
でデータ処理を行うのに役立つ基本構造がいくつか用意されており、これらを
Matt Rocklin
の
Toolz
ライブラリと組み合わせると、極めてメモリ効率がよく、エレガントで簡潔なコードを書
くことができます。本章では、ストリーミング処理の概念を適用して、お使いのコンピュータの
RAM
に載るよりもはるかに大きなデータセットを取り扱えるようにする方法を解説します。
ストリーミング処理という概念を知らずに、すでにストリーミングを実行していた人もいるで
しょう。おそらく最も単純なストリーミングは、ファイル全体をメモリに読み込むことなく、ファ
イル中の行を反復して、
1
行ずつ処理することでしょう。例えば、以下のような、各行の平均値を
計算して総和を取るループなどがそうです。
import numpy as np
with open('data/expr.tsv') as f:
sum_of_means = 0
for line in f:
sum_of_means += np.mean(np.fromstring(line, ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access
More than 5,000 organizations count on O’Reilly
O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.Julian F.
I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.Addison B.
I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.Amir M.
I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.