Hands-On Deep Learning with Go

Getting data into Pachyderm

Let's prepare our data. In this case, we are using the CIFAR-10 dataset from Chapter 6, Object Recognition with Convolutional Neural Networks. If you need a refresher, pull the data from the source at the University of Toronto, like so:

wget https://www.cs.toronto.edu/~kriz/cifar-10-binary.tar.gz...cifar-10-binary.tar.gz 100%[==================================>] 162.17M 833KB/s in 2m 26s

Extract the data to a temporary directory, and create repo in Pachyderm:

# pachctl create repo data# pachctl list repoNAME CREATED SIZE (MASTER)data 8 seconds ago 0Bbash-3.2$

Now that we've got a repository, let's fill it with our CIFAR-10 image data. First, let's create individual directories and break up the various CIFAR-10 ...