Deep Learning with PyTorch

torchtext.datasets

The torchtext.datasets instance provide wrappers for using different datasets like IMDB, TREC (question classification), language modeling (WikiText-2), and a few other datasets. We will use torch.datasets to download the IMDB dataset and split it into train and test datasets. The following code does that, and when you run it for the first time it could take several minutes, depending on your broadband connection, as it downloads the IMDB datasets from the internet:

train, test = datasets.IMDB.splits(TEXT, LABEL)

The previous dataset's IMDB class abstracts away all the complexity involved in downloading, tokenizing, and splitting the database into train and test datasets. train.fields contains a dictionary where TEXT is ...