Skip to Main Content
fastText Quick Start Guide
book

fastText Quick Start Guide

by Joydeep Bhattacharjee
July 2018
Intermediate to advanced content levelIntermediate to advanced
194 pages
5h 22m
English
Packt Publishing
Content preview from fastText Quick Start Guide

Hierarchical softmax

Finding the softmax is highly computationally intensive. For each training instance, we have to iterate through every word in the vocabulary and compute the softmax. Thus, it is impractical to scale up to large vocabularies and large training corpora. To solve this problem, there are two approaches that are used in fastText: the hierarchical softmax and the negative sampling approach. We will discuss hierarchical softmax in this section and will discuss negative sampling in the next section. In both the approaches, the trick is to recognize that we don't need to update all the output vectors per training instance.

In hierarchical softmax, a binary tree is computed to represent all the words in the vocabulary. The V words ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

WebAssembly: The Definitive Guide

WebAssembly: The Definitive Guide

Brian Sletten
Spark: The Definitive Guide

Spark: The Definitive Guide

Bill Chambers, Matei Zaharia
Learning Go

Learning Go

Jon Bodner

Publisher Resources

ISBN: 9781789130997Supplemental Content