Rによるテキストマイニング ―tidytextを活用したデータ分析と可視化の基礎

4.1

 

n

グラムによるトークン化

■

57

austen_books() %>%

unnest_tokens(trigram, text, token = "ngrams", n = 3) %>%

separate(trigram, c("word1", "word2", "word3"), sep = " ") %>%

filter(!word1 %in% stop_words$word,

!word2 %in% stop_words$word,

!word3 %in% stop_words$word) %>%

count(word1, word2, word3, sort = TRUE)

## # A tibble: 8,757 x 4

## word1 word2 word3 n

## <chr>

<chr> <chr> <int>

## 1 dear miss woodhouse 23

## 2 miss de bourgh 18

## 3 lady catherine de 14

## 4 catherine de bourgh 13

## 5 poor miss taylor 11

## 6 sir walter elliot 11

## 7 ten thousand pounds 11

## 8 dear sir thomas 10

## 9 twenty thousand pounds 8

## 10

replied miss crawford 7

## # ... with 8,747 ...