forcats: Working with Categorical Data

Introduction


In this module, we will learn to:

  • tabulate levels
  • reorder levels
  • reverse levels
  • collapse levels
  • recode levels
  • recategorize levels
  • shift levels

Case Study


We will use a case study to explore the various features of the forcats package. You can download the data for the case study from here or directly import the data using the readr package. In this case study, we will:

  • compute the frequency of different referrers
  • collapse referrers with low sample size into a single group
  • club traffic from social media websites into a new category
  • group referrers with traffic below a threshold into a single category

Data


The data set we will use has just one column traffics i.e. the source of traffic for a imaginary website.

traffic

Data


Let us extract the traffics column from the above data using and save it in a new variable traffics. Instead of using traffic which is a tibble, we will use traffics which is a vector. We do this to avoid extracting the traffics column from the main data repeatedly.

traffics

Count


Count


Let us compute the traffic driven by different referrers using fct_count.

fct_count(traffics)

Levels


Use levels() to display the categories/levels.

levels(traffics)

Reorder


Reorder


The levels can be reordered based on their frequency using fct_infreq().

freq_traffic <- fct_infreq(traffics)
levels(freq_traffic)

Reorder


Reorder


The levels can be reordered based on the order of appearance using fct_inorder().

order_traffic <- fct_inorder(traffics)
levels(order_traffic)

Reverse Levels


Reverse Levels


The levels can be reversed using fct_rev().

reversed <- 
reversed <- fct_rev(traffics)
levels(reversed)

Collapse Categories


Collapse Categories


We want to group some of the referrers into 2 categories:

  • social
  • search

To group categories/levels, we will use fct_collapse().

traffics %>% 
  fct_collapse(
  social = c("facebook", "twitter", "instagram"),
  search = c("google", "bing", "yahoo")) %>% 
  fct_count() 

Recode Levels


Recode Levels


Categories/levels can be grouped using fct_recode() as well:

fct_recode(traffics, search = "bing", search = "yahoo", search = "google",
  social = "facebook", social = "twitter", social = "instagram") %>%
  levels()

Lump Categories


Lump Categories


Let us group together referrer types that drive low traffic to the website. Use fct_lump() to lump together categories.

traffics %>% 
  fct_lump() %>% 
  table()

Lump Categories


We want to retain the top 3 referrers and combine the rest of them into a single category. Use fct_lump() and set the argument n to 3 indicating we want to retain top 3 categories and combine the rest.

traffics %>% 
  fct_lump(n = 3) %>% 
  table()

Lump Categories


Lump Categories


Let us combine referrers that drive less than 10% traffic to the website. Since we are looking at proportion of traffic driven to the website and not the actual numbers, we use the prop argument and set it to 0.1, indicating that we want to retain only those categories which have a proportion of more than 10% and combine the rest.

traffics %>% 
  fct_lump(prop = 0.10) %>% 
  table()

Lump Categories


What if we want to retain 3 referrers which drive the lowest traffic to the website and combine the rest? We will still use the n argument but instead of specifying 3, we now specify -3.

traffics %>% 
  fct_lump(n = -3) %>% 
  table()

Lump Categories


Let us see how to retain referrers that drive less than 10 % traffic to the website and combine the rest into a single group.

traffics %>% 
  fct_lump(prop = -0.1) %>% 
  table()

Replace Levels


Let us assume we want to retain a couple of important categories and group the rest into a single category. In the below example, we retain google and yahoo while grouping the rest as others using fct_other().

Replace Levels with Other


keep_gy <- 
keep_gy <- fct_other(traffics, keep = c("google", "yahoo"))
levels(keep_gy)

Replace Levels


What if you want to drop a couple of categories instead of grouping them? Use the drop argument in fct_other() and specify the categories to be dropped. In the below example, we drop the following referrer categories:

  • instagram
  • twitter

Replace Levels with Other


drop_it <- 
drop_it <- fct_other(traffics, drop = c("instagram", "twitter")) 
levels(drop_it)

Reorder Levels


The categories can be reordered using fct_relevel(). In the below example, we reorder the categories to ensure google appears first.

Reorder Levels


Similarly, in the below example, we reorder the levels to ensure twitter appears first irrespective of its frequency or order of appearance in the data.

move_twitter <- 
move_twitter <- fct_relevel(traffics, "twitter") 
levels(move_twitter)

Reorder Levels


If the category needs to appear at a particular position, use the after argument and specify the position after which it should appear. For example, if google should be the third category, we would specify after = 2 i.e. google should come after the 2nd position (i.e. third position).

Reorder Levels


appear_2 <- 
appear_2 <- fct_relevel(traffics, "google", after = 2)
levels(appear_2)

Reorder Levels


If the category should appear last, supply the value Inf (infinity) to the after argument as shown below.

Reorder Levels


appear_last <- 
appear_last <- fct_relevel(traffics, "facebook", after = Inf) 
levels(appear_last)

Case Study 3


In this case study, we deal with categorical data which is ordered and cyclical. It contains response to an imaginary survey.

Data


##    [1] dislike          like somewhat    dislike          neutral         
##    [5] dislike          like somewhat    like somewhat    like somewhat   
##    [9] dislike somewhat neutral          like             dislike somewhat
##   [13] dislike somewhat dislike          like somewhat    like somewhat   
##   [17] like             like             dislike          dislike somewhat
##   [21] dislike somewhat neutral          like             like somewhat   
##   [25] dislike somewhat neutral          like             dislike         
##   [29] dislike somewhat like             like somewhat    dislike somewhat
##   [33] like             like             neutral          dislike somewhat
##   [37] like somewhat    like             like somewhat    neutral         
##   [41] dislike somewhat like somewhat    dislike somewhat dislike         
##   [45] like somewhat    dislike somewhat dislike          dislike somewhat
##   [49] like             dislike          like             dislike         
##   [53] like             dislike          like             like somewhat   
##   [57] neutral          neutral          like             like            
##   [61] dislike          dislike          dislike          like            
##   [65] like             dislike          dislike          dislike         
##   [69] dislike somewhat neutral          like             neutral         
##   [73] like somewhat    dislike somewhat dislike          neutral         
##   [77] dislike          like somewhat    like             neutral         
##   [81] like somewhat    dislike          like             like somewhat   
##   [85] neutral          like             like somewhat    like somewhat   
##   [89] dislike somewhat dislike          dislike somewhat like            
##   [93] like             like somewhat    dislike          like somewhat   
##   [97] dislike somewhat dislike          dislike somewhat dislike somewhat
##  [101] neutral          like somewhat    neutral          dislike somewhat
##  [105] like             like             dislike somewhat neutral         
##  [109] like             dislike somewhat like             like somewhat   
##  [113] dislike          like             dislike somewhat dislike somewhat
##  [117] dislike          like somewhat    dislike somewhat dislike         
##  [121] like somewhat    like             dislike somewhat like somewhat   
##  [125] like             dislike somewhat neutral          dislike somewhat
##  [129] like somewhat    dislike          dislike          like somewhat   
##  [133] dislike somewhat like             like             like            
##  [137] dislike          like             like somewhat    neutral         
##  [141] neutral          dislike somewhat neutral          like somewhat   
##  [145] neutral          like             dislike somewhat like somewhat   
##  [149] like somewhat    like             like somewhat    neutral         
##  [153] like somewhat    neutral          neutral          dislike somewhat
##  [157] like somewhat    dislike          like somewhat    like            
##  [161] neutral          like             dislike          like somewhat   
##  [165] like somewhat    like             dislike somewhat like somewhat   
##  [169] dislike          dislike somewhat like             dislike somewhat
##  [173] dislike somewhat like             like             like            
##  [177] like somewhat    like             like somewhat    neutral         
##  [181] like somewhat    dislike somewhat dislike somewhat like somewhat   
##  [185] like             dislike          dislike          dislike         
##  [189] like             dislike somewhat like somewhat    like            
##  [193] dislike          like             dislike somewhat like            
##  [197] like somewhat    dislike          dislike          like somewhat   
##  [201] neutral          neutral          like             dislike         
##  [205] dislike somewhat dislike          dislike somewhat like somewhat   
##  [209] dislike          dislike          dislike          like            
##  [213] like             neutral          neutral          neutral         
##  [217] like somewhat    like             dislike          neutral         
##  [221] dislike          like             neutral          like somewhat   
##  [225] like somewhat    like             neutral          like somewhat   
##  [229] like somewhat    like somewhat    neutral          dislike         
##  [233] like somewhat    dislike          dislike          like somewhat   
##  [237] dislike          dislike somewhat like somewhat    neutral         
##  [241] dislike          dislike          like             dislike         
##  [245] like             dislike somewhat like somewhat    dislike         
##  [249] dislike somewhat like somewhat    neutral          like somewhat   
##  [253] like somewhat    dislike somewhat dislike somewhat dislike somewhat
##  [257] neutral          dislike somewhat neutral          dislike         
##  [261] neutral          dislike somewhat dislike          like somewhat   
##  [265] dislike somewhat like somewhat    like             dislike somewhat
##  [269] like somewhat    dislike somewhat like somewhat    like            
##  [273] neutral          dislike somewhat dislike somewhat neutral         
##  [277] neutral          like             neutral          dislike         
##  [281] dislike          dislike somewhat like             like            
##  [285] like             dislike somewhat dislike somewhat like somewhat   
##  [289] neutral          like             dislike          dislike         
##  [293] like somewhat    neutral          dislike somewhat like            
##  [297] dislike          dislike          dislike somewhat neutral         
##  [301] neutral          neutral          dislike          like            
##  [305] like             like somewhat    dislike somewhat dislike         
##  [309] like somewhat    dislike          neutral          like somewhat   
##  [313] dislike          dislike          dislike somewhat neutral         
##  [317] like             dislike          like somewhat    neutral         
##  [321] dislike somewhat like             dislike          like somewhat   
##  [325] dislike somewhat like somewhat    like             dislike         
##  [329] dislike somewhat neutral          dislike          dislike somewhat
##  [333] like             dislike          dislike somewhat dislike somewhat
##  [337] like somewhat    neutral          neutral          dislike somewhat
##  [341] dislike somewhat like             dislike          like somewhat   
##  [345] dislike          like             neutral          neutral         
##  [349] dislike somewhat like somewhat    like somewhat    like            
##  [353] like somewhat    like             like             dislike somewhat
##  [357] like             like somewhat    dislike          dislike         
##  [361] neutral          neutral          neutral          like            
##  [365] dislike somewhat like             like somewhat    dislike         
##  [369] like             dislike          like somewhat    like            
##  [373] neutral          dislike          dislike somewhat dislike somewhat
##  [377] like somewhat    neutral          like somewhat    like            
##  [381] like             like somewhat    like somewhat    like somewhat   
##  [385] like             dislike          like somewhat    neutral         
##  [389] neutral          like somewhat    like             dislike somewhat
##  [393] neutral          dislike somewhat dislike somewhat dislike         
##  [397] like somewhat    neutral          dislike somewhat neutral         
##  [401] like somewhat    like             like             dislike somewhat
##  [405] dislike somewhat neutral          neutral          neutral         
##  [409] like somewhat    dislike somewhat dislike          neutral         
##  [413] like             like somewhat    like             like            
##  [417] like             like             neutral          dislike somewhat
##  [421] dislike somewhat like somewhat    dislike          neutral         
##  [425] dislike          dislike somewhat like             dislike         
##  [429] dislike somewhat like             like somewhat    like            
##  [433] dislike          like             neutral          dislike         
##  [437] like somewhat    like             like somewhat    dislike         
##  [441] neutral          neutral          dislike somewhat dislike         
##  [445] dislike somewhat dislike          dislike          like            
##  [449] neutral          neutral          dislike          neutral         
##  [453] dislike          dislike          neutral          neutral         
##  [457] like             neutral          neutral          dislike         
##  [461] dislike          dislike somewhat neutral          dislike         
##  [465] like             dislike          like somewhat    like somewhat   
##  [469] dislike somewhat dislike somewhat like somewhat    dislike somewhat
##  [473] like somewhat    like             like             like            
##  [477] like             like             dislike somewhat like somewhat   
##  [481] neutral          like somewhat    like             like somewhat   
##  [485] dislike          dislike somewhat dislike somewhat dislike         
##  [489] neutral          neutral          dislike          dislike somewhat
##  [493] like somewhat    dislike somewhat dislike somewhat neutral         
##  [497] dislike          dislike somewhat dislike          neutral         
##  [501] dislike          dislike          dislike somewhat like            
##  [505] like             like somewhat    dislike somewhat neutral         
##  [509] like             dislike          dislike          like            
##  [513] neutral          like             dislike          like somewhat   
##  [517] dislike          like             like             dislike somewhat
##  [521] neutral          like somewhat    like             neutral         
##  [525] dislike somewhat dislike          like somewhat    neutral         
##  [529] like somewhat    dislike          neutral          neutral         
##  [533] dislike          like             dislike somewhat like somewhat   
##  [537] neutral          like somewhat    like             like            
##  [541] dislike somewhat neutral          like             dislike         
##  [545] like somewhat    like somewhat    dislike          like            
##  [549] like             like somewhat    neutral          like            
##  [553] like somewhat    dislike somewhat like somewhat    neutral         
##  [557] dislike somewhat like             dislike          like            
##  [561] neutral          dislike          like             like            
##  [565] dislike          dislike somewhat dislike somewhat neutral         
##  [569] like somewhat    like             like             dislike somewhat
##  [573] dislike          neutral          like somewhat    dislike somewhat
##  [577] neutral          like somewhat    like somewhat    like            
##  [581] dislike somewhat dislike somewhat like             like            
##  [585] dislike          neutral          like somewhat    dislike         
##  [589] dislike          dislike somewhat neutral          dislike         
##  [593] dislike          like somewhat    like somewhat    like somewhat   
##  [597] dislike          dislike          dislike          like somewhat   
##  [601] neutral          dislike          dislike          dislike somewhat
##  [605] dislike somewhat dislike          like             like somewhat   
##  [609] dislike somewhat neutral          dislike somewhat dislike somewhat
##  [613] dislike          like somewhat    like             like            
##  [617] like             dislike somewhat like somewhat    dislike         
##  [621] neutral          like             neutral          dislike somewhat
##  [625] neutral          like somewhat    dislike          dislike somewhat
##  [629] dislike          like             like somewhat    neutral         
##  [633] dislike          like             neutral          neutral         
##  [637] like             neutral          dislike          like somewhat   
##  [641] neutral          dislike          like             like somewhat   
##  [645] dislike somewhat like somewhat    dislike somewhat like            
##  [649] dislike          neutral          like somewhat    dislike somewhat
##  [653] like somewhat    neutral          dislike somewhat dislike         
##  [657] like             like             dislike          neutral         
##  [661] like             like somewhat    like somewhat    like            
##  [665] like somewhat    dislike          like somewhat    like            
##  [669] like             dislike somewhat dislike          dislike somewhat
##  [673] like somewhat    like             like somewhat    like somewhat   
##  [677] dislike somewhat neutral          like             like            
##  [681] like             dislike somewhat like             dislike somewhat
##  [685] dislike          dislike somewhat neutral          neutral         
##  [689] neutral          dislike somewhat neutral          dislike         
##  [693] like             like             dislike          like somewhat   
##  [697] dislike somewhat dislike          neutral          dislike         
##  [701] dislike          neutral          like             dislike         
##  [705] like             dislike          dislike somewhat neutral         
##  [709] neutral          like somewhat    dislike somewhat neutral         
##  [713] dislike somewhat like somewhat    like             dislike         
##  [717] dislike          neutral          dislike          like            
##  [721] neutral          dislike          neutral          neutral         
##  [725] dislike somewhat like somewhat    like             like            
##  [729] like somewhat    dislike          like somewhat    like somewhat   
##  [733] like             dislike          dislike          like somewhat   
##  [737] dislike somewhat like             dislike          neutral         
##  [741] like             dislike somewhat like somewhat    like            
##  [745] like             like somewhat    like somewhat    like somewhat   
##  [749] dislike somewhat dislike          neutral          like somewhat   
##  [753] like somewhat    like somewhat    like             neutral         
##  [757] dislike          dislike          dislike          dislike         
##  [761] dislike somewhat neutral          like             dislike somewhat
##  [765] like somewhat    neutral          like somewhat    neutral         
##  [769] like somewhat    dislike somewhat dislike          like somewhat   
##  [773] dislike somewhat like             like somewhat    dislike somewhat
##  [777] like somewhat    like somewhat    dislike somewhat like            
##  [781] like somewhat    like somewhat    dislike          like somewhat   
##  [785] dislike          neutral          dislike somewhat like            
##  [789] neutral          like             like             like            
##  [793] neutral          like somewhat    dislike          dislike         
##  [797] like             dislike somewhat neutral          dislike somewhat
##  [801] dislike          like             like somewhat    dislike         
##  [805] like somewhat    dislike          neutral          neutral         
##  [809] dislike          neutral          like somewhat    dislike somewhat
##  [813] like             dislike somewhat neutral          dislike         
##  [817] like             like somewhat    dislike          neutral         
##  [821] neutral          like somewhat    neutral          dislike somewhat
##  [825] dislike          dislike somewhat neutral          neutral         
##  [829] like             like             neutral          dislike         
##  [833] dislike          like somewhat    like             neutral         
##  [837] like             dislike          like somewhat    like            
##  [841] neutral          neutral          like             like somewhat   
##  [845] dislike somewhat neutral          dislike          neutral         
##  [849] like somewhat    like             like somewhat    like somewhat   
##  [853] neutral          like             dislike somewhat like somewhat   
##  [857] like             like somewhat    like somewhat    dislike         
##  [861] like somewhat    like             neutral          like somewhat   
##  [865] neutral          dislike somewhat dislike somewhat like            
##  [869] like somewhat    dislike somewhat like             like somewhat   
##  [873] like somewhat    dislike          dislike          like            
##  [877] neutral          neutral          like             dislike         
##  [881] like             like             neutral          dislike somewhat
##  [885] neutral          dislike          dislike          like            
##  [889] dislike somewhat dislike          dislike          like            
##  [893] dislike somewhat neutral          dislike          like            
##  [897] like somewhat    like             dislike somewhat dislike somewhat
##  [901] neutral          like somewhat    neutral          like somewhat   
##  [905] like             like             dislike somewhat dislike         
##  [909] like somewhat    like             like             like somewhat   
##  [913] dislike          like somewhat    dislike somewhat neutral         
##  [917] like             neutral          dislike somewhat like somewhat   
##  [921] like             like             like             like            
##  [925] dislike          dislike          dislike          dislike somewhat
##  [929] like             dislike somewhat like             dislike somewhat
##  [933] dislike somewhat dislike          like somewhat    neutral         
##  [937] like somewhat    neutral          dislike          neutral         
##  [941] like somewhat    like             like             like            
##  [945] dislike          like somewhat    dislike somewhat like somewhat   
##  [949] dislike          like             like             dislike         
##  [953] like             like             dislike somewhat like somewhat   
##  [957] like somewhat    dislike          dislike          like somewhat   
##  [961] dislike          neutral          dislike          like somewhat   
##  [965] dislike          like somewhat    dislike          like somewhat   
##  [969] like somewhat    dislike somewhat like somewhat    dislike somewhat
##  [973] dislike somewhat dislike          like somewhat    like            
##  [977] neutral          dislike          dislike          dislike         
##  [981] like             dislike          dislike somewhat like somewhat   
##  [985] like somewhat    like somewhat    dislike          neutral         
##  [989] like             like somewhat    dislike somewhat dislike         
##  [993] like             like somewhat    dislike          like            
##  [997] dislike somewhat like somewhat    like somewhat    neutral         
## 5 Levels: like < like somewhat < neutral < ... < dislike

Shift Levels


Shift Levels


To shift the levels, we use fct_shift(). Use the n argument to indicate the direction of the shift. If n is positive, the levels are shifted to the left else to the right. In the below example, we shift the levels to the left by 2 positions.

shift_left <- 
shift_left <- fct_shift(responses, 2) 
levels(shift_left)

Shift Levels


Shift Levels


To shift the levels to the right, supply a negative value to the n argument in fct_shift(). In the below example, we shift the levels to the right by 2 positions.

shift_right <- 
shift_right <- fct_shift(responses, -2) 
levels(shift_right)