Using missRanger

Overview

{missRanger} is a multivariate imputation algorithm based on random forests. It is a fast alternative to the beautiful ‘MissForest’ algorithm of Stekhoven and Buehlmann (2011), and uses the {ranger} package (Wright and Ziegler 2017) to fit the random forests.

The algorithm iterates until the average out-of-bag (OOB) error of the forests stops improving. The missing values are filled by OOB predictions of the best iteration.

{missRanger} is fast.
It allows for out-of-sample applications.
It is intuitive: E.g., calling missRanger(data, . ~ 1) would impute all variables univariately, while missRanger(data, Species ~ Sepal.Width) would use Sepal.Width to impute Species.
It works for a variety of data types.
It combines random forest imputation with predictive mean matching. This avoids “new” values like 0.3334 in a 0-1 coded variable and helps to raise the variance of the imputations, which is especially important for multiple imputation (see additional vignettes).

Installation

# From CRAN
install.packages("missRanger")

# Development version
devtools::install_github("mayer79/missRanger")

Usage

library(missRanger)

set.seed(3)

iris_NA <- generateNA(iris, p = 0.1)
head(iris_NA)
#>   Sepal.Length Sepal.Width Petal.Length Petal.Width Species
#> 1          5.1         3.5          1.4         0.2  setosa
#> 2          4.9         3.0          1.4          NA  setosa
#> 3          4.7         3.2          1.3         0.2  setosa
#> 4          4.6         3.1          1.5         0.2    <NA>
#> 5           NA         3.6          1.4         0.2  setosa
#> 6          5.4         3.9          1.7         0.4    <NA>
 
imp <- missRanger(iris_NA, num.trees = 100)
#> 
#> Variables to impute:     Sepal.Length, Sepal.Width, Petal.Length, Petal.Width, Species
#> Variables used to impute:    Sepal.Length, Sepal.Width, Petal.Length, Petal.Width, Species
#> 
#> iter 1 
#>   |                                                                              |                                                                      |   0%  |                                                                              |==============                                                        |  20%  |                                                                              |============================                                          |  40%  |                                                                              |==========================================                            |  60%  |                                                                              |========================================================              |  80%  |                                                                              |======================================================================| 100%
#> iter 2 
#>   |                                                                              |                                                                      |   0%  |                                                                              |==============                                                        |  20%  |                                                                              |============================                                          |  40%  |                                                                              |==========================================                            |  60%  |                                                                              |========================================================              |  80%  |                                                                              |======================================================================| 100%
#> iter 3 
#>   |                                                                              |                                                                      |   0%  |                                                                              |==============                                                        |  20%  |                                                                              |============================                                          |  40%  |                                                                              |==========================================                            |  60%  |                                                                              |========================================================              |  80%  |                                                                              |======================================================================| 100%
#> iter 4 
#>   |                                                                              |                                                                      |   0%  |                                                                              |==============                                                        |  20%  |                                                                              |============================                                          |  40%  |                                                                              |==========================================                            |  60%  |                                                                              |========================================================              |  80%  |                                                                              |======================================================================| 100%
head(imp)
#>   Sepal.Length Sepal.Width Petal.Length Petal.Width Species
#> 1     5.100000         3.5          1.4   0.2000000  setosa
#> 2     4.900000         3.0          1.4   0.1608667  setosa
#> 3     4.700000         3.2          1.3   0.2000000  setosa
#> 4     4.600000         3.1          1.5   0.2000000  setosa
#> 5     5.061255         3.6          1.4   0.2000000  setosa
#> 6     5.400000         3.9          1.7   0.4000000  setosa

Predictive mean matching

It worked, but the new values appear overly exact. To avoid this, we can add predictive mean matching (PMM) to the OOB predictions:

imp <- missRanger(iris_NA, pmm.k = 5, num.trees = 100, verbose = 0)
head(imp)
#>   Sepal.Length Sepal.Width Petal.Length Petal.Width Species
#> 1          5.1         3.5          1.4         0.2  setosa
#> 2          4.9         3.0          1.4         0.2  setosa
#> 3          4.7         3.2          1.3         0.2  setosa
#> 4          4.6         3.1          1.5         0.2  setosa
#> 5          5.4         3.6          1.4         0.2  setosa
#> 6          5.4         3.9          1.7         0.4  setosa

Controlling the random forests

missRanger() offers many options. How would we use one feature per split (mtry = 1) with 200 trees?

imp <- missRanger(iris_NA, pmm.k = 5, num.trees = 200, mtry = 1, verbose = 0)

Extended output

Setting data_only = FALSE (or keep_forests = TRUE) returns a “missRanger” object. With keep_forests = TRUE, this allows for out-of-sample applications:

imp <- missRanger(
  iris_NA, pmm.k = 5, num.trees = 100, keep_forests = TRUE, verbose = 0
)
imp
#> missRanger object. Extract imputed data via $data
#> - best iteration: 3 
#> - best average OOB imputation error: 0.1468982

summary(imp)
#> missRanger object. Extract imputed data via $data
#> - best iteration: 3 
#> - best average OOB imputation error: 0.1468982 
#> 
#> Sequence of OOB prediction errors:
#> 
#>      Sepal.Length Sepal.Width Petal.Length Petal.Width    Species
#> [1,]    1.0000000   1.1108502   0.39671941  0.18322253 0.06666667
#> [2,]    0.2224743   0.5371919   0.06000731  0.05568752 0.03703704
#> [3,]    0.1732113   0.4517314   0.02408501  0.05583381 0.02962963
#> [4,]    0.1796650   0.4715697   0.02106975  0.05502143 0.03703704
#> 
#> Mean performance per iteration:
#> [1] 0.5514918 0.1824796 0.1468982 0.1528726
#> 
#> First rows of imputed data:
#> 
#>   Sepal.Length Sepal.Width Petal.Length Petal.Width Species
#> 1          5.1         3.5          1.4         0.2  setosa
#> 2          4.9         3.0          1.4         0.2  setosa
#> 3          4.7         3.2          1.3         0.2  setosa

# Out-of-sample application
# saveRDS(imp, file = "imputation_model.rds")
# imp <- readRDS("imputation_model.rds")
predict(imp, head(iris_NA))
#>   Sepal.Length Sepal.Width Petal.Length Petal.Width Species
#> 1          5.1         3.5          1.4         0.2  setosa
#> 2          4.9         3.0          1.4         0.2  setosa
#> 3          4.7         3.2          1.3         0.2  setosa
#> 4          4.6         3.1          1.5         0.2  setosa
#> 5          5.1         3.6          1.4         0.2  setosa
#> 6          5.4         3.9          1.7         0.4  setosa

Formulas

By default, missRanger() uses all columns to impute all columns with missings.

This can be modified by passing a formula: The left hand side specifies the variables to be imputed, while the right hand side lists the variables used for imputation.

# Impute all variables with all (default)
m <- missRanger(iris_NA, formula = . ~ ., pmm.k = 5, num.trees = 100, verbose = 0)

# Don't use Species for imputation
m <- missRanger(iris_NA, . ~ . - Species, pmm.k = 5, num.trees = 100, verbose = 0)

# Impute Sepal.Length by Species (or not?)
m <- missRanger(iris_NA, Sepal.Length ~ Species, pmm.k = 5, num.trees = 100)
#> 
#> Variables to impute:     Sepal.Length
#> Variables used to impute:    
#> 
#> iter 1 
#>   |                                                                              |                                                                      |   0%  |                                                                              |======================================================================| 100%
head(m)
#>   Sepal.Length Sepal.Width Petal.Length Petal.Width Species
#> 1          5.1         3.5          1.4         0.2  setosa
#> 2          4.9         3.0          1.4          NA  setosa
#> 3          4.7         3.2          1.3         0.2  setosa
#> 4          4.6         3.1          1.5         0.2    <NA>
#> 5          6.2         3.6          1.4         0.2  setosa
#> 6          5.4         3.9          1.7         0.4    <NA>

# Only univariate imputation was done! Why? Because Species contains missing values
# itself and needs to appear on the LHS as well:
m <- missRanger(iris_NA, Sepal.Length + Species ~ Species, pmm.k = 5, num.trees = 100)
#> 
#> Variables to impute:     Sepal.Length, Species
#> Variables used to impute:    Species
#> 
#> iter 1 
#>   |                                                                              |                                                                      |   0%  |                                                                              |===================================                                   |  50%  |                                                                              |======================================================================| 100%
head(m)
#>   Sepal.Length Sepal.Width Petal.Length Petal.Width    Species
#> 1          5.1         3.5          1.4         0.2     setosa
#> 2          4.9         3.0          1.4          NA     setosa
#> 3          4.7         3.2          1.3         0.2     setosa
#> 4          4.6         3.1          1.5         0.2 versicolor
#> 5          6.5         3.6          1.4         0.2     setosa
#> 6          5.4         3.9          1.7         0.4 versicolor

# Impute all variables univariately
m <- missRanger(iris_NA, . ~ 1, verbose = 0)

Speed-up things

missRanger() fits a random forest per variable and iteration. Thus, imputation can take long. Some tweaks:

Use less trees, e.g., num.trees = 100.
Use a smaller tree depth, e.g., max.depth = 6.
Use large leaves, e.g., min.node.size = 100.
Use smaller bootstrap samples, e.g., sample.fraction = 0.2.
Use less iterations, e.g., max.iter = 3.

The first three items also help to greatly reduce the size of the models, which might become relevant in out-of-sample applications with keep_forests = TRUE.

Trick: Use `case.weights` to reduce impact of rows with many missings

Using the case.weights argument, you can pass case weights to the imputation models. For instance, this allows to reduce the contribution of rows with many missings:

m <- missRanger(
  iris_NA,
  num.trees = 100,
  pmm.k = 5,
  case.weights = rowSums(!is.na(iris_NA))
)

References

Stekhoven, Daniel J., and Peter Buehlmann. 2011. “MissForest-non-parametric missing value imputation for mixed-type data.” Bioinformatics 28 (1): 112–18. https://doi.org/10.1093/bioinformatics/btr597.

Wright, Marvin, and Andreas Ziegler. 2017. “Ranger: A Fast Implementation of Random Forests for High Dimensional Data in c++ and r.” Journal of Statistical Software, Articles 77 (1): 1–17. https://doi.org/10.18637/jss.v077.i01.