fastverse

CRAN status cran checks R build status status downloads per month downloads

The fastverse is a suite of complementary high-performance packages for statistical computing and data manipulation in R. Developed independently by various people, fastverse packages jointly contribute to the objectives of:

The fastverse package integrates, and provides utilities for easy installation, loading and management of these packages. It is an extensible framework that allows users to (permanently) add or remove packages to create a ‘verse’ of packages suiting their general needs. Separate ‘verses’ can also be created.

fastverse packages are jointly attached with library(fastverse), and several functions starting with fastverse_ help manage dependencies, detect namespace conflicts, add/remove packages from the fastverse and update packages.

Core Packages

The fastverse consists of 6 core packages (7 dependencies in total) which provide broad C/C++ based statistical and data manipulation functionality and have carefully managed APIs. These packages are installed and attached along with the fastverse package.

Installation

Currently, there are 2 different versions of the fastverse on CRAN and GitHub. The GitHub version is recommended if you want to have matrixStats consistently preserve attributes of your matrices: it modifies functions in the matrixStats namespace making them preserve attributes consistently (and by default) whenever the fastverse is attached. This version was rejected by CRAN because it requires a call to unlockBinding. The CRAN version takes matrixStats as it is, which means most functions do not preserve attributes such as dimension names in computations.

# Install the CRAN version
install.packages("fastverse")

# Install the GitHub version (Requires Rtools)
remotes::install_github("SebKrantz/fastverse")

Note that the GitHub version is not a development version, development takes place in the ‘development’ branch. matrixStats is slowly evolving towards greater consistency, but it might take more than half a year until dimension names are handled consistently by default - due to the large number of reverse dependencies. Until then CRAN and GitHub versions of the fastverse are released together.

Extending the fastverse

In addition, users have the option (via the fastverse_entend() function) to freely attach extension packages offering more specific functionality. The fastverse can by extended by any R package, either just for the current session or permanently:

Click here to expand

# Loads and attaches the core fastverse packages
library(fastverse)
# -- Attaching packages --------------------------------------- fastverse 0.1.5 --
# v data.table  1.14.0     v collapse    1.6.5 
# v magrittr    2.0.1      v matrixStats 0.59.0
# v kit         0.0.7      v fst         0.9.4

# Permanently extends the core fastverse by certain packages
fastverse_extend(xts, roll, dygraphs, permanent = TRUE)
# -- Attaching extension packages ----------------------------- fastverse 0.1.5 --
# v xts      0.12.1      v dygraphs 1.1.1.6
# v roll     1.1.6
# -- Conflicts ------------------------------------------ fastverse_conflicts() --
# x xts::first() masks data.table::first()
# x xts::last()  masks data.table::last()

# If the fastverse is now loaded in a new session, these packages are added 
fastverse_detach(session = TRUE)
library(fastverse)
# -- Attaching packages --------------------------------------- fastverse 0.1.5 --
# v data.table  1.14.0      v fst         0.9.4  
# v magrittr    2.0.1       v xts         0.12.1 
# v kit         0.0.7       v roll        1.1.6  
# v collapse    1.6.5       v dygraphs    1.1.1.6
# v matrixStats 0.59.0
# -- Conflicts ------------------------------------------ fastverse_conflicts() --
# x xts::first()           masks data.table::first()
# x collapse::is.regular() masks zoo::is.regular()
# x xts::last()            masks data.table::last()

# We can also extend only the fastverse for the session, here adding Rfast2
# and any installed suggested packages for date-time manipulation (see following README section)
fastverse_extend(Rfast2, topics = "DT") 
# -- Attaching extension packages ----------------------------- fastverse 0.1.5 --
# v Rfast2    0.0.9      v clock     0.3.1 
# v lubridate 1.7.10     v fasttime  1.0.2
# -- Conflicts ------------------------------------------ fastverse_conflicts() --
# x lubridate::as.difftime() masks base::as.difftime()
# x clock::as_date()         masks lubridate::as_date()
# x lubridate::date()        masks base::date()
# x lubridate::hour()        masks data.table::hour()
# x lubridate::intersect()   masks base::intersect()
# x lubridate::is.Date()     masks collapse::is.Date()
# x lubridate::isoweek()     masks data.table::isoweek()
# x lubridate::mday()        masks data.table::mday()
# x lubridate::minute()      masks data.table::minute()
# x lubridate::month()       masks data.table::month()
# x lubridate::quarter()     masks data.table::quarter()
# x lubridate::second()      masks data.table::second()
# x lubridate::setdiff()     masks base::setdiff()
# x lubridate::union()       masks base::union()
# x lubridate::wday()        masks data.table::wday()
# x lubridate::week()        masks data.table::week()
# x lubridate::yday()        masks data.table::yday()
# x lubridate::year()        masks data.table::year()

# This shows a situation report of the fastverse, including all dependencies
fastverse_sitrep(recursive = TRUE)
# -- fastverse 0.1.5: Situation Report -------------------------------- R 4.1.0 --
#  * Global config file: TRUE
#  * Project config file: FALSE
# -- Core packages --------------------------------------------------------------- 
#  * data.table    (1.14.0)
#  * magrittr      (2.0.1)
#  * kit           (0.0.7)
#  * collapse      (1.6.5)
#  * matrixStats   (0.59.0 < 0.60.0)
#  * fst           (0.9.4)
#  * xts           (0.12.1)
#  * roll          (1.1.6)
#  * dygraphs      (1.1.1.6)
# -- Extension packages ---------------------------------------------------------- 
#  * Rfast2        (0.0.9)
#  * lubridate     (1.7.10)
#  * clock         (0.3.1 < 0.4.0)
#  * fasttime      (1.0.2)
# -- Dependencies ---------------------------------------------------------------- 
#  * base64enc     (0.1.3)
#  * cpp11         (0.3.1)
#  * digest        (0.6.27)
#  * ellipsis      (0.3.1 < 0.3.2)
#  * generics      (0.1.0)
#  * glue          (1.4.2)
#  * htmltools     (0.5.1.1)
#  * htmlwidgets   (1.5.3)
#  * jsonlite      (1.7.2)
#  * lattice       (0.20.44)
#  * RANN          (2.6.1)
#  * Rcpp          (1.0.7)
#  * RcppArmadillo (0.10.2.1.0 < 0.10.6.0.0)
#  * RcppGSL       (0.3.8 < 0.3.9)
#  * RcppParallel  (5.0.2 < 5.1.4)
#  * RcppZiggurat  (0.1.5 < 0.1.6)
#  * Rfast         (2.0.1 < 2.0.3)
#  * rlang         (0.4.11)
#  * tzdb          (0.1.1 < 0.1.2)
#  * vctrs         (0.3.7 < 0.3.8)
#  * yaml          (2.2.1)
#  * zoo           (1.8.9)

# Resets the fastverse to defaults, removing any permanent modifications
fastverse_reset()

In addition to a global customization, separate fastverse’s can be created for projects by adding a .fastverse config file in the project directory and listing packages there. Only these packages will then be loaded and managed with library(fastverse) in the project.

Suggested Extensions

High-performing packages for different data manipulation and statistical computing topics are suggested below. Each topic has a 2-character topic-id, which can be used to quickly attach all available packages with fastvere_extend(topcis = c(..id's..)), and to install missing packages by adding argument install = TRUE. The majority of these packages provide compiled code and have few dependencies. The total (recursive) dependency count is indicated for each package.


Time Series (TS)

Dates and Times (DT)

Strings (ST)

Statistics and Computing (SC)

Spatial (SP)

Visualization (VI)

Tidyverse-like data manipulation built on data.table (TV)

Data Input-Output and Serialization (IO)


Adding to this list

Feel free to notify me of any other packages you think should be included here. Such packages should be well designed, top-performing, low-dependency, and, with few exceptions, provide own compiled code. Please note that the fastverse focuses on general purpose statistical computing and data manipulation, thus I won’t include fast packages to estimate specific kinds of models here (of which R also has a great many).


  1. collapse functions can also handle irregular time series, but this requires passing an integer time variable to the t argument which has consecutive integer steps for regular parts of the time series and non-consecutive integers for the irregular parts.↩︎