
‘R’-package ‘broadcast’: Broadcasted Array Operations Like ‘NumPy’
‘broadcast’ is an efficient ‘C’/‘C++’ - based ‘R’ package that, as the name suggests, performs “array broadcasting” (similar to broadcasting in the ‘Numpy’ module for ‘Python’).
In the context of operations involving 2 (or more) arrays,
“broadcasting” refers to efficiently recycling array dimensions, without
making copies.
This is considerably faster and more memory-efficient than R’s
regular dimensions replication mechanism.
Key Features (click on the 🔍 to show or hide):
Consider computing the addition of these arrays x and y:
(x <- array(1:15, c(3, 5)))
#> [,1] [,2] [,3] [,4] [,5]
#> [1,] 1 4 7 10 13
#> [2,] 2 5 8 11 14
#> [3,] 3 6 9 12 15
(y <- array(1:5 * 100, c(1, 5)))
#> [,1] [,2] [,3] [,4] [,5]
#> [1,] 100 200 300 400 500
This cannot be done efficiently in base ‘R’; it can be done fast and memory-efficiently with the ‘broadcast’ package:
|
Base ‘R’ |
‘broadcast’ package |
|
|
‘broadcast’ supports a wide range of infix operators, including arithmetic-, relational-, Boolean- string- and bit-wise operators.
Using broadcasting, bind_array() from the ‘broadcast’ package can bind
arrays together in ways that cannot efficiently be done with rbind(),
cbind(), or abind::abind(). Consider, for example, column-binding
these arrays x and y:
(x <- array(1:12, c(3, 4)))
#> [,1] [,2] [,3] [,4]
#> [1,] 1 4 7 10
#> [2,] 2 5 8 11
#> [3,] 3 6 9 12
(y <- array(1:4 * 100, c(1, 4)))
#> [,1] [,2] [,3] [,4]
#> [1,] 100 200 300 400
This cannot be done efficiently in base ‘R’; it can be done fast and memory-efficiently with the ‘broadcast’ package:
|
Base ‘R’ |
‘broadcast’ package |
|
|
bind_array() is also considerably faster and more memory efficient
than abind(). See the
benchmarks.
The idea of broadcasted infix operations and broadcasted array binding
has been generalized to also include bcapply() (a broadcasted
apply-like function), bc_ifelse() (broadcasted version of ifelse()),
bc_strrep() (broadcasted version of strrep()).
Broadcast provides casting functions, that cast subset-groups of an
array to a new dimension, cast nested lists to dimensional lists, and
vice-versa.
These functions are useful for facilitating complex broadcasted
operations, though they also have much merit beside broadcasting.
For example, you cannot broadcast through hierarchies of a list, but you can broadcast along dimensions. So suppose you have the following list:
x <- list(
student1 = list(
homework1 = sample(0:100, 5),
homework2 = sample(0:100, 5),
homework3 = sample(0:100, 5)
),
student2 = list(
homework1 = sample(0:100, 5),
homework2 = sample(0:100, 5),
homework3 = sample(0:100, 5)
),
student3 = list(
homework1 = sample(0:100, 5),
homework2 = sample(0:100, 5),
homework3 = sample(0:100, 5)
)
)
Since all values in the list are numbers, you might want to turn this into a numeric array, to make mathematical computations and analyses on it easier.
This can be done with the ‘broadcast’ package with the following steps.
First, turn the nested list into a shallow (i.e. non-nested),
dimensional list using cast_hier2dim():
x2 <- cast_hier2dim(x, in2out = FALSE, direction.names = 1L)
print(x2)
#> homework1 homework2 homework3
#> student1 integer,5 integer,5 integer,5
#> student2 integer,5 integer,5 integer,5
#> student3 integer,5 integer,5 integer,5
Second, turn the shallow (i.e. non-nested), dimensional list into an
atomic array using cast_shallow2atomic():
x3 <- cast_shallow2atomic(x2, 1L)
print(x3)
#> , , homework1
#>
#> student1 student2 student3
#> [1,] 67 6 73
#> [2,] 38 72 41
#> [3,] 0 78 37
#> [4,] 33 84 19
#> [5,] 86 36 27
#>
#> , , homework2
#>
#> student1 student2 student3
#> [1,] 42 88 19
#> [2,] 13 36 43
#> [3,] 81 33 86
#> [4,] 58 100 69
#> [5,] 50 43 39
#>
#> , , homework3
#>
#> student1 student2 student3
#> [1,] 96 78 43
#> [2,] 84 32 24
#> [3,] 20 83 69
#> [4,] 53 34 38
#> [5,] 73 69 50
‘broadcast’ comes with a few linear algebra functions for statistics.
For example, the sd_lc() function to compute the standard deviation of
a linear combination of variables - regardless of what the distribution
of the variables is.
The Quick-Start Guide can be found here.
Efficiency
Broadcasting as implemented in the ‘broadcast’ package is much faster
and much more memory efficient than using base solutions like
sweep().
The implementations in the ‘broadcast’ package are also about as fast
as - and sometimes even faster than - ‘NumPy’.
Efficient programs use less energy and resources, and is thus better for
the environment.
Benchmarks can be found in the “About” section on the website.
Convenience
Have you ever been bothered by any of the following while programming in ‘R’:
abind::abind() being too slow, or ruining the structure of recursive
arrays?sweep() and outer() functions being too slow or too limiting?data.table::dcast()?If you answered “YES” to any of the above, ‘broadcast’ may be the ‘R’ - package for you.
Minimal Dependencies
Besides linking to ‘Rcpp’, ‘broadcast’ does not depend on, vendor, link to, include, or otherwise use any external libraries. Moreover, no generative AI of any sort was used in developing ‘broadcast’. ‘broadcast’ was essentially made from scratch and can be installed out-of-the-box.
Not using external libraries brings a number of advantages:
See the Other Packages page for more details on the above points regarding the advantages of minimizing dependencies.
Tested
The ‘broadcast’ package is frequently checked using a large suite of unit tests via the tinytest package. These tests have a coverage of over 95%. So the chance of a function from this package breaking completely is relatively low.
‘broadcast’ is still relatively new package, however, so (small) bugs are still very much possible. I encourage users who find bugs to report them promptly to the issues tab on the GitHub page, and I will fix them as soon as time permits.
install.packages("broadcast", type = "source")
‘broadcast’ is now available on CRAN! 🎉
If you have any suggestions or feedback on the package, its
documentation, or even the benchmarks, I encourage you to let me know
(either as an Issue or a
Discussion).
I’m eager to read your input!
The documentation in the ‘broadcast’ website is divided into 3 main parts: