AutoGrid {DelayedArray} | R Documentation |
Create automatic grids to use for block processing of array-like objects
Description
We provide various utility functions to create grids that can be used for block processing of array-like objects:
-
defaultAutoGrid()
is the default automatic grid maker. It creates a grid that is suitable for block processing of the array-like object passed to it. -
rowAutoGrid()
andcolAutoGrid()
are more specialized automatic grid makers, for the 2-dimensional case. They can be used to create a grid where the blocks are made of full rows or full columns, respectively. -
defaultSinkAutoGrid()
is a specialized version ofdefaultAutoGrid()
for creating a grid that is suitable for writing to a RealizationSink derivative while walking on it.
Usage
defaultAutoGrid(x, block.length=NULL, chunk.grid=NULL, block.shape=NULL)
## Two specialized "automatic grid makers" for the 2-dimensional case:
rowAutoGrid(x, nrow=NULL, block.length=NULL)
colAutoGrid(x, ncol=NULL, block.length=NULL)
## Replace default automatic grid maker with user-defined one:
getAutoGridMaker()
setAutoGridMaker(GRIDMAKER="defaultAutoGrid")
## A specialized version of defaultAutoGrid() to create an automatic
## grid on a RealizationSink derivative:
defaultSinkAutoGrid(sink, block.length=NULL, chunk.grid=NULL)
Arguments
x |
An array-like or matrix-like object for A matrix-like object for |
block.length |
The length of the blocks i.e. the number of array elements per block.
By default the automatic block length (returned by
|
chunk.grid |
The grid of physical chunks.
By default |
block.shape |
A string specifying the shape of the blocks.
See |
nrow |
The number of rows of the blocks. The bottommost blocks might have less. See examples below. |
ncol |
The number of columns of the blocks. The rightmost blocks might have less. See examples below. |
GRIDMAKER |
The function to use as automatic grid maker, that is, the
function that will be used by
The automatic grid maker is set to |
sink |
A RealizationSink derivative. |
Details
By default, primary block processing functions blockApply()
and blockReduce()
use the grid returned by
defaultAutoGrid(x)
to walk on the blocks of array-like
object x
. This can be changed with setAutoGridMaker()
.
By default sinkApply()
uses the grid returned by
defaultSinkAutoGrid(sink)
to walk on the viewports of
RealizationSink derivative sink
and write to them.
Value
defaultAutoGrid
: An ArrayGrid object on reference array
x
. The grid elements define the blocks that will be used to
process x
by block. The grid is optimal in the sense that:
It's compatible with the grid of physical chunks a.k.a. chunk grid. This means that, when the chunk grid is known (i.e. when
chunkGrid(x)
is not NULL orchunk.grid
is supplied), every block in the grid contains one or more full chunks. In other words, chunks never cross block boundaries.Its resolution is such that the blocks have a length that is as close as possibe to (but does not exceed)
block.length
. An exception is made when some chunks already have a length that is >=block.length
, in which case the returned grid is the same as the chunk grid.
Note that the returned grid is regular (i.e. is a RegularArrayGrid object) unless the chunk grid is not regular (i.e. is an ArbitraryArrayGrid object).
rowAutoGrid
: A RegularArrayGrid object on reference array
x
where the grid elements define blocks made of full rows of x
.
colAutoGrid
: A RegularArrayGrid object on reference array
x
where the grid elements define blocks made of full columns
of x
.
defaultSinkAutoGrid
: Like defaultAutoGrid
except
that defaultSinkAutoGrid
always returns a grid with a
"first-dim-grows-first" shape (note that, unlike the former, the
latter has no block.shape
argument).
The advantage of using a grid with a "first-dim-grows-first" shape in
the context of writing to the viewports of a RealizationSink
derivative is that such a grid is guaranteed to work with "linear write
only" realization backends. See important notes about "Cross realization
backend compatibility" in ?write_block
for more information.
See Also
-
setAutoBlockSize
andsetAutoBlockShape
to control the geometry of automatic blocks. -
blockApply
and family for convenient block processing of an array-like object. -
ArrayGrid for the formal representation of grids and viewports.
The
makeCappedVolumeBox
utility to make capped volume boxes.-
read_block
andwrite_block
.
Examples
## ---------------------------------------------------------------------
## A VERSION OF sum() THAT USES BLOCK PROCESSING
## ---------------------------------------------------------------------
block_sum <- function(a, grid) {
sums <- lapply(grid, function(viewport) sum(read_block(a, viewport)))
sum(unlist(sums))
}
## On an ordinary matrix:
m <- matrix(runif(600), ncol=12)
m_grid <- defaultAutoGrid(m, block.length=120)
sum1 <- block_sum(m, m_grid)
sum1
## On a DelayedArray object:
library(HDF5Array)
M <- as(m, "HDF5Array")
sum2 <- block_sum(M, m_grid)
sum2
sum3 <- block_sum(M, colAutoGrid(M, block.length=120))
sum3
sum4 <- block_sum(M, rowAutoGrid(M, block.length=80))
sum4
## Sanity checks:
sum0 <- sum(m)
stopifnot(identical(sum1, sum0))
stopifnot(identical(sum2, sum0))
stopifnot(identical(sum3, sum0))
stopifnot(identical(sum4, sum0))
## ---------------------------------------------------------------------
## defaultAutoGrid()
## ---------------------------------------------------------------------
grid <- defaultAutoGrid(m, block.length=120)
grid
as.list(grid) # turn the grid into a list of ArrayViewport objects
table(lengths(grid))
stopifnot(maxlength(grid) <= 120)
grid <- defaultAutoGrid(m, block.length=120,
block.shape="first-dim-grows-first")
grid
table(lengths(grid))
stopifnot(maxlength(grid) <= 120)
grid <- defaultAutoGrid(m, block.length=120,
block.shape="last-dim-grows-first")
grid
table(lengths(grid))
stopifnot(maxlength(grid) <= 120)
defaultAutoGrid(m, block.length=100)
defaultAutoGrid(m, block.length=75)
defaultAutoGrid(m, block.length=25)
defaultAutoGrid(m, block.length=20)
defaultAutoGrid(m, block.length=10)
## ---------------------------------------------------------------------
## rowAutoGrid() AND colAutoGrid()
## ---------------------------------------------------------------------
rowAutoGrid(m, nrow=10) # 5 blocks of 10 rows each
rowAutoGrid(m, nrow=15) # 3 blocks of 15 rows each plus 1 block of 5 rows
colAutoGrid(m, ncol=5) # 2 blocks of 5 cols each plus 1 block of 2 cols
## See '?write_block' for an advanced example of user-implemented
## block processing using colAutoGrid() and a realization sink.
## ---------------------------------------------------------------------
## REPLACE DEFAULT AUTOMATIC GRID MAKER WITH USER-DEFINED ONE
## ---------------------------------------------------------------------
getAutoGridMaker()
setAutoGridMaker(function(x) colAutoGrid(x, ncol=5))
getAutoGridMaker()
blockApply(m, function(block) currentViewport())
## Reset automatic grid maker to factory settings:
setAutoGridMaker()