dbFetchArrowChunk {DBI}

Fetch the next batch of records from a previously executed query as an Arrow object

Description

Fetch the next chunk of the result set and return it as an Arrow object. The chunk size is implementation-specific. Use dbFetchArrow() to fetch all results.

Usage

dbFetchArrowChunk(res, ...)

Arguments

res

An object inheriting from DBIResultArrow, created by dbSendQueryArrow().

...

Other arguments passed on to methods.

Value

dbFetchArrowChunk() always returns an object coercible to a data.frame with as many rows as records were fetched and as many columns as fields in the result set, even if the result is a single value or has one or zero rows.

The data retrieval flow for Arrow streams

This section gives a complete overview over the flow for the execution of queries that return tabular data as an Arrow stream.

Most of this flow, except repeated calling of dbBindArrow() or dbBind(), is implemented by dbGetQueryArrow(), which should be sufficient unless you have a parameterized query that you want to reuse. This flow requires an active connection established by dbConnect(). See also vignette("dbi-advanced") for a walkthrough.

1. Use dbSendQueryArrow() to create a result set object of class DBIResultArrow.

2. Optionally, bind query parameters with dbBindArrow() or dbBind(). This is required only if the query contains placeholders such as ⁠?⁠ or ⁠$1⁠, depending on the database backend.

3. Use dbFetchArrow() to get a data stream.

4. Repeat the last two steps as necessary.

5. Use dbClearResult() to clean up the result set object. This step is mandatory even if no rows have been fetched or if an error has occurred during the processing. It is good practice to use on.exit() or withr::defer() to ensure that this step is always executed.

Failure modes

An attempt to fetch from a closed result set raises an error.

Specification

Fetching multi-row queries with one or more columns returns the next chunk. The size of the chunk is implementation-specific. The object returned by dbFetchArrowChunk() can also be passed to nanoarrow::as_nanoarrow_array() to create a nanoarrow array object. The chunk size is implementation-specific.

See Also

Close the result set with dbClearResult() as soon as you finish retrieving the records you want.

Other DBIResultArrow generics: DBIResultArrow-class, dbBind(), dbClearResult(), dbFetchArrow(), dbHasCompleted(), dbIsValid()

Other data retrieval generics: dbBind(), dbClearResult(), dbFetch(), dbFetchArrow(), dbGetQuery(), dbGetQueryArrow(), dbHasCompleted(), dbSendQuery(), dbSendQueryArrow()

Examples

con <- dbConnect(RSQLite::SQLite(), ":memory:")

dbWriteTable(con, "mtcars", mtcars)

# Fetch all results
rs <- dbSendQueryArrow(con, "SELECT * FROM mtcars WHERE cyl = 4")
dbHasCompleted(rs)
as.data.frame(dbFetchArrowChunk(rs))
dbHasCompleted(rs)
as.data.frame(dbFetchArrowChunk(rs))
dbClearResult(rs)

dbDisconnect(con)