get_object {aws.s3}

Get object

Description

Retrieve an object from an S3 bucket. To check if an object exists, see head_object

Usage

get_object(
  object,
  bucket,
  headers = list(),
  parse_response = FALSE,
  as = "raw",
  ...
)

save_object(
  object,
  bucket,
  file = basename(object),
  headers = list(),
  overwrite = TRUE,
  ...
)

select_object(
  object,
  bucket,
  request_body,
  headers = list(),
  parse_response = FALSE,
  ...
)

s3connection(object, bucket, headers = list(), ...)

Arguments

object

Character string with the object key, or an object of class “s3_object”. In most cases, if object is specified as the latter, bucket can be omitted because the bucket name will be extracted from “Bucket” slot in object.

bucket

Character string with the name of the bucket, or an object of class “s3_bucket”.

headers

List of request headers for the REST call.

parse_response

Passed through to s3HTTP, as this function requires a non-default setting. There is probably no reason to ever change this.

as

Passed through to httr::content.

...

Additional arguments passed to s3HTTP.

file

An R connection, or file name specifying the local file to save the object into.

overwrite

A logical indicating whether to overwrite file. Passed to write_disk. Default is TRUE.

request_body

For select_object, an XML request body as described in the SELECT API documentation.

Details

get_object retrieves an object into memory as a raw vector. This page describes get_object and several wrappers that provide additional useful functionality.

save_object saves an object to a local file without bringing it into memory.

s3connection provides a connection interface to an S3 object.

select_object uses the SELECT API to select part of a CSV or JSON object. This requires constructing and passing a fairly tedious request body, which users will have to construct themselves according to the documentation.

Some users may find the raw vector response format of get_object unfamiliar. The object will also carry attributes, including “content-type”, which may be useful for deciding how to subsequently process the vector. Two common strategies are as follows. For text content types, running charToRaw may be the most useful first step to make the response human-readable. Alternatively, converting the raw vector into a connection using rawConnection may also be useful, as that can often then be passed to parsing functions just like a file connection would be.

Higher-level functions

Value

If file = NULL, a raw object. Otherwise, a character string containing the file name that the object is saved to.

References

API Documentation: GET Object API Documentation: GET Object torrent API Documentation: SELECT Object

See Also

get_bucket, object_exists, head_object, put_object, delete_object

Examples

## Not run: 
  # get an object in memory
  ## create bucket
  b <- put_bucket("myexamplebucket")
  
  ## save a dataset to the bucket
  s3save(mtcars, bucket = b, object = "mtcars")
  obj <- get_bucket(b)
  ## get the object in memory
  x <- get_object(obj[[1]])
  load(rawConnection(x))
  "mtcars" %in% ls()

  # save an object locally
  y <- save_object(obj[[1]], file = object[[1]][["Key"]])
  y %in% dir()

  # return object using 'S3 URI' syntax, with progress bar
  get_object("s3://myexamplebucket/mtcars", show_progress = TRUE)

  # return parts of an object
  ## use 'Range' header to specify bytes
  get_object(object = obj[[1]], headers = list('Range' = 'bytes=1-120'))
 
  # example of streaming connection
  ## setup a bucket and object
  b <- put_bucket("myexamplebucket")
  s3write_using(mtcars, bucket = b, object = "mtcars.csv", FUN = utils::write.csv)
  
  ## setup the connection
  con <- s3connection("mtcars.csv", bucket = b)
  
  ## line-by-line read
  while(length(x <- readLines(con, n = 1L))) {
    print(x)
  }

  ## use data.table::fread without saving object to file
  library(data.table)
  s3write_using(as.data.table(mtcars), bucket = b, object = "mtcars2.csv", FUN = data.table::fwrite)
  fread(get_object("mtcars2.csv", bucket = b, as = "text"))

  ## cleanup
  close(con)
  delete_bucket("myexamplebucket")

## End(Not run)