AsyncQueue {crul} | R Documentation |
AsyncQueue
Description
An AsyncQueue client
R6 classes
This is an R6 class from the package R6. Find out more
about R6 at https://r6.r-lib.org/. After creating an instance of an R6
class (e.g., x <- HttpClient$new(url = "https://hb.opencpu.org")
) you can
access values and methods on the object x
.
Super class
crul::AsyncVaried
-> AsyncQueue
Public fields
bucket_size
(integer) number of requests to send at once
sleep
(integer) number of seconds to sleep between each bucket
req_per_min
(integer) requests per minute
Methods
Public methods
Inherited methods
Method print()
print method for AsyncQueue objects
Usage
AsyncQueue$print(x, ...)
Arguments
x
self
...
ignored
Method new()
Create a new AsyncQueue
object
Usage
AsyncQueue$new( ..., .list = list(), bucket_size = 5, sleep = NULL, req_per_min = NULL )
Arguments
..., .list
Any number of objects of class
HttpRequest()
, must supply inputs to one of these parameters, but not bothbucket_size
(integer) number of requests to send at once. default: 5. See Details.
sleep
(integer) seconds to sleep between buckets. default: NULL (not set)
req_per_min
(integer) maximum number of requests per minute. if
NULL
(default), its ignored
Details
Must set either sleep
or req_per_min
. If you set
req_per_min
we calculate a new bucket_size
when $new()
is
called
Returns
A new AsyncQueue
object
Method request()
Execute asynchronous requests
Usage
AsyncQueue$request()
Returns
nothing, responses stored inside object, though will print messages if you choose verbose output
Method responses()
List responses
Usage
AsyncQueue$responses()
Returns
a list of HttpResponse
objects, empty list before
requests made
Method parse()
parse content
Usage
AsyncQueue$parse(encoding = "UTF-8")
Arguments
encoding
(character) the encoding to use in parsing. default:"UTF-8"
Returns
character vector, empty character vector before requests made
Method status_code()
Get HTTP status codes for each response
Usage
AsyncQueue$status_code()
Returns
numeric vector, empty numeric vector before requests made
Method status()
List HTTP status objects
Usage
AsyncQueue$status()
Returns
a list of http_code
objects, empty list before requests made
Method content()
Get raw content for each response
Usage
AsyncQueue$content()
Returns
raw list, empty list before requests made
Method times()
curl request times
Usage
AsyncQueue$times()
Returns
list of named numeric vectors, empty list before requests made
Method clone()
The objects of this class are cloneable with this method.
Usage
AsyncQueue$clone(deep = FALSE)
Arguments
deep
Whether to make a deep clone.
See Also
Other async:
Async
,
AsyncVaried
,
HttpRequest
Examples
## Not run:
# Using sleep (note this works with retry requests)
reqlist <- list(
HttpRequest$new(url = "https://hb.opencpu.org/get")$get(),
HttpRequest$new(url = "https://hb.opencpu.org/post")$post(),
HttpRequest$new(url = "https://hb.opencpu.org/put")$put(),
HttpRequest$new(url = "https://hb.opencpu.org/delete")$delete(),
HttpRequest$new(url = "https://hb.opencpu.org/get?g=5")$get(),
HttpRequest$new(
url = "https://hb.opencpu.org/post")$post(body = list(y = 9)),
HttpRequest$new(
url = "https://hb.opencpu.org/get")$get(query = list(hello = "world")),
HttpRequest$new(url = "https://ropensci.org")$get(),
HttpRequest$new(url = "https://ropensci.org/about")$get(),
HttpRequest$new(url = "https://ropensci.org/packages")$get(),
HttpRequest$new(url = "https://ropensci.org/community")$get(),
HttpRequest$new(url = "https://ropensci.org/blog")$get(),
HttpRequest$new(url = "https://ropensci.org/careers")$get(),
HttpRequest$new(url = "https://hb.opencpu.org/status/404")$retry("get")
)
out <- AsyncQueue$new(.list = reqlist, bucket_size = 5, sleep = 3)
out
out$bucket_size # bucket size
out$requests() # list requests
out$request() # make requests
out$responses() # list responses
# Using requests per minute
if (interactive()) {
x="https://raw.githubusercontent.com/ropensci/roregistry/gh-pages/registry.json"
z <- HttpClient$new(x)$get()
urls <- jsonlite::fromJSON(z$parse("UTF-8"))$packages$url
repos = Filter(length, regmatches(urls, gregexpr("ropensci/[A-Za-z]+", urls)))
repos = unlist(repos)
auth <- list(Authorization = paste("token", Sys.getenv('GITHUB_PAT')))
reqs <- lapply(repos[1:50], function(w) {
HttpRequest$new(paste0("https://api.github.com/repos/", w), headers = auth)$get()
})
out <- AsyncQueue$new(.list = reqs, req_per_min = 30)
out
out$bucket_size
out$requests()
out$request()
out$responses()
}
## End(Not run)