robxp {spiderbar} | R Documentation |
Parse a 'robots.txt' file & create a 'robxp' object
Description
This function takes in a single element character vector and parses it into a 'robxp' object.
Usage
robxp(x)
Arguments
x |
either an atomic character vector containing a complete 'robots.txt“ file _or_ a length >1 character vector that will concatenated into a single string _or_ a 'connection' object that will be passed to [readLines()], the result of which will be concatenated into a single string and parsed and the connection will be closed. |
Value
a classed object holding an external pointer to parsed robots.txt data
Examples
imdb <- paste0(readLines(system.file("extdata", "imdb-robots.txt",
package="spiderbar")), collapse="\n")
rt <- robxp(imdb)
[Package spiderbar version 0.2.5 Index]