R: Download data from the DWD CDC FTP Server

dataDWD {rdwd}

R Documentation

Download data from the DWD CDC FTP Server

Description

Get climate data from the German Weather Service (DWD) FTP-server. The desired dataset is downloaded into dir. If read=TRUE, it is also read and processed.
dataDWD handles vectors of URLs, displays progress bars (if the package pbapply is available) and by default does not re-download data already in dir (but see argument force to update files).
To solve "errors in download.file: cannot open URL", see https://bookdown.org/brry/rdwd/fileindex.html.

Usage

dataDWD(
  url,
  base = dwdbase,
  joinbf = FALSE,
  dir = locdir(),
  force = FALSE,
  overwrite = !isFALSE(force),
  read = TRUE,
  dbin = TRUE,
  method = getOption("download.file.method"),
  dfargs = NULL,
  sleep = 0,
  progbar = !quiet,
  browse = FALSE,
  ntrunc = 2,
  file = NULL,
  quiet = rdwdquiet(),
  ...
)

Arguments

`url`	Char (vector): complete file URL(s) (including base and filename.zip) as returned by `selectDWD()`. Can be a vector with several FTP URLs.
`base`	Single char: base URL that will be removed from output file names. DEFAULT: `dwdbase`
`joinbf`	Logical: paste `base` and `⁠file url⁠` together? Needed mostly for data at `gridbase`. DEFAULT: FALSE (selectDWD returns complete URLs already)
`dir`	Char: Writeable directory name where to save the downloaded file. Created if not existent. DEFAULT: `locdir()`
`force`	Logical (vector): always download, even if the file already exists in `dir`? Use NA to force re-downloading files older than 24 hours. Use a numerical value to force after that amount of hours. Use something like `c(Inf, 24)` or `force=c(24*365, 6)`, for per="hr". Note: if `force` is not FALSE, the `overwrite` default is TRUE. DEFAULT: FALSE
`overwrite`	Logical (vector): if force=TRUE, overwrite the existing file rather than append "_1"/"_2" etc to the filename? DEFAULT: `!isFALSE(force)`, i.e. true when `force` is specified.
`read`	Logical: read the file(s) with `readDWD()`? If FALSE, only download is performed and the filename(s) returned. DEFAULT: TRUE
`dbin`	Logical: Download binary file, i.e. add `mode="wb"` to the `download.file()` call? See Website for details. DEFAULT: TRUE
`method`	download.file `method`. Introduced in version 1.5.25 (2022-05-12) as triggered by https://github.com/brry/rdwd/issues/34. DEFAULT: `getOption("download.file.method")`
`dfargs`	Named list of additional arguments passed to `download.file()` Note that mode="wb" is already passed if `dbin=TRUE`
`sleep`	Number. If not 0, a random number of seconds between 0 and `sleep` is passed to `Sys.sleep()` after each download to avoid getting kicked off the FTP-Server, see note in `indexFTP()`. DEFAULT: 0
`progbar`	Logical: present a progress bar with estimated remaining time? If missing and length(file)==1, progbar is internally set to FALSE. Only works if the R package `pbapply` is available. DEFAULT: TRUE (!quiet)
`browse`	Logical: open repository via `browseURL()` and return URL folder path? If TRUE, no data is downloaded. If file has several values, only unique folders will be opened. DEFAULT: FALSE
`ntrunc`	Single integer: number of filenames printed in messages before they get truncated with message "(and xx more)". DEFAULT: 2
`file`	Deprecated since rdwd version 1.3.34, 2020-07-28.
`quiet`	Logical: suppress message about directory / filenames? DEFAULT: FALSE through `rdwdquiet()`
`...`	Further arguments passed to `readDWD()`, like `fread`, `varnames`, `hr`, etc.

Value

Presuming downloading and processing were successful: if read=TRUE, the desired dataset (as returned by readDWD()), otherwise the filename as saved on disc (may have "_n" appended in name, see newFilename()).
If length(file)>1, the output is a list of outputs / vector of filenames.
The output is always invisible.

Author(s)

Berry Boessenkool, berry-b@gmx.de, Jun-Oct 2016

Examples

## Not run:  ## requires internet connection
# find FTP files for a given station name and file path:
link <- selectDWD("Fuerstenzell", res="hourly", var="wind", per="recent")
# download file:
fname <- dataDWD(link, dir=locdir(), read=FALSE) ; fname
# dir="DWDdata" is the default directory to store files
# unless force=TRUE, already obtained files will not be downloaded again

# read and plot file:
wind <- readDWD(fname, varnames=TRUE) ; head(wind)
metafiles <- readMeta(fname)          ; str(metafiles, max.level=1)
column_names <- readVars(fname)       ; head(column_names)

plot(wind$MESS_DATUM, wind$F, main="DWD hourly wind Fuerstenzell", col="blue",
     xaxt="n", las=1, type="l", xlab="Date", ylab="Hourly Wind speed  [m/s]")
berryFunctions::monthAxis(1)


# current and historical files, keep historical in the overlap time period:
link <- selectDWD("Potsdam", res="daily", var="kl", per="hr"); link
potsdam <- dataDWD(link, dir=locdir(), hr=4)
plot(TMK~MESS_DATUM, data=tail(potsdam,1500), type="l")


# With many files (>>50), use sleep to avoid getting kicked off the FTP server
#links <- selectDWD(res="daily", var="solar")
#sol <- dataDWD(links, sleep=20) # random waiting time after download (0 to 20 secs)

# Real life examples can be found in the use cases section of the vignette:
# browseURL("https://bookdown.org/brry/rdwd")

## End(Not run)

[Package rdwd version 1.8.0 Index]