to_sdf {sparklyr.flint}

Export data from TimeSeriesRDD to a Spark dataframe

Description

Construct a Spark dataframe containing time series data from a TimeSeriesRDD

Usage

to_sdf(ts_rdd)

toSDF(ts_rdd)

Arguments

ts_rdd

A TimeSeriesRDD object

Value

A Spark dataframe containing time series data exported from 'ts_rdd'

See Also

Other Spark dataframe utility functions: collect.ts_rdd(), from_rdd(), from_sdf(), spark_connection.ts_rdd(), spark_dataframe.ts_rdd(), spark_jobj.ts_rdd(), ts_rdd_builder()

Other Spark dataframe utility functions: collect.ts_rdd(), from_rdd(), from_sdf(), spark_connection.ts_rdd(), spark_dataframe.ts_rdd(), spark_jobj.ts_rdd(), ts_rdd_builder()

Examples

library(sparklyr)
library(sparklyr.flint)

sc <- try_spark_connect(master = "local")

if (!is.null(sc)) {
  sdf <- copy_to(sc, tibble::tibble(t = seq(10), v = seq(10)))
  ts <- from_sdf(sdf, is_sorted = TRUE, time_unit = "SECONDS", time_column = "t")
  ts_avg <- summarize_avg(ts, column = "v", window = in_past("3s"))
  # now export the average values from `ts_avg` back to a Spark dataframe
  # named `sdf_avg`
  sdf_avg <- ts_avg %>% to_sdf()
} else {
  message("Unable to establish a Spark connection!")
}