sdf_utils {sparklyr.flint}

Utility functions for importing a Spark data frame into a TimeSeriesRDD

Description

These functions provide an interface for specifying how a Spark data frame should be imported into a TimeSeriesRDD (e.g., which column represents time, whether rows are already ordered by time, and time unit being used, etc)

Arguments

sc

Spark connection

is_sorted

Whether the rows being imported are already sorted by time

time_unit

Time unit of the time column (must be one of the following values: "NANOSECONDS", "MICROSECONDS", "MILLISECONDS", "SECONDS", "MINUTES", "HOURS", "DAYS"

time_column

Name of the time column