convertKafkaValueFromJson {analysisPipelines}R Documentation

Connect to a Spark session

Description

Connect to a Spark session

Usage

convertKafkaValueFromJson(streamObj, schema)

Arguments

streamObj

Spark Structured Streaming DataFrame which is returned by the castKafkaStreamAsString function

schema

A structType object created from SparkR specifying the schema of the json data present in the value attribute of the incoming Kafka stream

Details

Takes in a Structured Stream from Kafka created from read.stream(source = 'kafka', ...) and returns a Structured Streaming DataFrame where the key and value from the Kafka stream are cast to string

Value

Spark Structured Streaming DataFrame with the json data in the value attribute of the Kafka stream parsed into a DataFrame format

See Also

Other Spark utilities: castKafkaStreamAsString, sparkRSessionCreateIfNotPresent


[Package analysisPipelines version 1.0.2 Index]