Stream UDF
Use Aerospike Stream UDFs to filter, transform, and aggregate query results in a distributed fashion. Stream UDFs can process a stream of data by defining a sequence of operations to perform. Stream UDF aggregations run in a distributed fashion to exploit the power of multiple machines.
Stream UDFs perform read-only operations on a collection of records. The Stream UDF system allows a MapReduce style of programming, which is used for common, highly parallel jobs such as word count—each row is accessed and a list of words and their counts are emitted. The top results are calculated in a reduce phase. For simple aggregations where counters are simply incremented in a context instead of continually creating and destroying objects, Aerospike provides optimal implementation.
Operators
Aerospike Stream UDFs support these operators:
filter—Filter data in the stream that satisfies a predicate.map—Transform a piece of data.aggregate—Reduce partitions of data to a single value.reduce—Allow parallel processing of each group of output data.
These operators are considered primitives and can build complex operations (see Developing Stream UDFs).
filter
Filter values in the stream using the predicate p. p tests each value in the stream and returns true if the value should be passed through or false to drop the value.
Example
filter(p: (a: Value) -> Boolean) -> Stream
Parameters
p—The predicate to apply to each value in the stream.
Returns
A stream of filtered values.
map
Transforms a value to a different value using the identify function f.
map(f: (a: Value) -> Value) -> Stream
Parameters
f—The identify function to apply to each value in the stream.
Returns
A stream of the transformed values.
reduce
Reduce the values in the stream to a single value and apply the associative binary operator op to each value. The op return value is the parameter a for subsequent calls to op.
reduce(op: (a: Value, b: Value) -> Value) -> Stream
Parameters
op—The associative binary operation to apply to each value in the stream.
Returns
A stream containing a single value.
aggregate
Takes the current value (parameter x) and the next value in the input stream, and returns the aggregate value (parameter a).
aggregate(x: Value, op: (a: Value, b: Value) -> Value) -> Stream
Parameters
x—The initial (neutral) value passed to the operatorop.op—The identify function to apply to each value in the stream.
Returns
A stream containing a single value.
Examples
See Stream UDF Examples.