Data Types

This section covers Arcon's data types

Overview

Arcon is a hybrid row and columnar system.

Declaring an Arcon Type

Types inside the Arcon runtime must implement the ArconType trait. To define an ArconType, one can use the Arcon derive macro.

use arcon::prelude::*;

#[arcon::proto]
#[derive(Arcon, Copy, Clone)]
pub struct Event {
    id: u64,
    data: f32,
    timestamp: u64,
}

The arcon::proto attribute macro helps remove some boilerplate annotations required by the prost crate. Note that the Arcon derive macro needs prost to be in scope. Make sure it's listed as a dependency:

[dependencies]
arcon = "0.2"
prost = "0.9"

Enabling Arrow

Data types within Arcon are not required to support Arrow conversion. However, the Arrow derive macro signals that it's an Arrow type to the runtime. Arrow analytics is limited as of now. You can follow the progress here.

use arcon::prelude::*;

#[arcon::proto]
#[derive(Arcon, Arrow, Copy, Clone)]
pub struct ArrowEvent {
    id: u64,
    data: f32,
}