Input

Decode

Fetch and decode images

Fetches images from URL inputs, decodes them, and outputs images with the given pixel format.

Debug only

Decode Options
type
Image pixel type

Files

List files from AWS S3 or URLs

Generates a list of file inputs. File lists can include publicly accessible URLS, or AWS S3 files or folders. Click + on the configuration screen to add a new URL or folder and then configure:

Debug only

Files Options
AWS S3
Select S3 from the dropdown list on the left side of the input box, and then click the ... button to browse and select an S3 file or folder.
URL
Select string from the dropdown list on the left side of the input box and then enter the URL of the file. Supported URLs include direct file downloads with HTTP and YouTube URLs of the form https://www.youtube.com/watch?v=ID . Indirect downloads are not supported.

OVCam

Captures images from Omnivision image sensor

Captures YUV422 images from the on-board Omnvision image sensor.

Targets: Monocle

OVCam Options
MODE
sensor mode
Programmability
enable programmability
OVCam Programmability

The optional programmability API provides the following run-time control via the system's API interface.

0x00 (R/1b): Status
current status: bit 0 - continuous capture on/off, bit 1 - xclk on/off
0x03 (W): Capture
trigger a single frame capture
0x04 (W): Stop
end continuous frame capture
0x05 (W): Start
begin continuous frame capture
0x08 (W): Disable XCLK
turn off camera xclk output
0x09 (W): Enable XCLK
turn off camera xclk output

VideoDecode

Fetch and decode videos

Fetches videos from URL inputs, decodes them, and outputs the video frames as a sequence of images with the given pixel format.

Debug only

VideoDecode Options
type
Frame image pixel type

Output

OLED

OLED display

Targets: Monocle

Output images to OLED display. Display devices require constant refresh at a certain frequency, so the input must be attached to a frame buffer pipeline.

Console

Send input messages to console pane

Outputs all input messages in JSON format to the console pane. Values should be converted to JSON via JsonOut before sending to Console. Optionally, outputs only a specific field of the input message.

Debug only

Console Options
data
Enter a field (e.g. payload ) to extract from input messages (leave blank to output full message) or select J: to enter a Jsonata expression.

CsvOut

Encode the input stream in CSV format

Collects input data as rows and outputs the rows encoded in CSV format. Can output one file per video or one file per image.

Debug only

CsvOut Options
stream
Group into one file per image or video
header
Optional header to include with each CSV output
extra
Extra columns prepended to the CSV row. Each extra column is the result of a Jsonata expression evaluted on the input message . The input data is in the payload field of the message and there is additional information in the meta field of the message such as source , src_fps , fps , frame , element .

Encode

Encode images into the configured image file format

Input images are encoded in the given image file format and an internal URL to the encoded file is output.

Debug only

Encode Options
format
The image file format

JsonOut

Output input stream values as Json

Converts input stream values to Json.

Debug only

JsonOut Options
frames
Include frames in output stream

Preview

Display video/images in the preview pane

Displays output from ImageOut or VideoOut operators in the preview pane in TEST MODE ONLY by default.

Debug only

Preview Options
test only
When checked, only preview inputs for Test Run

VideoEncode

Encode videos into a video file format

Debug only

Input frame images are encoded into MP4 video file format and an internal URL to the encoded video is output.

Filters

Blur

Apply a blur kernel to the image.

Use an averaging kernel of size N to blur the input image.

Targets: Monocle

Blur Options
N
kernel size

ChScale

Scale pixels by a constant.

Multiple each input channel by k. In CLAMP mode, out-of-range values are mapped to the closest in-range value and in TRUNC mode, the multiplication is allowed to overflow.

Targets: Monocle

ChScale Options
mode
mapping mode
k
amount to scale input

Corners

Apply Harris corner detection.

Targets: Monocle

Applies a 3x3 Harris filter to the image to detect corners. The input image must be a single-channel 8-bit image and the output is also a single-channel 8-bit image where the pixel values (0-255) represent the corner response measure. The reponse measure is interpreted as follows. Corner pixels have values close to 255, non-corner edges have values close to 0 and flat ranges will have mid-range values.

FPGA This is a high-resource operation that does not fit on smaller devices like the ICE40.

Dilate

Apply dilate filter to a binary image.

Applies a maximum kernel of size N to the input image. This dialates the edges of the image. Useful to restore original size after erode.

Targets: Monocle

Dilate Options
N
kernel size

Erode

Apply erode filter to a binary image.

Applies a minimum kernel of size N to the input image. This erodes the edges and eliminates patches smaller than the kernel size.

Targets: Monocle

Erode Options
N
kernel size

FastCorners

Apply FAST corner detection.

Applies a 3x3 FAST corner filter to the image to detect corners. The input image must be a single-channel 8-bit image and the output is also a single-channel 8-bit image where the pixel values are 255 where a corner is detect or 0 otherwise. The filter works by subtracting the center value from each border pixel and counting how many have a difference above the specified threshold. If there are 5 or more than such pixels, it's classified as a corner.

Targets: Monocle

FastCorners Options
threshold
contrast threshold

GaussianBlur

Apply 3x3 Gaussian blur.

Targets: Monocle

Applies a 3x3 Gaussian averaging kernel to the image.

Grayscale

Convert colors to gray values.

Targets: Monocle

Convert the input image pixels to gray color with the same pixel type, i.e. if the input is Rgb the output is still Rgb. See ToGray to convert images to a grayscale format.

Hue

Extract hue from HSV image.

Extracts and transforms the hue value from HSV pixels to produce a single channel Gray image of hue values. Hue values are in the range 0-180. The hue value can be optionally shifted counter clockwise to move the reds away from the 0/180 discontinuity. For example with shift = 90, the reds will be centered around 90. A minimum saturation can be specified as well to eliminiate unreliable hue values (hue is unstable for low saturation levels). The hue value is mapped to 255 if the saturation is below the minSat value.

Targets: Monocle

Hue Options
minSat
minimum saturation value
minVal
minimum brightness value
shift
counter clockwise hue shift

InRange

Convert to binary image by color range test.

Convert each input image to a binary image, mapping pixels within the specified color range to the given match value (or 255 if match not given) and everything else to 0.

Targets: Monocle

InRange Options
low
minimum pixel value
high
minimum pixel value
match
output value of pixels in range

Laplacian

Apply Laplacian edge detection.

Targets: Monocle

Applies a 3x3 Laplacian kernel to the image for edge detection.

Sharpen

Apply sharpen filter to image.

Applies a 3x3 sharpen kernel to the image

Targets: Monocle

Sharpen Options
amount
magnitude of sharpening

Sobel

Apply Sobel edge detection.

Targets: Monocle

Applies a 3x3 Sobel kernel to the image for edge detection.

Threshold

Convert image to binary by threshold test.

Replaces pixels in the input image by the given replacement value if the input pixel is greater than the given threshold value or zero otherwise.

Targets: Monocle

Threshold Options
threshold
the threshold value
replacement
an optional replacement value to use instead of the maximum pixel value

Image

CropTo

Crop an image to a given size.

Outputs a rectangular region of the input image of the given size from the center or any of the four corners. The hpos/vpos arguments control the cropping position where -1 means left/top, 0 means center, 1 means right/bottom.

Targets: Monocle

CropTo Options
width
crop width
height
crop height
hpos
horizontal crop position
vpos
vertical crop position

Downsize

Downsize input images to a fixed size.

Scales input images to the specified size using a nearest-pixel algorithm. NOTE: When video image frames are resized, the VideoSize operator must also be used to update the video's meta data to match the frame size.

Targets: Monocle

Downsize Options
width
output image width
height
output image height

Invert

Invert image.

Targets: Monocle

Invert each pixel of the input images to produce output images. For a grayscale image or mask, this means map each pixel value x to 255-x.

MeanReduce

Reduce image size via mean kernel.

Reduce image size by a factor of N replacing windows of NxN with the mean value. Image width and height must be evenly divisible by N.

Targets: Monocle

MeanReduce Options
N
reduce factor

PadTo

Pad an image to a given size.

Outputs an image of the given size with the input image inserted at the center or any of the four corners. The hpos/vpos arguments control the cropping position where -1 means left/top, 0 means center, 1 means right/bottom.

Targets: Monocle

PadTo Options
width
crop width
height
crop height
hpos
horizontal position
vpos
vertical position

SkipReduce

Reduce image size by skipping.

Reduce image size by a factor of N replacing windows of NxN with the lower-right corner value. Image width and height must be evenly divisible by N.

Targets: Monocle

SkipReduce Options
N
reduce factor

Conversions

Debayer1

Demosaic Bayer image (low-resource)

A low-resource demosaic or debayer operation that outputs either a gray or rgb image from the Debayer image. The output is half resolution.

Targets: Monocle

Debayer1 Options
U
output image pixel type

Debayer2

Demosaic Bayer image.

Converts the Bayer imput image to a high-quality color image based on 5x5 filter templates.

Targets: Monocle

Debayer2 Options
U
output image pixel type

ToGray

Convert image to grayscale.

Convert the input image to a grayscale image with the given grayscale pixel type.

Targets: Monocle

ToGray Options
U
output image pixel type

ToHsv

Convert image to HSV.

Convert the input image to the Hue/Saturation/Value colorspace.

Targets: Monocle

ToHsv Options
U
output image pixel type

ToRgb

Convert image to RGB.

Convert the input image to an RGB image with the given RGB pixel type.

Targets: Monocle

ToRgb Options
U
output image pixel type

ToRgba

Convert image to RGBA.

Convert the input image to an RGBA image with the given RGBA pixel type.

Targets: Monocle

ToRgba Options
U
output image pixel type

ToYuv

Convert image to YUV.

Convert the input image to the YUV colorspace.

Targets: Monocle

ToYuv Options
U
output image pixel type

Audio

Energy

Computes RMS energy.

Compute the energy of the incoming audio signal over a fixed sized window. The engery over the fixed window is the root mean-squared (RMS) value of the window samples. The output values are 16-bit unsigned integers. Windows are formed every S samples using the N most recent samples. If S is less than N, then the spectrum is computed for overlapping windows with an overlap of N-S samples, otherwise the spectrum is computed for non-overlapping, consecutive rolling windows.

Targets: Monocle

Energy Options
N
window size
S
window stride

Spectrum

Outputs the audio spectrum.

Computes the spectrum of the incoming audio signal over a fixed sized window with the Discrete Fourier Transform. The spectrum represents the frequency components of the signal. The N/2+1 frequency components are output for each window. Each component is an approximate 16-bit unsigned value of the magnitude of the corresponding frequency bin. Windows are formed every S samples using the N most recent samples. If S is less than N, then the spectrum is computed for overlapping windows with an overlap of N-S samples, otherwise the spectrum is computed for non-overlapping, consecutive rolling windows.

Targets: Monocle

Spectrum Options
N
window size
S
window stride (<= window size)

Window2D

Output 2D sliding window.

Collect scalar data into a 2D sliding window. This can be used with Spectrum to construt a Spectrogram. Every WW consecutive elements of input form a new row. Each time a row is completed, the first row of the window is dropped, the new row is appended at the bottom, and the full window is output.

Targets: Monocle

Window2D Options
WW
window width
WH
window height

Data

Element

Project single element from vector.

Outputs the element at the given index from input vectors.

Targets: Monocle

Element Options
idx
element to project (from 0)

Scale

Scale the input by a constant.

For each input x , output k * x . In CLAMP mode, out-of-range values are mapped to the closest in-range value and in TRUNC mode, the multiplication is allowed to overflow.

Targets: Monocle

Scale Options
mode
mapping mode
k
amount to scale input

ScaleType

Map input to another data type.

Maps scalar input values to a smaller (or larger) data type. The input value is divided (or multiplied) by the appropriate value to uniformly scale numbers in the input type's range to the output type's range.

Targets: Monocle

ScaleType Options
U
output data type

ToType

Change input to another data type.

Changes input values to a smaller (or larger) data type. When the output type is a smaller type, the mode parameter determine how to map input values that are out of range. In CLAMP mode, out-of-range values are mapped to the closest in-range value and in TRUNC mode, the upper bits of the input value are ignored. The mode value is ignored if the output type is larger than the input type.

Targets: Monocle

ToType Options
U
output data type
mode
mapping mode (see description)

Frame Buffer

FBBackground

Solid background source frame buffer

Acts as a frame buffer containing a solid background image of a given size. This can be used with frame buffer overlays to create a display pipeline.

Targets: Monocle

FBBackground Options
T
output pixel type
width
image width
height
image height
color
color of background in hex (e.g. ff7f32)

FBGraphics

2D vector graphics overlay

Draws 2D graphic objects (lines, polygons, etc.) over the incoming image to produce an output image in real time. The 2D graphics are described by a custom format, RVGR, which is stored in a memory buffer. The programmability API is used to load the memory buffer with the graphics description. RVGR is a compact format that can be updated frequently engough at typical transfer speeds to perform animation.

The buffer size limits the number of primitives that can be displayed together. For flicker-free animation, the buffer can be used as a double buffer (loading the next graphics, while still displaying the current graphics), but limits the maximum size to half the size of the buffer.

Colors are chosen from cusomizable palette (updated vai the programmability API). The default RGB color palette is:

 0: 000000 - black
 1: ad2323 - red
 2: 1d6914 - green
 3: 2a4bd7 - blue
 4: 29d0d0 - cyan
 5: 8126c0 - magenta
 6: ffee33 - yellow
 7: ffffff - white
 8: 1c1c1c - gray 1
 9: 383838 - gray 2
10: 555555 - gray 3
11: 717171 - gray 4
12: 8d8d8d - gray 5
13: aaaaaa - gray 6
14: c6c6c6 - gray 7
15: e2e2e2 - gray 8

Targets: Monocle

FBGraphics Options
buffer size
graphics description buffer size in bytes
FBGraphics Programmability

The programmability API provides the following run-time control via the system's API interface.

0x00 (R/1b): Status
reserved
0x01 (W/2b): Offset
active display offset in buffer used if double buffering
0x02 (W): 128-Color Palette
write RGB values to palette at offset given in first 2 bytes (big endian)
0x03 (W): Buffer
write data to buffer at offset given in first 2 bytes (big endian)

FBOut

Frame buffer output

Reads and outputs images from the attached frame buffer. In Continuous mode, this node can be used to create a display pipeline. A programmability API is available to control output when this node is connected directly to a display node.

Targets: Monocle

FBOut Options
MODE
output mode
U
output pixel type
FBOut Programmability

The follow API is available only when connected directly with a display.

0x00 (R/1b): Status
current status: bit 0 - output on/off, bit 1 - replay active
0x01 (W): Stop
Stop image output
0x02 (W): Play
Start live image (latest frame) output
0x03 (W): Pause
Continue outputing current frame until resumed
0x04 (W): Replay
Output all stored frames from oldest to newest
0x08 (W/1b): Playback speed
During replay operation, each frame is output this number of times. This value and the display refresh rate determine the playback speed.

FBReadout

Frame buffer read API

Provides a programmability API for reading images out of the attached frame buffer. The block size must go evenly into the total image size in pixels.

Targets: Monocle

FBReadout Options
block size
block size in pixels
FBReadout Programmability

This API provides a fixed-size readout of the most recent image in the source frame buffer. The readout is initiated by writing to the Readout register. That triggers the read stream which can be read from the Image stream register. Once triggered, the Status regsiter can be polled to see when a block is available to read. Once available, the block can be read by reading the block size * pixel size bytes from the Image stream. When the entire image is read the active readout flag will be false.

0x00 (R/1b): Status
current status: bit 0 - active readout flag, bit 1 - buffer available flag
0x04 (W): Start
Begins readout of image
0x05 (R/stream): Image
Stream of pixel data

FBText

Text graphics overlay

Draws text over any input image to product an output image in real time. The location, color and text to display are stored in a memory buffer. The programmability API is used to load the memory buffer. The buffer format is one or more text descriptors followed by the end-of-buffer marker 0xff,0xff,0xff. The text descripter consists of a 5-byte header followed by the character IDs (the ASCII value - 32). The header consists of these 5 bytes:

HEADER:
byte 0: Bit[7:0] - x location (high)
byte 1: Bit[7:4] - x location (low)
        Bit[3:0] - y location (high)
byte 2: Bit[7:0] - y location (low)
byte 3: Bit[3:0] - palette color
byte 4: Bit[7:0] - text length

Colors are chosen from cusomizable palette (updated vai the programmability API). The default RGB color palette is:

 0: 000000 - black
 1: ad2323 - red
 2: 1d6914 - green
 3: 2a4bd7 - blue
 4: 29d0d0 - cyan
 5: 8126c0 - magenta
 6: ffee33 - yellow
 7: ffffff - white
 8: 1c1c1c - gray 1
 9: 383838 - gray 2
10: 555555 - gray 3
11: 717171 - gray 4
12: 8d8d8d - gray 5
13: aaaaaa - gray 6
14: c6c6c6 - gray 7
15: e2e2e2 - gray 8

Targets: Monocle

FBText Options
buffer size
text description buffer size in bytes
FBText Programmability

The programmability API provides the following run-time control via the system's API interface.

0x00 (R/1b): Status
reserved
0x01 (W/2b): Offset
active display offset in buffer used if double buffering
0x02 (W): 128-Color Palette
write RGB values to palette at offset given in first 2 bytes (big endian)
0x03 (W): Buffer
write data to buffer at offset given in first 2 bytes (big endian)

FrameBuf

Frame buffer

Buffers one or more images for slow or multi-pass downstream consumers.

Targets: Monocle

FrameBuf Options
N
total number of frames
Memory
memory type for frame buffer
Programmability
enable programmability
FrameBuf Programmability

The optional programmability API provides the following run-time control via the system's API interface.

0x00 (R/1b): Status
current status: bit 2 - full, bit 3 - empty

SpiApi

SPI API Port

SPI slave port exposing the progammability API to an external SPI master.

Targets: Monocle

SpiApi Options
mode
SPI mode
sclk pin
sclk external pin
mosi pin
mosi external pin
miso pin
miso external pin
ssn pin
optional ssn external pin

CNN

CnnBias

Add a bias to a multi-channel fixed-point image.

Targets: Monocle

CnnBias Options
U
bias type
biases
bias values

CnnBiasRelu

Add a bias to a multi-channel fixed-point image.

Targets: Monocle

CnnBiasRelu Options
V
output element type
U
bias type
biases
bias values
bS
bias shift left
rS
result shift right

CnnConv

Apply multiple parallel kernel convolutions.

Targets: Monocle

CnnConv Options
V
output element type
U
weight type
N
kernel size
O
number of output channels
coeff
weights

CnnConvGr

Apply multiple parallel kernel convolutions to signal channel.

Targets: Monocle

CnnConvGr Options
V
output element type
U
weight type
N
kernel size
O
number of outputs
coeff
weights

CnnConvS

Apply multiple parallel kernel convolutions to signal channel.

Targets: Monocle

CnnConvS Options
V
output element type
U
weight type
N
kernel size
O
number of outputs
coeff
weights

CnnDense

Perform a full-image convolution.

Targets: Monocle

CnnDense Options
V
output element type
U
weight type
IH
input height
IW
input width
O
number of output channels
coeff
weights

CnnInputGr

Convert a grayscale image to a fixed-point image.

Targets: Monocle

CnnMaxPool

Apply max pool reduction to image.

Targets: Monocle

CnnMaxPool Options
N
pool width
M
pool height
coeff
weights

CnnReduce

Reduce a fixed-point image to lower precision fixed point.

Targets: Monocle

CnnReduce Options
U
output element type
S
result shift right

CnnRelu

Apply relu operation to image.

Targets: Monocle

NnBias

Bias a fixed-point input vector.

Targets: Monocle

NnBias Options
U
bias type
biases
bias values
S
bias shift left

NnBiasRelu

Bias a fixed-point input vector.

Targets: Monocle

NnBiasRelu Options
V
output element type
U
bias type
biases
bias values
bS
bias shift left
rS
result shift right

NnDense

Perform the dot product of the input vector withh a constant vector.

Targets: Monocle

NnDense Options
V
output element type
U
weight type
O
number of output channels
coeff
weights

NnReduce

Reduce a fixed-point vector to a lower precision.

Targets: Monocle

NnReduce Options
U
output element type
S
result shift right

NnRelu

Map elements of the input vector to positive values.

Targets: Monocle

TorteCnn

CNN inference with torte model.

Torte is a low-resource inference engine for quantized CNNs. The workflow is based on Tensorflow and a utility is available to quantize and conver the tensorflow model to the yaml model required here.

Targets: Monocle

TorteCnn Options
model
torte model
project
optional output projection
scale
optional log2 output scale

Waveform

Sawtooth

Generate sawtooth waveform.

Generates output values from start to end and then repeats.

Targets: Monocle

Sawtooth Options
T
output type
start
start value
end
end value
step
step value
interval
milliseconds between values

Triangle

Generate triangle waveform.

Generates output values from start to end, end back to start and then repeats.

Targets: Monocle

Triangle Options
T
output type
start
start value
end
end value
step
step value
interval
milliseconds between values

Miscellaneous

comment

A node you can use to add comments to your flows.

Debug only

The edit panel will accept Markdown syntax. The text will be rendered into this information side panel.