Streaming

Streaming in the openDAQ™ architecture refers to the transport layer mechanism for continuous and real-time receiving of data from a data acquisition Device, often facilitated by the use of protocols like Streaming LT, or openDAQ™ native streaming. The data transmission follows a subscription-based model, where the client subscribes to Signals' data produced by the Device. Streaming mechanism involves the receiving data in binary format over a communication channel (e.g. network), and deserialization it into openDAQ™ Packets. The Packets are added to Mirrored Signal which may be connected to Input Ports.

Data Streaming and Device structure Modules

The main purpose of the generalized Streaming mechanism is to decouple data streaming from the Device’s structure information and configuration transfer mechanisms. The separation is achieved by assigning specific responsibilities to different openDAQ™ Modules. Modules such as the OPC UA server/client Modules manage the transfer of structural information, while data streaming is overseen by diverse Streaming server/client modules.

Despite their distinct roles, these two types of Modules can indirectly interact with each other, both for Server Modules and Client Modules. A Server Module responsible for transferring Device structural information can gain awareness of Streaming Servers added to the openDAQ™ instance. This enables the publication of information about available Streaming connections alongside the device’s structural details. Simultaneously, the Client Module handling structural information transfer receives the information about Streaming connections propagated by the remote Device. It then delegates Streaming connections establishment to the appropriate Streaming Client Modules based on user-defined configuration.

How-to: Server side configuration

Streaming connection string

A specific Streaming protocol implementation is associated with an openDAQ™ Module. The SDK handles the routing and delegation of Streaming object instantiation to the appropriate Module through the use of a connection string. The connection string always starts with the protocol prefix, followed by a set of parameters unique to the protocol type. Some parameters, such as the IP address, might be obligatory, while others, like the port number, are optional and adopt default values if not explicitly specified.

For example, the connection string "daq.ns://127.0.0.1" will be accepted by openDAQ™ Native Streaming Client Module initiating a connection with the openDAQ™ Native Streaming Server on the local machine, utilizing the default values: port number 7420 and path "/" . Similarly, the connection string "daq.lt://192.168.1.3:7413/streaming" will be accepted by the Streaming LT Client Module, and this module will initiate a connection with the Streaming LT Server hosted on a remote machine with the address "192.168.1.3", employing the specified values for the port number (7413) and path ("/streaming") .

Instantiating Streaming upon Device connection

The openDAQ™ SDK supports two categories of Devices characterized by their approach to Streaming utilization:

Devices exclusively built on the Streaming protocol, commonly referred to as pseudo-devices. These Devices do not provide full structural information, but only offer a flat list of Signals. Such Devices are created through the Module responsible for establishing the relevant Streaming connection.
Devices equipped with comprehensive tree structure information (e.g. available Function Blocks, Subdevices, etc.) obtained from structure transfer Modules. For those, Streaming is integrated with Mirrored Signals and is only used for data transfer purposes.

Regardless of the Device type, upon connection, it should be able to immediately start the data streaming. To accomplish this, the client Module responsible for creating the Device object also instantiates a Streaming object. It is achieved in two ways:

for pseudo-devices with a flat list of signals the Module creates a Streaming object on its own using the connection parameters specified in the device connection string.
for devices supporting a complete tree structure, the Module delegates the creation of the Streaming object to another Module using the information about supported Streaming protocols published by the Device.

The above also implies that, when connecting to a structure-enabled Device, all its streaming capabilities are discovered, including those of its sub-devices, making it possible to establish connections with all of them, enabling flexibility in choosing which to use. However, there also exists a configuration mechanism to control and limit the automatic establishment of streaming connections during device connection. The configuration mechanism enables the filtering of Streaming capabilities available per device based on protocol type. Additionally, it provides control over the number of hops the streamed data must make between nested devices.

How-to: Connecting to Streaming protocol based pseudo-devices
How-to: Configure Streaming for structure-enabled Device

Mirrored signals

A Mirrored Signal serves as a mirrored representation of the Signal originating from a remote Device. It allows for the inspection of the structure of the remote device and modification of its configuration. Additionally, a Mirrored signal acts as the medium for receiving device data on a client. Each mirrored signal has a list of streaming sources based on established streaming connections. Among these sources, users can individually select a Streaming connection for each Signal to act as the active streaming source — only that connection will transmit data through the Signal.

In addition to the capability to select the streaming source for a signal, it is also possible to completely disable data streaming for each Signal individually. This is achieved by controlling the Streamed boolean flag of the signal through setStreamed / getStreamed calls.

Signals can be associated or disassociated with instantiated Streaming objects using the addSignals or removeSignals calls, respectively, in the Streaming object. These automatically add / remove the Streaming source to / from list of available Streaming sources for signal.

How-to: Configure Streaming per Signal

Automatic Signal subscription

To minimize network traffic and overall load, the openDAQ™ Streaming mechanism incorporates the automatic deactivation of data streaming for Signals that are not in use. A signal is considered in use only if it is connected to an Input Port. The client automatically sends a subscribe request to the streaming server when a signal is connected to an Input Port, and an unsubscribe request is sent when the Signal is disconnected from Input Port. The Streaming server handles these requests, facilitating the enabling or disabling of data transmission.