Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Note
Documentation under development... 

The Automated Processing section of the RediGate configuration is a collection of processes designed primarily to collect data from various sources (RTDB, text files) and format them into a "report" which may be published via MQTT, HTTP, and/or stored to the gateway for later retrieval.

Specifically, this feature allows the automation of data collection from various types of flow computers, such as Omni, Spirit, AutoPilot Pro, Control Microsystems, TotalFlow, or others. The Automated Processing section also includes miscellaneous features such as device time synch, PLC heartbeat, and other forms of script-driven logic, without requiring ISaGRAF or POD programming.

The Automated Processing configuration consists of the following elements:

  • One or more "Process" objects, specifying a particular form of logic and data to act on (flow meter type, or other processing logic)
    (under Clients → Automated_Processing → Processes).
  • One or more Report definition definitions, specifying the form of the output data to be generated (from RTDB or text file) and a process to publish the data
    (under Clients → Automated_Processing → Reports).
  • Optional - Optional Process Script, allowing user-defined custom logic to assist in the report generation or publishing.
    (under Clients → Automated_Processing → Proc-Scripts).
  • In most cases, the data will come from a Master Channel polling process, such as a Modbus Field Unit.
    Specific flow computers may require certain configuration objects under the Master Channel in order to function properly with the Automated Processing objectsfeature.
    See the RediGate Configuration Manual for details on other configuration properties.

Image Added

(The Automated Processing feature became available in the RediGate beginning in 2019.)

...

AttributesFunction
Object TypeReports
Parent(s)System → Clients → Automated_Processing
InstanceMust be 0

  

Examples of Report Output Formats

Below are samples of the kinds of Report contents that can be created with the Automated Processes Report feature, using different Output Mode selections.

 

Binary

If the Report is defined with three rows, representing three values, such as registers 40001 (UINT16 value=1001=0x03E9), 41001 (UINT32 value=100,000=0x000186A0, with Word Swap), and 42001 (REAL32 value=3.1415=0x40490E56, with Byte and Word Swap), a 10-byte file will be created with the numeric values indicated. The contents of the file (shown here in ASCII equivalents) would be:

03 E9 86 A0 00 01 56 0E 49 04

 

XML (as Attributes Only) - Value is given as a "Value=" attribute.

<?xml version="1.0" encoding="UTF-8"?>
<REPORT Name=”ReportName”>
    <field Name="40001", Value="9999", Units="degF" />
    <array Name=”1stArray”>
       
<field Name="40002", Value="9999", Units="degC" />
        <1stObject>
            <field Name="MyFloat", Value="3.1415", Units="psi" />
            <field Name="MyString", Value="This is a String32 register", Units="none" />
        </1stObject>
    </array>
    <field Name="Calc-CRC-32", Value="0x0", Units="none" />
</REPORT>

 

XML (Values and Attributes) - Value is include between start and end <field> tags.

<?xml version="1.0" encoding="UTF-8"?>
<REPORT Name=”ReportName”>
    <field Name="40001", Units="degF" >9999</field>
    <array Name=”1stArray”>
        <field Name="40002", Units="degC" >9999</field>
        <1stObject>
            <field Name="MyFloat", Units="psi" >3.1415</field>
            <field Name="MyString", Units="none" >This is a String32 register</field>
        </1stObject> 
    </array>
    <field Name="Calc-CRC-32", Units="none" >0x0</field>
</REPORT>

 

JSON

{
    "40001": 9999,
    "1stArray":
    [
        "40002":9999,
        "1stObject":
        {
            "MyFloat": 3.1415,
            "MyString": "This is a String32 register"
        }
    ]
    "Calc-CRC-32": 0x0
}

 

CSV (Two rows, multiple columns)

40001,40002,MyFloat,MyString,Calc-CRC-32
9999,9999,3.1415,This is a String32 register,0x0

 

TXT (Property=Value)

40001=9999
40002=9999
MyFloat=3.1415
MyString="This is a String32 register"
Calc-CRC-32=0x0

 

Process-RTDB-Report

The Process-RTDB-Report object generally defines the structure of a packet of data to be reported using MQTT or some other process on a non-real-time basis. The Report may be defined as either a binary (byte) structure or text format (CSV, JSON, XML, etc.). The Automation “Process” objects contain one or more table rows which are responsible for triggering on certain conditions and calling the Report generation, using data from the Channel/RTU, and then publishing the report using a defined publisher process (in most cases). A Report may be called by more than one Process table row.

...

PropertiesValues
Report Name

This property defines what filename will be used to store the Report contents after it has been triggered by the Process. It is also used as the MQTT topic name if the Report is published using MQTT.

The Report Name may contain replacement variables, allowing the Report filename to be used by more than one Process, resulting in differently named files or MQTT topics. See below this table for a list of Report Name variables.

Forward slash characters (/) in the Report Name will be converted to tilde (~) characters in the filename. Tilde (~) characters in the filename will be converted to a forward slash (/) in the MQTT topic when it is published.

Any part of the filename before a double tilde (~~} will be excluded from the MQTT topic.

For instance, if the Report Name is: ${TIME}~~MyData~${CHAN##}~${RTU#####}/Group${RUN#}/json
then the Report will be saved to a filename something like:
  /tmp/director/20180228T090537.000Z~~MyData~00~00001~Group2~json
  /tmp/director/20181231T235937.000Z~~MyData~03~00044~Group5~json
when triggered by a Process with Channel,RTU,Run that is configured for 0,1,2 or 3,44,5.

The above files would be published via MQTT using the topic names:
  MyData/00/00001/Group2/json
  MyData/03/00044/Group5/json

When using the ProcessScript option for Publisher Process, the Report Name (with full path) is passed into the script as the first command-line parameter.

Output Mode

Select a data format for the Report triggered by an Automated Process. Options are:

  • 0=Binary (no tag names) - Output file contains packed bytes of raw data in the order and byte/word orientation defined by the Report.
  • 1=XML (as Attributes Only) - Output file is XML text, starting with Report Header and with values contained in a "Value=" attribute.
  • 2=XML (Values and Attributes) - Output file is XML text, starting with the Report Header and with values contained inside XML tags.
  • 3=JSON - Output file is JSON text.
  • 4=CSV (Two rows, multiple columns) - Output file is a two-line comma-separated text document, with the first column containing the tag names, and the second row containing the values.
  • 5=TXT (Property=Value) - Output file is text document, with each line containing "Property=Value", where Property is the tag name of the register.
  • 6=HTML - Output file is HTML text (not presently supported).

See below above for examples of each Report type using the Output Mode typeselection.

Publisher Process

Select what to do with the Report file once it has been generated. Options are:

  • MQTT Client - Publish using the MQclient3_1 process.
  • MQTT-Extra Client 0 - Publish using the MQ_Extra_Client3_1 (instance #0).
  • MQTT-Extra Client 1 - Publish using the MQ_Extra_Client3_1 (instance #1).
  • HttpPost Client - Publish via HTTP using the HttpPost object (not presently supported).
  • SparkPlugB - Publish via MQTT using the SparkplugB_RBE process. The Report Name must have the "@" symbol and a register address appended to the name (will be excluded from the MQTT topic), such as "@49001". The register must be a String-32 or String-256 type, which will be published to Ignition using that register's Tagname and the contents of the file as the tag data.
  • Not publishing - Create the Report file, but don't publish it.

 

  • ProcessScript0 through ProcessScript9 - Create the Report file, but instead of publishing it, run a Bash script. The Bash script may use the Bash script or Proc-Script-Text object and must have the script name configured as "ProcessScript#", where "#" is the number 0 through 9. The Process logic will attempt to run the script from /usr/director/bin (Bash scripts created in ACE are stored there, with an extension ".blnk" added).

When running a Bash script for the Report process, the name and location of the Report file is passed in as the first script parameter, followed by the contents of the Report Header field. These items may be used by the Bash script logic as optional command-line parameters that can be used within the script. For instance:

/usr/director/bin/ProcessScript0.blnk /tmp/director/MyData~03~00222~json Report Header Contents

Report Header

When using an XML type Report, the Report Header will be used in the first line of XML text between the "<? ?>" tags, such as:

<?xml version="1.0" encoding="UTF-8"?>

When using the ProcessScript option for Publisher Process, the Report Header is passed into the script as command-line parameters, after the Report filename. This might be used by the Bash script as (space-separated) variables $2, $3, etc.

Number of ArchivesWhen using a Report Name that allows for multiple Reports of the same type/instance to be created (i.e., using the ${TIME} variable), the Number of Archives determines how many copies to retain before eliminating old files, to avoid filling up the file system.
Definition TableThe Definition Table defines the content of the Report, which will generally consist of one data element (RTDB register) per row in the Definition Table. See below for a description of the RTDB Report Definition Table.
Info

Note that every row in the Report Definition table represents one data value to be included in the generated Report. For text-based reports, each data item is identified with a Tagname, or register address if no Tagname is available. (In some cases, a table row represents a start or end of a sub-group or array, rather than a data value. JSON elements within an ARRAY include the value only, without a Tagname.

RTDB Report Definition TableValues
DataReg Offset

If the Process row that calls this Report uses a non-zero DataReg register address, then the Report will reference RTDB registers that are offset from that location. The DataReg Offset may be positive or negative (-65535 to 256,000).

For instance, if DataReg=40101, then if the Report includes rows with DataReg Offset = 0, 2, and -1, the Report data will be taken from RTDB registers 40101, 40103, and 40100.

If the Process row that calls this report has DataReg=0, then the Report must reference absolute RTDB register addresses.

For instance, if DataReg=0, then the Report's would reference DataReg Offset registers of 40101, 40103, and 40100 explicitly.

SrcReg Count

Number of source registers starting at DataReg Offset which are concatenated together into a single data value.

If the data value is a 32-bit integer and comes from two 16-bit registers, the SrcReg Count=2. If the data value is a String and the ASCII values of the string come from multiple registers, SrcReg Count is the number of registers making up a single text string.

Output Format 

Format of data to be stored into the Report output (not necessarily the format of the source data registers). Formats include:

TypeOutput Mode=BinaryBbBinaryOutput Mode=text (JSON, CSV, etc.)
BooleanBooleanBooleanOutput a whole 8-bit byte of 0 or 1.Boolean becomes a string value of "false" or "true", depending on a zero or non-zero sourceBsource value.
Char-8 Store lower 8 bits of register (or a different set of 8 bits, depending on the Byte and Word swapping option Don't use this format option – use the Text String format instead.
Unsigned or Signed Integer, 16-bit or 32-bitOnly used in ACE for visual indication of the source data type. If reading 16 bits from a 32-bit register, use a 16-bit typeOutput either a 16-bit or 32-bit signed or unsigned number. 
32bit Floating Point Only used in ACE for visual indication of the source data type.Output a floating point number.
 Text String
  • If source is STRING-32 register(s), store 32*SrcReg_Count bytes.
  • For integer source If the source register type is an integer, take use SrcReg_Count registers and apply byte/word swapping.
  • If source is STRING-32 register(s), concatenate SrcReg_Count registers together as one string.
  • For integer source register type, assume registers contain ASCII characters. Concatenate up to SrcReg_Count registers until reaching a Null character.
Hexadecimal Integern/a Represent 16-bit or 32-bit source registers as ASCII hex bytes "00" through "FF" in the text Report.
Unsigned or Signed Integer, 64-bit Same as above for 16-bit or 32-bit integers.Same as above for 16-bit or 32-bit integers. 
64bit Floating Point Same as above for 32-bit floating point. Same as above for 32-bit floating point. 
Hexadecimal 64bit Integern/aSame as above for 16-bit or 32-bit hexadecimal.
YYYY-MM-DDTHH:MM:00 from UINT32 EPOCH Seconds or (2) INT16n/aStore one UINT32 or two UINT16 registers as Linux ISO time with "00" seconds.
YYYY-MM-DDTHH:MM:00+TZN:00 from UINT32 EPOCH Seconds or (2) INT16n/aSame, with timezone.
YYYY-MM-DDTHH:MM:00 from (2)UINT32 Date(MMDDYYYY), Time(HHmm)n/aStore two UINT32 registers as Linux ISO time with "00" seconds (first register contains MMDDYYYY, second contains (HHmm). 
YYYY-MM-DDTHH:MM:00+TZN:00 from (2) UINT32 Date(MMDDYYYY), Time(HHmm)n/aSame, with timezone.
YYYY-MM-DDTHH:MM:SS from (2)UINT32 Date(MMDDYY), Time(HHmmSS)n/aStore two UINT32 registers as Linux ISO time with "SS" seconds (first register contains MMDDYY, second contains (HHmmSS).
YYYY-MM-DDTHH:MM:SS+TZN:00 from (2) UINT32 Date(MMDDYY), Time(HHmmSS)n/aSame, with timezone.
YYYY-MM-DDTHH:MM:SS from Omni STRING-16 MM-DD-YYHH:mm:SSn/aTake date from String register in Omni text report format ("MM-DD-YYHH:mm:SS") and store to Report as Linux ISO time with "SS" seconds.

YYYY-MM-DDTHH:MM:SS+TZN:00 from Omni STRING-16 MM-DD-YYHH:mm:SS

n/aSame, with timezone.

Text-8 Chars only
Text-16 Chars only
Text-24 Chars only
Text-32 Chars only
Text-48 Chars only
Text-64 Chars only
Text-128 Chars only
Text-All Chars

Output text fields in fixed length (8, 16 characters, etc.) to ensure the binary Report payload fixed width is not exceeded for a string.n/a - use Text String type.
NESTING STARTn/a

For JSON Report type, create a start of JSON object, using text from the ReplaceName column as the element name.

For XML Report type, create a nested XML start tag using ReplaceName.

Examples:
JSON:    MyTag: {
XML:     <MyTag>

SEARCH-TEXT is ignored for this row. The object or tag should be closed with a "NESTING END" row. All rows in between will be nested data elements in the structure.

NESTING END n/a 

Close a JSON or XML nested structure (SEARCH-TEXT is ignored).

Examples:
JSON:    }
XML:     </MyTag>

 ARRAY STARTn/a 

For JSON Report type only: create a JSON array object with object by adding an opening square bracket (SEARCH-TEXT is ignored). This array should be closed with an "ARRAY END" row. All rows in between will be comma-separated elements in the array, without with values only (no keywords/tag names).

ARRAY END n/a For JSON Report type only: add closing square bracket to an array. SEARCH-TEXT is ignored.
IP-ADDRESSn/a Take four numeric octets (from 32-bit integer, or two 16-bit integer registers) and output IP address in dotted decimal notation. 
Swapping 

Swapping is sometimes needed to reverse byte or word order of the source registers in order to produce the correct byte order for a Binary Report type, or to display the correct value for a text type Report. Swapping options are:

  • No Swapping
  • Swap Bytes (within words)
  • Swap 16-bit Words
  • Swap Bytes and Words
  • Swap DWords (64bit data)
  • Swap DWords,Bytes (64bit data)
  • Swap DWords,Words (64bit data)
  • Swap DWords,Words,Bytes (64bit data)
Name 

Not used for Binary Report format.

For text Report types (JSON, XML, CSV, TXT), the following options are used:

  • "+"  indicates that the Tagname of the DataReg register will be used for identifying the data value. If no Tagname exists, the register number will be used for the tag.
  • Otherwise, the text in the Name column will be used to identify the data value.
Units Units code (in text) is added to XML Report as an Attribute. Not used for all other Report types.

Scale_1
Scale_2
Scale_3

Scale fields are used for all Report types to apply engineering scaling before storing the value into the Report. Only one operation is allowed per field, but up to three Scale fields may be used, which are applied sequentially. Scale should be entered with operator first, then a number. Operators include:

+ (add), - (subtract), * (multiply), / (divide), or  i  (invert source value, then multiple by a number)

Examples (Scale_1, 2, and 3 fields are shown):
Convert Celsius to Fahrenheit:    *9    /5    +32
Convert Fahrenheit to Celsius:    -32   /9    *5
Divide 5.9 over the register value, then add 20.3:   i5.9    +20.3   ____

Spare Unused property. 
Comment Optional column, allowing a descriptive comment to be entered for each row in the table. The Comment field is unused in the configuration.

...

The Report Name may contain a combination of fixed text and replacement variables, which are listed below. When the Report is created, the associated value will be substituted in the filename for in place of the ${ } variable. Any variables shown with one or more “#” symbols indicate that a number will be substituted for the variable, with minimum a fixed number of digits, using leading zeros where necessary.

...

  • ${GATE} - Director/RediGate name configured in the System object.
  • ${GATE#####} - Director/RediGate number configured in the System object (with leading zeros, if more than one #).
  • ${RTU} - Field Unit name from the Channel/RTU in the Process row calling this Report.
  • ${RTU#####} - Field Unit address from the Channel/RTU in the Process row (with leading zeros, if more than one #).
  • ${CHAN} - Channel name from Process row.
  • ${CHAN#####} - Channel number from the Process row (with leading zeros, if more than one #)
  • ${ETH0} - IP address of eth0 (3 digits per octet), e.g. 010.020.030.040
    (presently, only ${ETH0} is supported, but other interfaces might be included in the future).
  • ${SERIAL} - RediGate serial number, e.g. 54310-0001.
  • ${RUN##} - Value from the "Run" column in the Process row calling this Report (with leading zeros, if more than one #).
  • ${PARM1#} - Value from the Parm1 column in the Process row (may be named differently in some Process types).
  • ${PARM2#} - Value from the Parm2 column in the Process row (may be named differently in some Process types).
  • ${GROUP} - Substitute the Group ID from the "Processes" object.
  • ${TIME} - Date/time of system, as "YYYYMMDDThhmmss.000Z" (e.g. 20181231T235937.000Z).
    Note: ${TIME} has to be the first element in the topic.Typically, it should be followed by double tilde, such as "${TIME}~~", so the time is excluded from MQTT publish topic.

 

Examples of Output Mode types

Below are some samples of the types of Report contents that can be created using the Automated Processes.

 

Binary

If the Report is defined with three registers, 40001 (UINT16 value=1001=0x03E9), 41001 (UINT32 value=100,000=0x000186A0, with Word Swap), and 42001 (REAL32 value=3.1415=0x40490E56, with Byte and Word Swap), a 10-byte file will be created with the numeric values indicated. The contents of the file (shown here in ASCII equivalents) would be:

...

XML (as Attributes Only)

...

 

XML (Values and Attributes)

...

 

JSON

...

 

CSV (Two rows, multiple columns)

...

 

TXT (Property=Value)

...

 

Example of a ProcessScript

...