OpenLCB: Memory Access Configuration Protocol

This document defines a protocol for configuring OpenLCB nodes by directly accessing their configuration memory.

See also the separate note on a Configuration Description Information.

Environment of Protocol

Requirements

• Nodes must carry enough context that a stand-alone configuration tool can provide a useful human interface without getting any data from an external source, e.g. needing an Internet download to handle a new node type.

• It must be possible to configure a node entirely over the OpenLCB, without physical interactions, e.g. pushing buttons. This configuration must be compatible with local configuration, including e.g. the Blue/Gold method.

• It must be possible to configure one or more nodes while the rest of the OpenLCB is operating normally.

• It must be possible to read, store, and reload the configuration of a node.

Preferences

• Small nodes shouldn't need a lot of processing power, e.g. to compress or decompress data in real time. Memory usage should also be limited, but is a second priority.

• Configuration operations should be state-less and idempotent to simplify software at both ends.

• Multiple independent configuration operations can proceed at the same time. Multiple devices should be able to configure separate nodes at the same time. Multiple overlapping reads of the same node should be possible. There should be a method to coordinate separate configuration operations to simplify configuration software.

• For efficiency, atomic reads or writes of small amounts of data should fit into a single-frame CAN datagram. Single-bit writes also add efficiency.

• Multiple address spaces make it easier to handle multiple types of data.

• For large transfers, it's desirable to be able to use streams. Not all nodes support them, though, so it must be possible to enquire about capabilities.

• Addresses should be four bytes. Two address bytes is a failure of imagination.

Design Points

Basic configuration is done with datagrams, which can carry 64 bytes of data to read or write.

For efficiency, writes don't need any reply except the datagram response. By sending that only after the write is complete, including non-volatile memory delays, the written node can control the transfer rate.

Read operations must return data in a separate datagram. If this carries the address, etc, in addition to the data, the operation is idempotent.

Read and write operations can address separate kinds of information in the node via specifying specific address spaces. Three of these have specified uses, and the reset are optional tools for future expansion.

There are a large number of possible configuration options, and more may be developed in the future: Reset, identify, etc. Not all nodes will implement all methods, so a query operation is needed so that tools can know what they can do.

Can't require that nodes do anything in particular to put changes into effect. Some will do it right away, some will require a reset, etc. Need flexibility for node developer, yet consistency for configuration tool builder.

CAN Aspects

CAN configuration datagrams have seven payload bytes in the 1^st frame (the first byte identifies that it's a configuration transfer). See the “Datagram Protocol” document for more info. Read operations and 2-byte (or 1-byte-under-mask) writes can be done with a single CAN frame.

Read-Stream and Write-Stream need to rendezvous on the stream IDs to know which stream is carrying the data. See Examples section below.

Protocol

Configuration messages use a specific datagram format consisting of a single byte combining the operation “Command Type” field and flags. This is followed by data in an operation-specific format. When present, the four-byte memory address follows first, then the address space byte if present, then other data.

Address Space Size

Configuration addresses are 32 bits. The addressable quantity is the byte. This allows directly addressing 4GB of data. The use of address spaces (see below) allows direct access to 1TBi.

The large address range removes the need for address registers and other non-idempotent accesses when accessing e.g. sound information in a large memory.

Address Space Selection

Although a 32-bit address space is large enough to cover combined uses of memory, it can be more convenient to consider separate address spaces in the node. (This can also be considered to be a top digit in a global address space, if you want to, but note that the separate address spaces may cover the same memory objects, e.g. “all memory” and “Event ID configuration” spaces may reference the same configuration memory)

Required space definitions (these may or may not have content on a particular node); these address space numbers can only be used for this, and if the information is available, it must be accessable by these numbers (in addition to any others the designer might provide):

• (0xFF, flag=11) Configuration definition – reading this is how you get the configuration definition

• (0xFE, flag=10) All memory – provides access to “all” memory in the device, where “all” is defined by the designer. Single, flat address space for access. Can be used for e.g. dynamic access to RAM for monitoring & debugging.

• (0xFD, flag=01) Configuration - basic configuration space, with the structure of the 32-bit space defined by the designer.

These three spaces, inclusive, can be addressed without an extra byte in the datagram using control bits in the flag byte. All others need to be specified as a byte value. The high addresses (0xFD through 0xFF) were chosen for the dedicated spaces so that space numbers 00, 01, 02 could be used as a 5^th bytes in a contiguous address if desired.

Message Formats

The following table shows available configuration operation formats. All others reserved. They must not be skipped during identification. Items in {} are optional.

Operations

Read, Read-Reply

Attempts to read from an invalid location, either outside the available address range in an valid address space, or from an invalid address space, still require a return message, requires a reply datagram with an data length of zero.

In general, a read reply may provide less than the requested data, but always at least one byte if it's a valid read.

Get Configuration Options Reply

To make it possible to make simple/cheap nodes, not every configuration operation & option needs to be provide. The reply to “Get Configuration Options” provides information that a configuring device can use to control how it communicates with the node so that it only uses available modes.

• Available operations mask (2 bytes, bit coded): Indicate which operations are available so the using software can know whether convenience operations (which are not possible on some hardware) are available.

  • 0x8000 Write under mask supported

  • 0x4000 Unaligned reads supported. If not set, reads have to start on an address with the low bits, as given by the data size, all zero. For example a 4-byte write must have the low two address bits zero.

  • 0x2000 Unaligned writes supported. If not set, reads have to start on an address with the low bits, as given by the data size, all zero. For example a 4-byte write must have the low two address bits zero.

  • 0x0800 Read from address space 0xFC available (this is the manufacturer part of Abbreviated CDI)

  • 0x0400 Read from address space 0xFB available (this is the user-entered part of Abbreviated CDI)

  • 0x0200 Write to address space 0xFB available (this is the user-entered part of Abbreviated CDI)

  • Others reserved, must be ignored on receipt and sent as zero.

• Write lengths supported (One byte, bit coded): (provided for devices that can only write certain sizes to memory) (at least one bit must be set)

  • 0x80 1 byte write

  • 0x40 2 byte write

  • 0x20 4 byte writes

  • 0x10 64 byte writes (full datagram, but not 63 bytes or arbitrary length, just exactly 64)

  • 0x02 arbitrary writes of any length OK

  • 0x01 stream writes supported (stream support will identify buffer size)

  • Others reserved, must be ignored on receipt and sent as zero.

• Highest Address Space (byte): Highest number space available. Not all up to that need be available, but sparse allocation will slow down the process as “Get Address Space Information” is needed to determine whether they are present.

• Lowest Address spaces (byte): Lowest number space available. Note that spaces 0xFD, 0xFE and 0xFF are assumed to be included even if the low space ↔ high space range doesn't include them. (also 0xFC, 0xFB of Abbreviated Default CDI if bits indicate they're available)

A node that only has the high spaces could have Highest Address Space = 255, Lowest Address Space = 253 or 251.

A node that has additional low address spaces, e.g. to make more memory available with a 28-bit address, could have Highest Address Space = 127, Lowest Address Space = 0 and leave the top spaces assumed.

Get Address Space Information Reply

To ease automated access, a configuring node can enquire about the address spaces in the being-configured node. Whether or not the address space is present, a reply is required.

• Present: This is carried in the lowest bit of the command byte, just below the reply bit

  • 0x01 == 1: Present. == 0 not present.

• Space ID – provided to identify request this is in response to

• Highest Address (4 bytes)

• Flags (byte) – (Alignment and size were going to be here but were made global above); Read-Only is LSB, can write if 0, can only read if 1

• Lowest Address (4 bytes) – optional, omit if zero, as that will let reply fit in single CAN frame.

• Description (variable length) – optional null-terminated string giving the user-readable name of this space

Lock/Reserve and Freeze/Unfreeze

An OpenLCB node can, in general, be configured while the network and even the node itself is operating.

Code can be simplified by disabling operation of a node while it's being configured, so that there's no concern about it trying to react to transient incomplete information. The Freeze/Unfreeze command, if supported, can be used to tell a node that it should “freeze” operation, ignoring inputs, while the configuration is being updated. A reset of the node releases the freeze option, if set.

Although nodes can be configured by multiple other nodes, this can also lead to inconsistencies. The optional Lock/Release command can be used to avoid this. At the start of configuration, a configuring node sends a Lock message with its NodeID. If no node has locked this node, indicated by zero content in the lock memory, the incoming NodeID is placed in the lock memory. If a node has locked this node, the non-zero NodeID in the lock memory is not changed. In either case, the content of the lock memory is returned in the reply. This acts as a test&set operation, and informs the requesting node whether it successfully reserved the node. To release the node, repeat the lock operation with a zero NodeID. The lock memory is set to zero when the node is reset. Note that this is a voluntary protocol in the configuring nodes only; the node being configured does not change it's response to configuration operations when locked or unlocked.

Get Unique EventID

Nodes maintain a list of unique EventIDs for use in configuration. These are allocated based on the node's unique NodeID. This command allows a configuration tool to get new unique EventIDs from the node's pool, for example to interact with the Blue/Gold configuration process. Each request must provide a different EventID, without repeat, even through node resets and factory resets.

Update Complete/Reset/Reboot/Reinitialize

This is a collection of three operations, distinguished by what are normally the flag bits.

The configuration protocol does not specify the meaning of the transferred data. In particular, it doesn't specify when new configuration information takes effect. Depending on how the node is constructed, this might be immediately upon transfer (although this raises issues of write boundaries), or when an entire sequence of transfers is complete. “Update Complete” is the command that indicates that a series of configuration writes is consistent and complete, and the node can put it into effect. Nodes do not have to require this operation, but receiving it must be permitted. Configuration tools should send it at the end of operations. Nodes may, but are not required to, reset after sending the reply to this message.

The “Reboot/Reset” command is meant to reinitialize a node, equivalent to powering it up. Nodes should finish any pending operations, e.g. non-volatile memory writes, before doing the initialization. It's expected that the datagram reply will be sent before the reset, but this might not be entirely reliable. Configuration tools should not count on the reply. The configuring node will receive a “Node Initialization Complete” when the node is back up. This operation must not reset any configuration information to default contents.

“Reinitialize/Factory Reset” is similar, but includes restoring the node's configuration as if factory reset. (This may require creating new unique EventIDs, see other note) This is a heavy-weight operation which may require some form of interlock, e.g. the user pressing a button, to prevent inadvertent data loss. As a small safety precaution, the NodeID of the note being reset is redundantly carried in the data part of the datagram.

Indicate

This command tells the board to somehow identify itself to the user, for example by flashing a LED or operating it's outputs. This allows the user to be absolutely sure that he's configuring the correct board. “Start” (bit 0 = 1) means that the board should start indicating, and “Stop” (bit 0 = 0) means that the board should stop indicating. The data portion carries information that lets the board know what kind of indication to do. It's not always appropriate to operate outputs if they're e.g. driving large mechanical systems like doors. (This needs to be specified more precisely)

Implementation Notes

This section is non-normative notes and suggestions for implementors.

Example CAN Operations

These are over CAN and includes some information from the datagram protocol to make traffic clearer. [d] is a single datagram; [di][di][de] is a datagram that's in multiple frames.

Get initial information

Get Configuration Info [d] →

← datagram reply

← Get Configuration Info Reply [d]

datagram reply →

Write byte

Write [d] →

← datagram reply

Write 64 bytes

Write [di] →

Write [di] (7 times) →

Write [de] →

← datagram reply

Read byte

Read [d] →

← datagram reply

← Read Reply [d]

datagram reply →

Read 64 bytes

Read [d] →

← datagram reply

← Read Reply [di]

← Read Reply [di] (6 times)

← Read Reply [de]

datagram reply →

Large read via stream

Read [di] →

Read [de] →

← datagram reply

← Read Reply [d] (carries stream ID)

datagram reply →

← Stream Initiate Request (carries a buffer size equal to memory write line size)(stream ID from above)

→ Stream Initiate Reply

← Stream Data Send (N times; broken down to frames, 8 bytes each)

Stream Data Proceed →

(repeats until done, then)

← Stream Data Complete

Large write via stream

Write [d] →

← datagram reply

Stream Initiate Request →

← Stream Initiate Reply

Stream Data Send (N times; broken down to frames, 8 bytes each) →

← Stream Data Proceed (after write to memory complete)

(repeats until done, then)

Stream Data Complete →

Delays Due to Non-Volatile Memory

Some microcontrollers can't continue to operate while writing configuration information to non-volatile memory.

If the CAN buffering is sufficient (at about 1usec per buffer) for the node to become active at the end of the memory operation and process buffered frames at the full rate, there's no issue.

If a node has missed one or more frames, it's possible that some state interaction has started and the node is inconsistent. For example, a RIM/CIM sequence could have started or even finished.

If the node misses one or more frames, the CAN controller needs to flag that and bring it to the attention of the node's microcontroller. The node then needs to emit an “Initialization Complete” message so that other nodes realize that this node may not have complete state information.

The node should also defer the reply to a write datagram until after the write is complete, to make it less likely the next-in-sequence operation arrives during dead time. Nodes doing the configuring should limit the amount of traffic they send to the node-under-configuration to reduce the need for e.g. datagram retransmission.

The transfer size for stream access is negotiated. By requiring a transfer size equal to or smaller than the memory write size, the node can ensure that the stream will pause during a write operation.

Large Volume Operations

The stream protocol is meant for large reads and writes, but the datagram protocol can also work well on a single CAN segment. The difference in performance comes from the (potential) larger datagram buffer size.

All nodes support datagrams for configuration; not all support streams. So a least-common-denominator configuration tool would use sequences of datagrams for even large transfers. Because the need for reply, short datagrams are not particularly efficient. In the limiting case, you can only write two bytes per frame exchange. The sending node should look at the Get Configuration reply and use the largest available size.

On the other hand, if non-volatile memory timing requires that write operations to a node use a 64-byte or smaller stream buffer size, then datagrams are a more efficient method than streams. In that case, the node should indicate that stream-write operations are not supported in its reply to Get Configuration. Since read operations don't have the same timing issues, they may still be useful in that case.

Site hosted by

This is SVN $Revision: 2160 $