Morse Micro Persistent Configuration Store

Detailed Description

This provides key/value based persistent storage.

The Morse Micro Persistent Configuration Store API is provided to access the values stored in the Morse Micro Persistent Configuration Store. An additional set of Helper functions for loading configuration simplify loading from persistent store into commonly used data structures.

Storage format

Two equal sized copies of persistent storage are stored in separate flash partitions. One is designated primary - this is the highest versioned valid partition. Whenever data needs to be written, the secondary is updated and then the pointers are switched to make the now updated secondary the primary. This ensures that primary always points to a valid and latest copy. For redundancy, this process is done twice so that both primary and secondary are up to date. Flash life can be doubled by doing this only once with the disadvantage that if primary partition gets corrupted then the secondary data will be stale.

Partition Header

Each Flash partition consists of a header containing the following data:

|---------------------|------------------------------------------------------------------|
| Signature (32 bits) | This is the text ‘MMCS’ in little Endian format                  |
|---------------------|------------------------------------------------------------------|
| Version (32 bits)   | This is the version number of the store. It starts at 0 and is   |
|                     | incremented with every update.                                   |
|---------------------|------------------------------------------------------------------|
| Checksum (32 bits)  | This is a bytewise sum of every byte following the checksum to   |
|                     | the last item in the list.                                       |
|---------------------|------------------------------------------------------------------|

The partition header is immediately followed by a sequential list of key value pairs as shown below. The list is terminated when a 0xFF is read instead of a key value pair.

|-------|-----------|---------|----------|---------------------------|-------------------|
|       | Signature | Version | Checksum | Key/Value Pairs (repeats) | End Marker (0xFF) |
|-------|-----------|---------|----------|---------------------------|-------------------|
| Bytes |    4      |    4    |     4    |             n             |          1        |
|-------|-----------|---------|----------|---------------------------|-------------------|

Key-Value pair

Keys

Keys are unique in each copy of the persistent store and are case insensitive. A key must start with an alphabet [A..Z,a..z]. Subsequent letters in the key may contain alphanumeric characters, underscores or periods[A..Z,a..z,0..9,_,.]. The reasoning for this is to allow them to be used as well defined identifiers in a future scripting or CLI implementation.

A key ending with an asterisk '*' is accepted for deletions only (i.e. the data is NULL). Any key matching up to the asterisk will be deleted.

A key starts with a single byte indicating the length of the key in bytes followed by the characters of the key itself. The key must not be null terminated.

A length value of 0xFF indicates end of the list. The 0xFF is excluded from the checksum calculation.

Values

Values are streams of raw data bytes and may be opaque binary data or NULL terminated strings. The Data may be up to 65279 bytes in length. (A length of 0xFFxx could mean a partially programmed flash so is prohibited as it may cause buffer overruns)

Values start with a 16bit value indicating the length of the data stream followed immediately by the raw data bytes.

Supported data types

Internally the persistent store stores all data as raw bytes of opaque binary data. However, for user convenience, we provide helper functions that can convert from common C data types to the binary format and back. Some of the data types we support are described below.

Binary data

The mmconfig_read_bytes() and mmconfig_write_data() function calls can read and write arbitrary binary data like arrays and C data structures into the persistent store. However, please be aware that if you store data structures in raw binary format there is a risk that future versions of the software will be incompatible if the data structure changes.

Strings

The mmconfig_read_string() and mmconfig_write_string() function calls can be used to read and write NULL terminated strings into the persistent store. mmconfig_read_string() will return an error if attempting to read any data from the persistent store whose last byte is not 0 as the function determines that this is not a valid NULL terminated string.

Unsigned integers

Unsigned integers are stored as a string for maximum compatibility - this removes confusion about word width and endianness. The API can be expanded to cater to wider integers and even floating point numbers if needed. The functions mmconfig_read_uint32() and mmconfig_write_uint32() allow the user to read and write 32 bit (and smaller) unsigned integers to/from the persistent store. The mmconfig_read_uint32() will return an error if the data being read is not a NULL terminated string or contains characters that are invalid in an unsigned decimal or hexadecimal number. This function supports reading integers in hexadecimal format if it detects a string starting with 0x.

Signed integers

Signed integers are stored as a string for maximum compatibility. mmconfig_read_int() and mmconfig_write_int() allow the user to read and write signed integers to/from the persistent store. These functions simply test for a negative number and then call the unsigned versions of the conversion functions above after correcting for the sign. The mmconfig_read_int() will return an error if the data being read is not a NULL terminated string or contains characters that are invalid in a signed decimal or hexadecimal number. This function supports reading integers in hexadecimal format if it detects a string starting with 0x or -0x.

Boolean

mmconfig_read_bool() and mmconfig_write_bool() functions can be used to read and write boolean data types the persistent store. Once again for maximum compatibility the data is represented as strings in persistent store. mmconfig_write_bool() simply writes the strings true or false to the persistent store depending on the boolean value being written. mmconfig_read_bool() is a bit more accommodating and in addition to the case insensitive strings true and false also returns true for a non zero numeric string.

Note

Since the above functions with the exception of the binary data functions all store the data in a string representation, it is possible to write the data in one format and read it in another. For example you can use mmconfig_write_int() to write an integer and then use mmconfig_read_string() to read it back as a string and vice versa. You can also choose not to use these conversion functions at all and use only the raw mmconfig_read_bytes() and mmconfig_write_data() functions, in which case the string conversion functions will be optimized out giving you considerable code size savings provided you also exclude the helper functions in mm_app_loadconfig.h which are used in mm_app_common.c.

Operations

Initialization

On startup the system will scan both partitions looking for the signature value. If found, it will then perform a checksum calculation by summing all bytes after the partition header till if finds a 0xFF key length, it then compares this computed checksum with the checksum stored in the header.

If both partitions contain a valid signature and matching checksum, then the partition with the higher version number is designated the primary partition and the other is designated the secondary. If only one is valid then that is designated the primary partition and the other is designated the secondary. If neither are valid then both partitions are erased and the header is written to both with version number 0. Since the first byte following the header is 0xFF this is treated as an empty list.

Writing a new Key-Value pair

To write a new key value pair the system first erases the secondary partition. Then it copies all key-value pairs from the primary partition to the secondary partition. If a key with the same name as the new key is found then it is excluded from the copy. The system then appends the new Key-Value pair to the end of the list. If the new data is NULL then it skips writing the new Key-Value pair effectively deleting the named key. Once this is done, the new checksum is computed and the header is written after incrementing the version number by 1. The written data is then validated and if correct the partitions are swapped and the newly written partition becomes the primary.

Reading Data

To read data we simply scan the primary partition Key-Value by Key-Value till we find the requested key or run into the 0xFF marker signifying end of the list.

Programming the config store from a host PC

This section provides instructions for programming the config store from a host PC via the command line. Alternatively the config store may be programmed through the Platform IO UI.

• Connect the target on USB and start OpenOCD in a terminal by running the following command from the framework directory. If OpenOCD starts successfully it will print some information messages and then pause waiting for a connection. If OpenOCD exits then this indicates that an error was encountered while starting (for example, the device was not connected or another OpenOCD instance was already running).

  openocd -f src/platforms/mm-ekh08-u575/openocd.cfg

  Replace mm-ekh08-u575 with the actual name of your platform.
• In a separate terminal, also from the framework directory, run the following command to load the keys to the device:

  pipenv run ./tools/platform/program-configstore.py -H localhost -p mm-ekh08-u575 write-json
  <config.hjson>

  Replace mm-ekh08-u575 with the actual name of your platform, and substitute <config.hjson> with the path to the configuration file (e.g., ../examples/<app>/config.hjson where <app> is the name of the appropriate application directory).
• You can now stop OpenOCD by returning to the original terminal and pressing CTRL-C.

Modules
	Morse Micro Persistent Configuration Store API
	This API provides functionality for managing configuration data in a key/value store with a flash backend.