WebSockets JSON Communication Protocol

[Cube]

In order to help facilitate and expedite the Web User Interface development, I have begun implementing a new WebSocket protocol that exchanges messages in JSON format. The development of this protocol will be described here, since it is also a good medium in which to discuss the development process and its' progress.

[Cube]

Here is an example of the first working command in the JSONii protocol. When a new client connects to Decider's websocket interface, one of the first things it is going to want to do is register to receive the real-time status of one or more Value Interpreters (VINTs). For example, to register to receive status from VINTs 21315, 21319 and 21311, the client would send the following JSON to the server on the websocket interface:

Code: [Select]

{
   "command":"RegisterVUIDs",
   "vuids":
   [
      21315,
      21319,
      21311
   ]
}
Decider would add these value interpreters to the session of this client, send the current value and status of each of the requested VINTs, and any subsequent updates to any of those values would be sent to the client. This is the message that would be initially received after the registration request above:

Code: [Select]

{
   "VintValues": [
     {
       "VUID": 21315,
       "DeviceStatus": 1,
       "Value": 0
     },
     {
       "VUID": 21319,
       "DeviceStatus": 1,
       "Value": 0
     },
     {
       "VUID": 21311,
       "DeviceStatus": 1,
       "Value": 0
     }
   ]
 }


From this point forward, this client will receive updates to either the value or the device status of these Value Interpreters.

[Cube]

Now I am faced with a bit of a shift in approach. Whereas with the vinterface protocol was so succinct that I could just fire off update messages as their own, single entities, it probably makes sense to amalgamate similar messages in the JSON format into a single message with multiple members. However, this changes the way messages get prepared and there is a bit more overhead since you have to create a message "object", populate it with your payload data, and then commit or send the completed message object in the end. This, as I said, contrasts with the vinterface approach where each payload was it's own message. Is there a benefit to keeping this mentality, even if it adds overhead to the packages? For the example above, would it make more sense to deliver the message as shown, or to send three individual "VintValues" messages? It is certainly possible that I am over-thinking this. :| I think I will take the same approach as the vinterface style protocol - one message per JSON object.

[Cube]

Quite happy with the progress in Decider today on both the cleanup and re-thinking of the vinterface protocol and the newly forming jsonii web socket protocol, I went to update my test container to the latest versions of the code. Quickly I ran into the issue that the version of libwebsockets that ships with Fedora 26 (which I had been running) was 2.2-based. Decider's code now includes features and functionality of the 2.4 series of libwebsockets, so I upgraded the container to Fedora 27, which brought with it libwebsockets 2.3. Still not quite enough. I then installed the 2.4.0-1 package of LWS from Fedora 28 rawhide, compiled and ran, only to be met with this old error (segmentation fault) when I tried to connect to the web socket server:

Thread 7 "websocket" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffee5ea700 (LWP 15462)]
0x00007ffff577fdc0 in SSL_get_SSL_CTX () from /lib64/libssl.so.1.1
Missing separate debuginfos, use: dnf debuginfo-install gsoap-2.8.49-3.fc27.x86_64 json-c-0.12.1-5.fc27.x86_64 libwebsockets-2.4.0-1.fc28.x86_64 libxml2-2.9.7-1.fc27.x86_64 mariadb-libs-10.2.10-2.fc27.x86_64 openssl-libs-1.1.0g-1.fc27.x86_64 xz-libs-5.2.3-4.fc27.x86_64 zlib-1.2.11-4.fc27.x86_64
(gdb) info stack
#0  0x00007ffff577fdc0 in SSL_get_SSL_CTX () from /lib64/libssl.so.1.1
#1  0x00007ffff67ba378 in lws_server_socket_service_ssl () from /lib64/libwebsockets.so.12
#2  0x00007ffff67bd771 in lws_adopt_descriptor_vhost () from /lib64/libwebsockets.so.12
#3  0x00007ffff67bdb13 in lws_server_socket_service () from /lib64/libwebsockets.so.12
#4  0x00007ffff67af529 in lws_service_fd_tsi () from /lib64/libwebsockets.so.12
#5  0x00007ffff67bbc0b in _lws_plat_service_tsi () from /lib64/libwebsockets.so.12
#6  0x0000000000445f3a in websocket_server_main () at ws/ws-serv.c:197
#7  0x00007ffff717d61b in start_thread () from /lib64/libpthread.so.0
#8  0x00007ffff4ffb98f in clone () from /lib64/libc.so.6
(gdb)

Ah, the dawning recollection that in order to get it working on my development VM I'd had to compile lws from source with SSL disabled in order to get it to work properly. Here lies the fork now set in the road before me: Do I pursue resolution with the stock standard version of the library, trying to figure out why it crashes with an SSL error even though I am not using encrypted connections in my tests, or do I build lws from source, remove SSL functionality (which I know works) and continue on my merry way?

[Cube]

In the end I went with the correct option, solve the problem once and for all. I did not want to be bound to a "specific" configuration of the library, but considering that there was practically nothing turning up in my searches of the Internet, I came to the determination that the problem had to be in my code, the most likely of all scenarios. As it turned out, this was the case and it came down to my use of the LWS_SERVER_OPTION_ALLOW_NON_SSL_ON_SSL_PORT option when configuring the libwebsocket context. In 2.2, this did not cause an issue, and I recall using both http+ws:// and https+wss:// on the same port, for which (I believe) this option was necessary. Now, however, that may have changed but whatever the case - removing that option causes the library to run as intended and I can now move on to more interesting (and less frustrating) matters.

[Cube]

I have created a JSON test web interface, for easy testing of this protocol. The .php file itself can serve as a good example of how to write web apps that interface with Decider using the jsonii protocol, and it has been very responsive.

[Cube]

Alright, I have completed implementing the RegisterVUIDs and SetDVAL methods, though I am not sure what the best way to describe the protocol is, so here's my take at it with words:

In order to receive notifications of the current values of things, each user interface session must register the VINT Unique ID numbers (VUIDs) of each Value Interpreter they are interested in getting the current value of and any subsequent updates for the duration of the Websockets connection (session.) This is accomplished by sending one or more RegisterVUIDs JSON objects to the Websockets connection, with an array of integers as the value for the RegisterVUIDs object. For example:

Code: [Select]

{"RegisterVUIDs":[1,5,9]}
Decider will acknowledge this message by registering the current session as being interested in the given VINT values, and will send the current values and Device Status values of each VINT that was registered as JSON objects. Note: There is a stub function in Decider that will check the permission of the current user (once this feature is implemented) which could inhibit one or more VINTs from returning its status.

Code: [Select]

{"VintValues":[{"VUID":1,"DeviceStatus":1,"Value":1000}]}
{"VintValues":[{"VUID":5,"DeviceStatus":1,"Value":1000}]}
{"VintValues":[{"VUID":9,"DeviceStatus":1,"Value":1000}]}

Any subsequent updates to registered VINT values will be sent to the client as JSON objects in this fashion.  If the device that supplies the VINT value goes offline (Decider loses communication with the Venturii Module through which that device is communicating, or the Venturii Module itself loses communication with the physical device) the Device Status will change from 1 to 0 (offline) and the value should be considered at best "Last Known Value" or more accurately "Indeterminate" . It will be up to the user interface to decorate the representation of this data source in a manner suitably indicating its loss of communication.

SetDVAL works in a similar manner: If the client (usually at the direction of the user) wishes to change the value of any Value Interpreter, a SetDVAL JSON object must be created and sent to Decider. Multiple VINTs can have their desired values set in the same command, the SetDVAL method must contain an array of at least one object, and that object must be comprised of two members: A vuid member whose value is an integer representing the VUID number of the Value Interpreter to change the Desired Value of, and a value member whose value may be a double, a float, an integer or a string representation of any numeric format, and represents the value the user wishes to set this particular Value Interpreter to. For example:

Code: [Select]

{"SetDVAL":[{"vuid":1,"value":"1000"}]}
Decider will attempt to effect this change, but will only send a VintValues object if the value of the VINT actually changed from it's previous value.