Recent Posts

11

Decider / Websockets JSON and Web UI Control Surface Pages

« Last post by Cube on February 20, 2018, 04:16:03 PM »

I re-implemented a function today at the request of JH whereby upon registration of one or more VINTs for a Websocket session, Decider will send the current values and Device Status of all registered VINTs to the client in a single JSON object (an array) like so:

Sample Request:

Code: [Select]

{"RegisterVUIDs":["1","5","9"]}
Sample Response:

Code: [Select]

{"VintValues":[
   {"VUID":1,"DeviceStatus":1,"Value":2000},
   {"VUID":5,"DeviceStatus":1,"Value":2000},
   {"VUID":9,"DeviceStatus":1,"Value":2000}
]}
One thing that occurred to me, and this from a place of ignorance about the way in which JH's node.js web interface works, as he changes pages from one control surface to another, it is possible (likely?) that the client will retain the web socket connection, but need to register for a new set of VINTs (and possible un-register other VINTs) While there would be nothing wrong with keeping them registered, we would simply send values it had no place to display, at present there is no mechanism yet in place to unregister VINTs within a given session. A couple of options lay before me:

• Implement an UnregisterVUIDs command.
• Instruct the client to disconnect and re-connect when a new set of VINT subscriptions is required.
• Do nothing (Keep all registered VINTs in the session and continue to send their updates.

I'll probably go with option #1, but is there any other option I am overlooking? Is there an advantage to going with #2 or #3 that is not apparent?

12

V_Buscom / Re: Crossroads

« Last post by Cube on February 08, 2018, 04:55:57 PM »

Of course, no sooner did I get to creating the first plugin module (Modbus) than I hit a hurdle: The libmodbus library takes care of all it's own communication. One of the first examples it provides reads:

Code: [Select]

modbus_t *ctx;
ctx = modbus_new_tcp("127.0.0.1", 1502);

Thinking about it deeper though, this may not be a problem at all. CLI handling will need to be modified somewhat, but a series of hooks and callback functions ought to modularize this aspect nicely. For example, I could have the base command handler function check to see if we have a module type installed matching the value of arg[0], and pass off cmd processing to that plugin's command parser.

Another problem I encountered is that under the present system the channels run in their own threads, talking to whatever devices are associated with them. If we create a Modbus RTU device, it will not be associated with any channel, and therefore will not have any thread to work in. We have a couple of options then, we could:

• Make it a plugin option to create a thread for the module, passing control to a module worker function tasked with looking after the communication of all that module's devices.
• Make an option per device to have a dedicated worker function managing that device's communication, each of which runs in it's own thread.
• Something else?

I'm leaning towards option #2 personally, since this gives the most flexibility. This way, a device can be put on a communication channel, or it can do it's own thing. Of course, each module could define it's own communication requirements as part of the API. A module for DSC panels may specify that it's devi ... BREAKING THOUGHT We could add a parameter to the channel structure so that it designates what each channel is for. Channels may be unreserved, IE: Are not linked or tied down to any one module (such as they presently are.) They could also be reserved for a specific module type (this would help clarify their purpose / function when listing the channels and would also prevent the user from inadvertently placing devices from two or more different modules on the same channel. We now return you to your previously scheduled programming thoughts... Back to the DSC panel example, we could create a Plugin for DSC panels. (Maxsys, Power Series and Neo). This plugin would indicate that it's communication is handled at the channel level, and would provide a function pointer to it's main worker thread, one worker for each channel thread. The panel structures would be shared within the plugin code, but protected from multi-threaded access by rwlocks as we have gotten very good at writing, Thus each worker thread would scan the plugin's array of devices, acting only on devices registered for communication on that specific channel.

That approach would then be complimented by the modbus mode, whereby the plugin's devices do not need to associate with any channels on their own (maybe we could even prohibit this (or at least ensure it does not become and option for selection) and enabling a modbus plugin device would create a new thread specifically for handling that device. I think this makes sense, it's becoming clear in my mind -- now to try to implement it!

13

V_Buscom / Crossroads

« Last post by Cube on February 08, 2018, 11:06:52 AM »

Now it comes to that time. Decision time. A Fork In The Road. Buscom appears to be working nicely, and is nearing the point where I would be cloning it into a new project to begin implementing the Mosbus connectivity. But herein lies the crossroad: Is there any benefit to making that an external module versus just incorporating the modbus functionality into buscom? Obviously it need not be utilized if not needed, but what reasons could be contrived to have them separate? Perhaps there is an argument to be made here for using the codebase as it is as a plug-in host for all manner of device-specific processing, since all the other code largely remains the same. I'm thinking of revitalizing the DSC integration module, but again - it would essentially be based on buscom with all it's established code and functionality. I wonder if there might be a way we could standardize on the buscom code and somehow create "plug-in" modules that interact with specific hardware, building on what we've already created. This would simplify the maintenance of code, bugfixes and enhancements to the core would need to be applied to each and every module. What does a modularized system even look like? I've never built "kernel modules" (well, built - yes. Designed / coded: no.) We'd need some demarcation point, probably after the channel had been connected, to turn processing control within each thread over to a function within the module space for "doing something" with that connection, whether that be querying one or more devices, listening for data, or what have you. The module space code would need to be able to allocate it's own memory and Venturii Value Interpreters, generate it's own events, etc. Perhaps this may even be a good place to introduce Lua to the mix, and code the module-specific stuff in Lua script (if that's even what you call it.)

Within the Modbus module, for example, there are dozens of registers, each one holding 16 bits of data. My plan was to create the modbus module such that when you created a channel, you would then add a device (similar to the vdac system) and tell that module how many registers to allocate and the starting address within the PLC. This would then create n * sizeof(register) Value Interpreters (since for us to be able to do things intelligently within Venturii, we pretty much need access to each individual register as a separate entity that can be read and written to independently of all the others. While not the most efficient use of space, (multiplying storage by at least a factor of the functionality requirement outweighs the "waste of bits". But then from there, we haven't much to do. Read all the registers, set the values of each register if any of our VINTs have dvals set, and sleep between poll cycles. The module code would retain control until or unless an error condition occurred by which it became apparent (or desirable / necessary) that the underlying channel connection be reconnected. The module space code need only return (or maybe call some "disconnect()" function and then return) to pass control of the thread back to the buscom code base, which would try to re-establish the connection.

Hmm, the more I think of it, the more I like this paradigm. In fact, i can see all the Venturii modules unifying under this architecture, which would simplify a LOT of code, partially by actually removing most of it, and then only incorporating the device or hardware-specific code itself within these modules. You could even have a single Venturii module talking to many different device types, all within the same process. Arguments could be made on both sides of the table for or against this approach from a single-point-of-failure, redundancy, division of labour or many other basket-to-egg-ratio perspectives, but having the option to hold all the eggs would not mean you couldn't then run multiple "baskets", even having a separate basket for each egg.

This is why I love writing these things down and working them out in text; great ideas form this way and I'm sure down the road it may even be interesting to have a record of the thought process as it formulated within my mind! Haha  The inner workings of my brain on display. Viva Venturii!

14

General Discussion / Snow Day!

« Last post by Cube on February 08, 2018, 08:23:49 AM »

I climbed up on the roof of my garage a few days ago to clear off the snow from my solar panel, only to wake up this morning to another 25 cm of snow sitting on the roof. The entire solar panel is buried so that you cannot even make out where it lays on the roof! Everyone is staying home today. It is remarkable how much snow can accumulate!

One of my favorite sounds on a frigid day like this is the gentle Ahhhh of heated air rising through the vents in the floor, warming the house. Venturii calculates the difference in temperature between the inside air and outside air, and adds more heat inside when it gets colder outside to compensate for the accelerated heat loss through windows, walls and ceiling. I've been wondering and doing a bit of research on applying hot water lines to the underside of the floor sheathing - particularly beneath the stone tiles that lay inside our front, and back doors and main floor bathroom that always feel cool to the foot. Even if we could raise their temperature by a few degrees, it would go far to take the edge off. In my mind a simple recirculating pump and a couple of zone valves could control the flow of hot water from the hot water tank, out through the heating loops and back into the tank. I'm not sure if there would be any issues with doing this with shared domestic hot water, but I guess that's where further research comes into play. Ideally I'd like to do one zone for the front entrance, one for the main floor washroom, one for the kitchen / dining room and one for the back entrance, though it may be more difficult to get lines under the back entrance since part of the basement beneath it is finished. Eventually when we finish the rest of the basement, we'll probably (most likely) do some sort of in-floor heating. I've been in basements where this is the case and it's funny - the effect is so subtle you don't even notice it until you step into the furnace room or some part of the basement without in-floor heating you are suddenly (and startlingly) aware of the difference. For that project, I may even consider electric in-floor heating, thought I am loth to use electricity for heating anything but my tea since it is the most expensive form of heat available here, despite the convenience and ease of precise control.

15

V_Buscom / Buscom - The New Mod Shell

« Last post by Cube on February 05, 2018, 03:55:38 PM »

Years ago I started creating a Venturii module coined Modshell that was designed to be a generic code base from which to easily whip up new modules as I encountered additional systems to integrate Verturii with. Modshell has trailed behind the leading edge of development, though it's concept has recently been revitalized by the development of Buscom. Among other things, Buscom essentially accomplishes one of the same goals Modshell was supposed to.

Buscom is a generic Venturii module. Unlike every other Venturii module which was created to interface with a specific system or piece of hardware, Buscom has been designed to interface with interfaces, not devices. What that means is that Buscom includes all of the code necessary to set up the following connection types:

• TCP Server (Remote device connects to a listening TCP socket served by Buscom)
• TCP Client (Buscom connects to a listening TCP socket served by some external host
• UDP Server (Remote device connects to a listening UDP socket served by Buscom)
• UDP Client (Buscom connects to a listening UDP socket served by some external host
• Serial Port (Buscom connects to an external device over a native (16550 or similar based UART) or USB-based Serial Port
• (Unix) Named Pipe (Buscom connects to another service running on the same host that can communicate over a named pipe

Having established a connection to one or more of these connection types, data can be sent directly to and from that device through Decider. One of the first simple uses I had in mind for this was to set up a dot matrix printer that recorded a log of sorts, printing one line at a time as it was received from Decider. All Decider needs to do is generate a string of text and send it to the appropriate channel on the corresponding Buscom module, and that string will be delivered verbatim to that printer. This, as it turns out, has many other practical uses as well. It turns out that many systems utilize a simple yet powerful system for providing API's to their inner workings. This will actually be the basis for both the Pelco Endura integration and the Harding DXL Intercom System integration, both of which use simple, low level serial protocols for allowing external control of their workings.

Today I have begun returning my focus on Buscom, with the plan being to clean up it's code, run it through a series of tests (including both functional and performance & memory usage - IE: Valgrind) before I begin to fork this project and create a new one, the first child of Buscom, which will be a Modbus module for talking to RTU devices using the protocol already named.

16

Decider / Re: WebSockets JSON Communication Protocol

« Last post by Cube on February 05, 2018, 12:53:08 PM »

Alright, I have completed implementing the RegisterVUIDs and SetDVAL methods, though I am not sure what the best way to describe the protocol is, so here's my take at it with words:

In order to receive notifications of the current values of things, each user interface session must register the VINT Unique ID numbers (VUIDs) of each Value Interpreter they are interested in getting the current value of and any subsequent updates for the duration of the Websockets connection (session.) This is accomplished by sending one or more RegisterVUIDs JSON objects to the Websockets connection, with an array of integers as the value for the RegisterVUIDs object. For example:

Code: [Select]

{"RegisterVUIDs":[1,5,9]}
Decider will acknowledge this message by registering the current session as being interested in the given VINT values, and will send the current values and Device Status values of each VINT that was registered as JSON objects. Note: There is a stub function in Decider that will check the permission of the current user (once this feature is implemented) which could inhibit one or more VINTs from returning its status.

Code: [Select]

{"VintValues":[{"VUID":1,"DeviceStatus":1,"Value":1000}]}
{"VintValues":[{"VUID":5,"DeviceStatus":1,"Value":1000}]}
{"VintValues":[{"VUID":9,"DeviceStatus":1,"Value":1000}]}

Any subsequent updates to registered VINT values will be sent to the client as JSON objects in this fashion.  If the device that supplies the VINT value goes offline (Decider loses communication with the Venturii Module through which that device is communicating, or the Venturii Module itself loses communication with the physical device) the Device Status will change from 1 to 0 (offline) and the value should be considered at best "Last Known Value" or more accurately "Indeterminate" . It will be up to the user interface to decorate the representation of this data source in a manner suitably indicating its loss of communication.

SetDVAL works in a similar manner: If the client (usually at the direction of the user) wishes to change the value of any Value Interpreter, a SetDVAL JSON object must be created and sent to Decider. Multiple VINTs can have their desired values set in the same command, the SetDVAL method must contain an array of at least one object, and that object must be comprised of two members: A vuid member whose value is an integer representing the VUID number of the Value Interpreter to change the Desired Value of, and a value member whose value may be a double, a float, an integer or a string representation of any numeric format, and represents the value the user wishes to set this particular Value Interpreter to. For example:

Code: [Select]

{"SetDVAL":[{"vuid":1,"value":"1000"}]}
Decider will attempt to effect this change, but will only send a VintValues object if the value of the VINT actually changed from it's previous value.

17

Web-Based User Interface / Developing a new Web Interface Framework

« Last post by Cube on February 02, 2018, 10:15:28 AM »

One of my colleagues has been putting forth considerable effort on developing a new web-based user interface framework for Venturii. From what he has told me, it is built using Angular, though I am not familiar with this technology at the moment. Still, one of the things I love about working on Venturii is that it gives me time and cause to investigate things like Angular with a specific purpose and task in mind. This makes it much easier to focus one's attention to a task with real goals in mind; I find this approach much more productive than simply trying to work my way through hello world examples that are purely conceptual and have little application to anything I am working on. Hopefully this colleague shall join the forum here (he's complained about the site's certificate so I may need to get that fixed) but he brings a wealth of information, experience and value to this project and could contribute much to the development of Venturii.

18

Decider / Re: WebSockets JSON Communication Protocol

« Last post by Cube on February 02, 2018, 10:11:03 AM »

I have created a JSON test web interface, for easy testing of this protocol. The .php file itself can serve as a good example of how to write web apps that interface with Decider using the jsonii protocol, and it has been very responsive.

19

VSAC / Obsolete

« Last post by Cube on January 31, 2018, 01:26:08 PM »

This firmware, while once intended for a separate PCB design, has now been merged with the main VDAC code base. I believe myself and Mark are the only two people now with running VSAC boards... These will eventually get updated with VDAC firmware, which adds much functionality and requires no hardware changes.

20

Decider / Re: WebSockets JSON Communication Protocol

« Last post by Cube on January 31, 2018, 10:05:12 AM »

In the end I went with the correct option, solve the problem once and for all. I did not want to be bound to a "specific" configuration of the library, but considering that there was practically nothing turning up in my searches of the Internet, I came to the determination that the problem had to be in my code, the most likely of all scenarios. As it turned out, this was the case and it came down to my use of the LWS_SERVER_OPTION_ALLOW_NON_SSL_ON_SSL_PORT option when configuring the libwebsocket context. In 2.2, this did not cause an issue, and I recall using both http+ws:// and https+wss:// on the same port, for which (I believe) this option was necessary. Now, however, that may have changed but whatever the case - removing that option causes the library to run as intended and I can now move on to more interesting (and less frustrating) matters.