Topics

31

Energy Management & Monitoring / The Downside To Energy Monitoring and Management

« on: December 05, 2015, 12:54:46 PM »

I've been tracking my utility usage with Venturii for quite a few years now. At first it was merely an amusement to me, but as my budget grew tighter it became an invaluable tool for making effective changes to the way I consume energy. One of the greatest challenges we as consumers face when it comes to our energy consumption is that there are not a lot of convenient ways to monitor this information in a timely manner - most of us have no idea how much energy or water we've used until we get our bill in the mail a month later, and by that time several weeks have already past since that data was collected. This cumulative total every 30 days gives only the broadest glimpse of our utility consumption patterns, and provides almost no information about influences like "How cold was it outside? How many hours of sunshine did I have to compensate with artificial light? How many times and/or how long did I run the stove, dryer, furnace, etc. this month?" Venturii's real-time feedback on your consumption at any given point allows you to walk around your house and turn on or off appliances, lights, or devices and see right then and there how much of a difference it makes to your power consumption.

This kind of feedback can be invaluable for determining where your biggest consumers of electricity, gas and water are around your property, but there is a downside. As we enter the Christmas season, we've set up our Christmas tree. Last year we decided to move away from buying a real tree every year for a number of reasons, not the least of which was the fire hazard the tree posed by the time we actually got to Christmas. Not wanting to decrease the appeal, we splurged a bit and got a very nice fake tree that came pre-lit with 1,300 miniature lights. It looks amazing. However, due to Venturii I know that it draws 470 watts of power, and makes a considerable blip on my power consumption graphs. Therefore, illuminating the Christmas tree is a bittersweet event - it looks beautiful, but I know exactly how much it costs to run.

32

Web-Based User Interface / Report Groups

« on: November 13, 2015, 08:48:34 PM »

One of the things I would like to add to the web based user interface in the reports section is the ability to pre-configure multiple reports (graphs) that can all be run concurrently over the same date / time range. For example, every day I review the power, water and gas consumption of the previous day and/or that day up until [now], the temperature readings around the house & garage, salt levels in my water softener, Solar system power collection and discharge rates, etc. Currently I keep these reports all open in different browser tabs and refresh each one every day. It would be great if I could configure a set of reports that all run at the same time in the same window, showing me all the days' activity in a single report.

33

V_ION / Insteon Module

« on: October 17, 2015, 09:17:11 AM »

Years ago, in my last house, I had purchased and installed a number of X10 switches. They were cheap, and didn't work half bad. I wrote the "ecksten" module which phonetically reads "X10" and had incorporated the ability to turn on or off the X10 devices in my house on a variety of events (alarm system arming, for example, turned off all the lights) and even from the web interface I had developed at the time. However they had two major problems:

• Commands sent to devices did not always execute.
• There was no way to query devices, so manual status changes could not be reflected in the web interface.

I'd even add a third point being that they were very slow to respond when they did. Each command took one full second to execute, when it did. At one point I had the ecksten module send periodic "refresh" commands to the devices, so that if the first command failed the second might succeed. However this meant that manual switch control was virtually impossible (and frustrating) as it would eventually be overridden by whatever state Venturii thought the light should be in.

I began to look for a better system, and found a couple of options. There was actually quite a number of alternate systems for home lighting and appliance control, but the one that won out based on feature set, functionality, price and availability was the Insteon family of products from Smarthome. (www.smarthome.com)

When I moved out of that house, one of the first investments I made in my new home was a complete retrofit of every switch and dimmer in the house with Insteon devices from Smarthome. Although fully functional on their own, the primary purpose of their deployment was the ability to control them from Venturii, which meant writing a whole new module to talk to the devices. That module is the Venturii ION module.

To date it supports all devices in the Switched and Dimmable lighting categories, including linking and un-linking devices. It has the ability (in beta form) to read memory from devices and interpret the link tables. And it can receive commands from devices (status updates) so that Venturii can respond to manual activations of Insteon devices and use them as event generators.

If any or all of that interests you, this is the place to discuss it.

34

Decider / Committed New Revision of Decider & VLib

« on: October 16, 2015, 05:44:35 PM »

I committed a number of updates to the VLIB and Decider repos today, which should help in the following areas:

- Changed the way send() is called, preventing SIGPIPE signals from being emitted. This should let the send() call handle broken pipes by returning 0, which is handled in the code and should trigger a proper shutdown of the socket and a reconnection.
- Changed the way the formula processing function handles registration of interested parties of VINTs, specifically adding a parameter to request a specific trigger type when linking. This was necessary in the case of Torx pegs that were registering themselves as being interested in VINTs used for Conditions and/or Inhibitors. The problem that was happening was that because they were registered as Refresh On Change (the old default) if a CONDI VINT was changed, the peg would get refreshed, causing it to execute it's commands if CONDI was met. In some cases, this would cause a peg to get re-invoked MANY times, even though it should not. Now TORX pegs register themselves with the TRIGGER_NONE type, so that it can check the value of the given VINT in CONDI at the time of peg invocation, if CONDI is met [at that time] the commands will execute, but if not then nothing further will happen until the next scheduled invocation of that peg.
- Tweaked the keepalive parameters in the Websockets part of the code. I looked at including a heartbeat system in websockets to send, say, the current unix timestamp at regular intervals to keep the connection open if there is no VINT updates or ui commands being transferred. Previously it would simply disconnect the idle socket. Even as I write this however, I can see that it did not appear to have the desired effect and the clients still disconnect after a period of inactivity. I guess a proper heartbeat will be required after all, though maybe it can be initiated from the client end. Clients can "request" the current time and Decider will respond, confirming bidirectional communication is functional in the socket.

35

V_ECKSTEN / Legacy X10 Support

« on: October 12, 2015, 05:40:58 PM »

Ecksten was an X10 control module for a much older version of Venturii. It's code base has fallen by the wayside, primarily because I don't have any X10 devices in my house anymore. I've since upgraded to Insteon-based devices, and will never go back. However, if anyone has an interest in using X10 devices, let me know and I'll bring the module back into the current code base and library system.

36

V_DAC / The Venturii Data Acquisition and Control (VDAC) System

« on: October 12, 2015, 05:39:06 PM »

When I started Venturii, I did so not because there weren't other integration systems out there, but because I wanted to see if I could build one myself. I'm pretty proud of what I've accomplished so far, and the one area that probably has the most excitement for me now is this one. Early versions of Venturii were software based solutions that interfaced with hardware made by other companies. In some cases I wrote code based on published APIs, but in some cases I had to debug and distill their communications protocols on my own. This was exciting, challenging, mentally stimulating work and brought with it a greater appreciation for these products, their features, and even in some cases, some un-exploited features. I began to understand elements of the native software, particularly limitations of it, that were actually imposed by the hardware. However, as my interest in the field extends far beyond software, I started to look into the possibility of developing my own hardware for Venturii.

Some of this was out of necessity, for I wanted to connect to sensors that I knew of no other off the shelf hardware platform that integrated with them, some of it was cost driven, and some of it was simply curiosity. Can I do this? To date I've developed two different types of relay control systems, one for traditional "ice-cube" style relays and one for latching relays. Why? Because I had them and had no other way to make use of them. As it is, they now run a whole whack of lights in my garage - just because they can. The NIM-1, discussed elsewhere, will be the next PCB to be tested, but I've also been working on some proto-board-and-wire based prototypes of other VDAC-based devices. It is so much fun for me to create these things that talk to different hardware, adding the ability to collect interesting data into the mix, and also to create circuit boards with specific tasks in mind that hopefully others may find useful as well. Really, I'm building Venturii for me, but if others can use it and learn from it and improve their lives, houses, buildings, surroundings, etc. with it, then I am more than happy with that.

In short, VDAC is the hardware branch of Venturii, native hardware developed specifically for Venturii and designed to talk to a wide variety of sensors, devices, and systems, as well as to control them. It is all open source, the hardware designs are available to download as is the firmware for them. If you can improve on my designs, (and believe me - there is lots of room for improvement!) I welcome all the help anyone is willing to offer, and I hope that many people will find these ideas and creations useful. Of course, to help fund my R&D and coffee, I'm also selling complete units for those who prefer a more turnkey approach. I work in a day job where every major player keeps their cards close to their chests and there is very little to no innovation encouraged where the products we sell are concerned. These large companies have grown their empires by producing proprietary systems that work within their own ecosystems very nicely but generally do not play well with others. And then those companies hold you over the barrel with recurring licensing costs! You buy their products, then you have to pay for licenses to use those products, and have to pay annual license costs to maintain support for those products. I don't want to be like that. I like collaboration and structured community contributions to make the products better. Every time I've installed some part of Venturii in someone else's house, they've come up with uses for it that I would have never thought of, and this has lead to new features, improvements and design changes that have made Venturii even better. I'm really hoping that can be extended to the hardware as well, so let's talk about it here.

37

V_BURG / The Venturii Burg Interface

« on: October 12, 2015, 05:22:02 PM »

Years ago I wrote a module for interfacing with the DSC Maxsys series of intrusion alarms - primarily because I had one in my house. It supported the full command set of the PC4108 (if memory serves) serial interface, providing Venturii with interpretations of all the status codes, ready state, armed/disarmed states, zone states, trouble & supervisory conditions, etc - and allowed arming and disarming of partitions and control of PGM outputs from Venturii also. In fact, my first underground sprinkler control was accomplished with the help of PC4204 relay modules. Funny story, because this was their original task, all four relays were set up on schedules within the Maxsys. Later, when I moved control of the sprinklers onto a different system, I re-purposed one of the relay modules as a keybus expander in the garage. Every night at 4:00 am, the system would go into trouble. A sound sleeper, I never noticed this but my roommate who lived upstairs and slept much closer to a keypad had smashed one right off the wall because it kept driving him nuts every night. As it turned out, the first relay is set up on the expander as a keybus power interrupter to allow com bus resets to take place remotely by pulsing that relay. I had wired the garage combus through relay 1 on the PC4204, but forgot it was still on a schedule. Every night at 4:00 am the 4204 would disconnect the keypad power from the garage keypad, putting the system into trouble and waking my roommate Bo up to an infuriating beep-beep every 20 seconds. Of course, the power would be restored 30 minutes later and I never bothered to check the event logs. It wasn't until I noticed the front keypad all smashed up that I asked him about it and began to investigate.

I've since moved and the code for the module that interacted with the Maxsys has fallen MANY versions of Venturii behind. However, with Mark having a Honeywell alarm in his house and me having a newer DSC PowerSeries model, there is renewed interest in reviving this module and bringing this data back into the system. This time what I'd like to do is create a single module that can talk to as many alarm systems as we can find interfaces for, thus reaching a much greater audience with less coding. Mark's even got an interface module that talks both DSC and Honeywell, changeable by dip switch I believe, which simplifies things even further. It's all time. Right now this is the next module to be (re)written on the road map for Venturii.  Any thoughts, ideas, suggestions, or volunteers to help develop and/or test and/or document this module are welcome.

38

V_ASCII / The Venturii ASCII Module

« on: October 12, 2015, 05:11:53 PM »

Technically, this module is still a concept. The idea being that it will become a standard Venturii module that can connect to any socket or serial port, and provide a bidirectional path for plain text and/or hex slash binary communication to a thing so connected. My first idea for this module is to use it to send PTZ commands to my video surveillance system such as instructing a nearby PTZ camera to go to a specific preset when the doorbell rings or at certain times of day. Of course this would not be limited to PTZ control, in fact it wouldn't even be geared to this task specifically. It could also parse data from other systems, perhaps even house vints and read data from serial or IP streams and convert the values into VINTs. If you have ideas or suggestions or feature requests for such a function, please let me know here.

39

V_LIBS / Venturii Libraries

« on: October 12, 2015, 05:08:33 PM »

The Venturii libraries contain much of the core functions used by most - if not all - of the Venturii modules and Decider.

40

Electronics / AD7705 16-Bit Analog to Digital (ADC) Converter Hanging After Seconds-To-Minutes

« on: October 10, 2015, 11:21:14 PM »

Background / Preamble:

I'm working on a prototype PCB involving an AD7705 Analog to Digital converter, which I have been wrestling with over the past couple of hours. For my testing purposes, I've soldered together a little prototype PCB using socketed components and quite a few wires, housing the basic components. Some of the pieces, including the AD7705 are on external PCB's, as they come from Aliexpress that way. I've put together 6x header blocks on my PCB to connect to the various off-board modules in a star configuration using standard 4-pin connectors on each. This makes for some clean connections, and a fairly presentable package all things considered, as I will need to mount this thing in my house once I finish the hardware and software in the garage and begin collecting data and tuning all the working parameters before digesting all things learned and producing a PCB that has all these things onboard itself.

The prototype as it is uses an Arduino Pro Mini (one of the knockoffs I'm sure) and has a 2x16 LCD display on the I2C bus, the AD7705 on the SPI bus, a couple of LEDs for realtime feedback, and a TTL-to-USB adapter for updates and Serial Port monitoring. I have an Infrared sensor monitoring ambient IR levels using the Atmel 328's onboard ADC, and I'm feeding the whole assembly power off a regulated power supply with onboard digital voltage readout, dialed in at 5.1Vdc. When it starts up, the LCD initializes and displays the values from the AD7705 and the IR sensor (basically values from both ADC's oversampled 15x.) ... and nothing else.

Present Problem:

It works for several seconds to several minutes, and then the AD7705 reports a value of 512 and stops updating. I connected a wire to the Data Ready pin on the AD7705 board and mirror that pin on one of the status LED's. When things are working, that LED flashes rapidly. When it stops, the LED is extinguished, which means that the Data Ready pin is low. If I hit Reset on the MCU, it starts up and everything works again, but inevitably the AD7705 stops working after a number of seconds. During this time, the readouts from the onboard ADC continue on the display, so the MCU itself is not hung, I just do not know what is going on with the AD7705.

This is probably one of the many instances where an oscilloscope would be very useful, but I am left with print() and digitalWrite() statements in the meantime. Actually, one other thing I can test is that I have another AD7705 board, I will swap them and see if the other one does the same thing. I noticed that the Analog Voltage Reference IC on this one looks like it wasn't sitting properly when it got soldered and I think there are gaps between the pads and a few of the pins on it. Begin troubleshooting...

It's funny, these projects always get snagged in the wrong places. For example, I expected trouble getting it working with a different Chip Select pin, but once that was resolved and I started getting data from it, I thought I was out of the woods. Then I encountered seemingly high amounts of noise, so knocked a few bits of the ADC off (it is a 16-bit converter so fairly high precision, most of which is probably not necessary for my application.) Got the readings pretty stable and thought I was out of the woods again, ready to write [proper] firmware to begin interfacing this with the Venturii VDAC module - only to discover that the external ADC seems unstable. I don't want to move on from here until I have a proven platform I can fall back on in case the Venturii firmware is buggy, but now I am left troubleshooting something else I had not anticipated. So it goes...

41

Sensors / AD7705 16-Bit Analog to Digital (ADC) Converter

« on: October 10, 2015, 12:12:20 PM »

I've been scratching my head on this one for quite some time, and maybe someone can shed some light on this. I have an AD7705 16-Bit BiPolar Dual Channel Analog to Digital converter, which I am trying to incorporate into a prototype that I am building as part of my water meter project. Initial prototype on breadboard worked wonderfully, and there are posts about it on www.venturii.net. I've been building a perf board version, to make the circuitry a little more firm and to incorporate numerous other circuits and supporting gear as a firmware-proving platform. Out of curiosity, rather than any sort of necessity, I moved the chip select pin off of pin 10 and onto pin 4. In the AD7705 library there are static declarations for MISO, MOSI, CLK and Chip Select pins, and I changed the Chip Select pin designation from 10 to 4. No go. No matter what I tried, it would not work in this configuration, but if I choose pin 10 (and adjust wiring accordingly) it works perfectly. I've read posts on the Arduino forum and others where people have insisted that they've been able to get this to work, some claiming that you have to set the pin as an output before calling the SPI functions, which this library does, yet no matter what I've tried, it does not work on anything but pin 10.

As I said, this is fine on this project, I can use pin 10, I am just wondering if there is some way to use other pins for chip select (and have it actually work) as I've never been able to accomplish this seemingly simple and documented possible task.

42

VDAC-NIM-1 / Pre-Prototype Prototype Built

« on: October 02, 2015, 10:35:11 PM »

So before I put through a $400 PCB order, I wanted to "pre-test" a few of the circuit designs that I put into the NIM-1. It's simple things, but I want to make sure that everything will work "the way I think it will" before making such a commitment and investment. This also forces advancement of the VDAC firmware, which will form the heart of the NIM-1. I am also pretty excited to "announce" that I may have found a way to incorporate OWI code into VDAC in such a way that the other OBD types will not get starved for attention and potentially miss important updates.

The first update to the VDAC firmware will actually be a pretty major one; implementation of what I've been calling the "Multi-Drop Firmware" to allow the creation and deployment of [smaller|purpose-built|multiple] VDAC boards together on the same 4-wire bus (GND, +12VDC, RS485A, RS485B). I wrestled back and forth with the idea of making every single device Ethernet / WiFi enabled, and that is still an option where the application warrants it, but there are certainly places throughout my house where it makes far more sense to simply run a 4-wire bus between multiple modules all tied back to a single NIM or even directly into a USB port or 485-Ethernet/WiFi adapter.

I've ordered quite a few parts from China today to begin testing specific circuits that form the NIM-1. My goal is to prove each circuit before the PCB run as it is quite expense and I've already caught a number of glitches / mistakes / omissions / problems through this process. It may even inspire me to add support for a number of other devices via pin headers for future expansion or user options. One such idea is the incorporation of a display port, for something like a 2x16 or 4x20 LCD display, or maybe even something simpler like one of those TM1638-based LED/Button interfaces; the possibilities are endless. I love this stuff!

43

General Discussion / Minigroup Shutting Down

« on: October 02, 2015, 10:04:27 AM »

I received an email from Minigroup yesterday announcing that they will be shutting their service down at the end of October. We had been using Minigroup for Venturii discussions, among other things, over the past couple of years and it was a decent service. There were a few things that it lacked, (Two come to mind off the top of my head: 1. Their message editor did not have the ability to preview messages before you posted them, and 2. Only the first message in a conversation thread would display images - all comments below it did not and I don't think there was a way to [force] it to either.)

I'd like to grow this forum into a bustling place to discuss automation and integration topics. So please, tell your friends, ask your questions and let's get this automation party started!

John

44

Sensors / Preliminary Test Results For Using a Hall Effect Sensor to "Read" Water Meter

« on: October 01, 2015, 05:37:09 PM »

Last night I confirmed a very exciting theory. Using a simple SS49E Hall Effect Sensor taped to the body of my Badger Water Meter, I was able to capture VERY detailed flow data as the magnets within the body of the water meter rotated proportionally to the amount of water flowing through it. For example, here is what I read from the sensor when using a very small amount of water:



And here is what I read from the SS49E sensor while using a fairly moderate amount of water:



The sensor itself came to me as part of an assorted sensor kit purchased from AliExpress:



Data was captured using an AD7705 16-Bit BiPolar Analog-To-Digital (ADC) Converter with a 500Hz sample rate. I will have to incorporate support for this ADC board into the VDAC Firmware as I have a few other planned uses for it on other projects. It certainly performed VERY nicely in this little experiment, and I am very excited to begin using it for other data capture application in the near future.

A much more detailed writeup on this experiment, including some of the history behind it, can be read on the Venturii Adventure Blog at http://www.venturii.net/blog/2015/10/01/measuring-water-magnetically/

- John

45

Decider / New Feature - TORX Time Definitions

« on: September 26, 2015, 06:18:53 PM »

Decider 0.5.0-459 marks a key new feature, one which has long been on my wish list. It gives Decider a number of new capabilities for scheduling things to happen, and it also incorporated an idea I've been meaning to implement for years. In order to appreciate the Torx Timer, you have to understand the history. For that, let me begin with a story.

When I was a little boy, my father and I used to collect stuff from commercial and industrial bins around the city. You simply would not believe some of the items that people threw out in those days, and my collection quickly grew to include all manner of security systems, built-in vacuum cleaner parts, intercom systems, cameras, sensors, motors, switches, speakers, transformers, tools, amplifiers, sprinklers, light fixtures, humidifiers (both ultrasonic and steam) - the list was nearly without end. Some things needed minor repair, but most worked as we found them. One such find was a brand new Tork 7102 mechanical timer. It had never even been powered up, nor had any of the knockouts been punched. It was brand new, in it's box.



Anyone who's ever managed a building has seen or set one of these things, knows of their simplicity and reliability. I put a cord and a plug on mine so that I could use it to control the operation of things around the house, and it's probably still in one of my dad's sheds to this day. I'd even reckon that were I to find it, it would probably still work. You control the load by screwing little Pegs onto a wheel with markings indicating time of day within a 24 hour period. White pegs switch the load on, black pegs switch the load off. A special copper wheel (to the bottom-left of the main wheel) allows you to install or remove additional pegs to specify days of week for operation. A simple description of what it could do might be written like this: Turn the load on at 7:00 PM and off at 6:00 AM, Monday through Friday.

Up until now, all of the time definitions within Venturii have been stateful. In other words, they have a start and an end point in time. Day of Week has a Value Interpreter that becomes a 1 at midnight on any day that is selected in it's configuration, and becomes a 0 at midnight on any days NOT in it's configuration. Hour periods have a start and end time, as do Month of Year, Absolute Time Periods, etc. While excellent at making sure plants are watered for 30 minutes every morning at 4:00 on Mondays, Thursdays and Saturdays, these time definitions do not allow you to configure something that happens every ten minutes. They also require something like a comparison to attach commands to them. The comparison evaluates the value of the time period, and then executes different commands based on whether that time period is true or not. I have an amplifier for my whole home audio system that I'd like to make sure gets turned off every day. Everyone in the house knows how to turn it on when they want to listen to music, but no one remembers to turn it off. I don't want to turn it on automatically, but I would like to turn it off automatically as it draws half an amp (~60 watts) when idle. I could set up an hour period and a comparison with a command that executes only when the time period becomes true to turn the amp off at 11:00 pm every night, but it feels a bit clumsy.

Perhaps the biggest motivation for creating the Torx timer has, like all good features, been invented to solve a problem. We have this requirement at work to sign in and out every day by sending an email to the office, telling them where you're working for the day so they know where to start looking if you don't make it home. It's a good idea in theory, however if you haven't signed in by 9:00 am, and haven't signed out by 4:45 pm, the office staff send an email to your supervisor tattling on you for your negligence. The problem I've had with this system is that I work 9-5 or later, sometimes 6 or 7. What's worse is that I'm often IN the office, around a glass wall from the people tasked with policing the sign in board. In latter weeks they've grown accustomed to walking around that corner and seeing if I'm still at the office, but still sign me out regardless of whether I'm done working for the day or not. That system is completely broken. What I've been doing is using Venturii to send my sign in and out emails to the office, which has saved me no end of grief (except one time when I forgot to inhibit it on a long weekend. Oops! haha) Even so, I wondered if someone might notice that my messages to sign in and out came at [exactly] the same time, every day.

A summary of the features of the new Torx Timer system in Decider is this:

• 24-hour Timing Cycle, perfect for things that need to happen at certain times each day.
• A single timer can have an unlimited number of Pegs.
• Each Peg has a fixed time, with microsecond resolution.
• Each Peg can have a repeat interval, and repeat limit, allowing you to execute that peg's commands at recurring intervals in a single construct.
• Each Peg can randomly deviate up to a set number of seconds before and/or after the set time, solving the Home Alone problem.
• Each Peg can have an unlimited number of commands to execute.
• Each Peg uses the Condition / Inhibitor system for allowing / blocking execution based on other Value Interpreter values.
• The commands in each peg can be manually executed at the command prompt or from within other objects in Decider.

The Random Deviation functionality has been an idea I've been toying with for a long time, which may see itself emerge in some of the other time definition constructs as well. It solves what I call the Home Alone problem with traditional timers. If you remember the movie, the burglars had memorized when the lights in each house turned on, confirming the vacancies of those premises. By incorporating a random deviation to the set time of any given Peg, you can obtain a more unpredictable result while still maintaining the convenience of automation. Random Deviation has two parameters, number of seconds before the peg time and number of seconds after the peg time to randomly deviate. This allows you to constrain the behavior to something like "Turn on the lights AFTER 6:00 PM but before 7:00 PM." The CONDI integration allow each peg's command execution to require meeting a set of conditions and clear a list of inhibitors, giving you the ability to constrain operation to certain days of the week, times during the year, or only if any other Value Interpreter is a certain value or not a certain value. The repeat feature provides an easy way to do things at regular intervals like resetting counters or sending status emails.

As with many of the features I've come up with, I can't wait to see how other people find interesting uses for them that I had never even considered.