Development progress update (Utility measuring)

Marcel did not had a lot of time the last couple of months which put a little hold on device development. Two weeks ago we finally sat together again and where thinking about quick and dirty devices to develop. We got a couple of ideas like specific simple devices like light intensity measurements, temperature readings, and last but not least, utility usage measuring. He decided to firstly go for the utility measurement, and thinking this was a very good idea, i followed this idea.



We both have analog meters which measure electricity, gas and water usage. This means we have rotation disks which are reflective at the moment a specific amount of usage has passed (bu our electric disks are just the opposite of this). So we will be using Optical detector/Phototransistor‘s for capturing the “pulses”. We decided to use I²C for reading an Arduino  which has the sensors connected. But because of the I²C interface on the Raspberry Pi is master only we can only read the amount of “pulses” when the pi initiates this. So the software on the Arduino will be counting pulses, and when read from the pi, reset it’s internal counters for depending on the sensor data which is requested. So this is what we are going to do. P.S. There is a big change it will be go to an attiny, depending on our personal needs.


Next to the hardware is of course the software needed to read these data in the server. Well, the device software is easy does it with the already done I²C driver interface. The question is now, am i going to use this device to also do the calculations concerning transforming these pulses to wattages, kwh, liters/gallons and gas m³ or kwh. We deciced it would be better to use a plugin for this. Well here are the catches coming.

The decision to create a plugin also means we should not only support a “every so much time read amount of pulses”, but also an “accept every pulse instantly” method. Also not all the meters are the same in measurement calculations. Some use so many rotations on a analog device is on kW/h, others use kh as measurement. And we are not done yet, there are differences in calculating gas usage, some use so much kW/h used and others use m³. And of course there are liters and gallons. And still we are not done.

When having a plugin you should keep users as happy as possible to deliver them options, but not too much or, even worse, too less. One of the features we thought off in the preparation is: “On what kind of information do we make suggestions on what is good usage, over usage or just: “Damn, that’s a lot!””. And this is something we/I still are not thought out of yet. Do we let users fill in they’re usage of last year and create a day normal, high, a lot, scale or based on the amount of house members? House members is the easiest way for everyone, but filling in amounts give more precise control because some use gas for heating, and some air coolers. When filling in the amount of house members there will be unfair messages, but using absolute data gives complexity.

So when the plugin comes available it will support continuous and timed pulse amount readings, and users will need to fill out they’re last years usage (or an user targeted usage) to create a fair usage scale. Costs calculations will not yet be in the first version of the plugin but will certainly come. Also the plugin will be able to hook to any device the user wants which measures usages pulses.

Current work in progress


Currently we are using an Arduino board with an Atmega328p to do the measurements which counts the amount of pulses, and the server reads the device every minute. Below pictures of the sensors and current setup

Analog readers

Analog readers

The above shows the sensors we are using to detect the dark spot on the electricity meters, and the below photo shows the current test setup

Current power utility usage test setup.

Current power utility usage test setup.

The above photo shows the Arduino used with three leds. The red led lids up as long as the non reflective part of the disk of the analog electricity usage is in front of the sensor. This makes it very easy to debug any reading errors. The yellow and blue do the exact opposite. They lit up when the reflectors come before the sensor and respectively for gas and blue for water usage. The white wire are 16 meters of I²C and the black ones are going to the sensors.


Currently the software can be configured to select a specific device and which sensor of that device is reading the specific utility usage, and what calculation has to be applied. Currenltly only analog disks are supported. See the screenshot below

Unfinished utility settings page

Unfinished utility settings page

The interface is still unfinished because some fields are missing like last years/goal measurements. And also high/low usage times and prices. Some extra fields will come where you fill in kwh/kh and liters/gallons. Gas will stay m³ but a possibility to supply a kW/h measurement unit is being thought of.

Current tests show a nice flow in usages which are being made with this plugin and is also already being graphed on the server’s web interface as seen below

Utility views

Utility views

The above is the beginning of the utility usage views in the web interface. IT supports real time gauge updates. The graphs can show the history for a day, week, month and year. More graphs and information will ofcourse follow as the plugin configuration get’s shape so more information can be shown.

In line with the above is the development of the client which of course should display this data. The screenshot below shows the clients progress

Utility gauges and little graphic updates

Utility gauges and little graphic updates

This screenshot is taken at a different moment, but shows the gauges of the current usages. Also in the client it will be possible to attach a gauge to the top bar. All the gauges are also in real time updated, and will of course be extended with graphs and information as soon it is available. The current shows work in progress and this was a first test with gauges in the upper task bar. Clicking on the task-bar widget opens up the bigger gauge.


The above is the current work in progress and when any feature has been build in it will be uploaded to the alpha version on the download page, and in between hot builds are available on the build server. Be aware that the build server is not always available because it is a workstation. I hop the above shines a little light on what we are doing right now. If you have any suggestion please let us know in the form field below, on twitter or facebook. The latter two will show more news and screenshots.

Hardware source code

Of course when stable, the source code for the hardware used will be available on bitbucket for download.

I’m tired, good night,

Web interface overhaul is done (including a change log).

Finally, i have completely replaced the old web interface with a new one. This was quite a lot of work because i did not only replaced current functionalities but while I was replacing stuff, a lot of new functionalities have been added and some nasty little beasts have been squashed. A very imporant notice is that the new interface is focused on HTML 5 and CSS 3. This means that older browsers are not supported (as the old interface did not support new browsers).

Browser support listing should be:

  • Internet Explorer: 9+,
  • Chrome: current,
  • Opera: current,
  • Safari: current,
  • Firefox: current.

In other words, JQuery 2.0 compatible browsers.

Some examples of additions that have been done:

  • Server localization has been added with modifiable timezone settings which includes a map you can use to just click on the exact location of your server.
  • Server state (what’s in a name) has been replaced with day parts and presence properties,
  • Graphs,
  • Initial tablet and phone interfaces,
  • A real time web interface, without refreshing pages the last known device/system/presence/etc.. is known.
  • A bunch load of RPC methods have been added (will be added to the manual pages asap).
  • Added floor locations to the locations itself.

Some examples of changes:

  • A far more faster web interface,
  • The trigger and macro system has been redesigned from the ground up and are by far more faster,

The above are just some examples, The list of changes is far more greater, The full change log is below: Read more of this post

How the trigger system works

Hi all,

This will be a quick post about the trigger system introduced in the new web interface. The old interface already had a trigger system but this was restricted to when device this has that value do a device command. There was also no easy way to follow the rule steps.

Below is a screenshot of the new trigger system where the rest of the post will try to explain it and also talks about how the system works.

An example trigger entry

An example trigger entry

Read more of this post

Web interface overhaul progress update

Hi all, I thought it was time to post a little web interface overhaul update. It is not going as fast as i hoped it would go, but some personal things got in the way and of course i have a family to take care of.

The changes being made are a bit more extensive then only the web design overhaul. This big change graphical change also caused some other stuff to change within the server code, add more RPC methods so others can also completely control the server with they’re own interface (if there is a need for it).

Overhauling and adding

Not only is the web interface getting an overhaul but in sync with the web interface there will also be a web interface for mobile devices. So there will be three interfaces in total. The web interface overhauled and a new interface for mobile phone and tablet devices. The main difference between the browser and the mobile versions is that the mobile version will only be able to control devices, media, etc. and the web interface does the maintenance next to the controlling.

Noticeable features added

Some noticeable new features are:

  1. An easier method to set the location of the server/your home.
    The server uses latitude and longitude to calculate sunrise and sunset. The web interface now has the possibility to ask the browser for location data which is reflected on a map. On this map you can adjust your location to exactly pinpoint where your home/server is. (This information is never ever or whatever send to any or whatever some online service or something else you can think of but stored in a local configuration file).
  2. A visual trigger editor. Triggers are used with input from devices which can, if a certain condition is met, execute macro’s or device commands.
    Here you can add/edit triggers with just some mouse clicks and you can exactly see how the trigger works.
  3. Real time web interface updates.
    The web interface in the browser as well on mobile devices supports web sockets. This means that if you and someone else are looking at the same device where one is on the computer and the other on a tablet. And the tablet user for example switches a light on, the web interface user will see it change on the web page without refreshing. This way you are always sure you see the latest device statuses.
  4. Graphs have been added for devices which have a numeric output.
    When a device is configured to have a numeric output which is or logarithmic or series the web interface will notice the user of the availability of a specific graph.

We are also posting screenshots of pages which are done on the web screenshots page.

Current change-log

Below is a list of changes currently implemented in the latest alpha release with the exception of the graphical trigger editor and not all the RPC changes are included. This list is a list i keep privately so i know what i have done next to the todo list. Because we occasionally renew the alpha release the current release shows two different web interfaces in the browser. If a page has not been updated you will be redirected to the old page and vice verse. Also the web sockets on phones and tablets is currently not supported in the default android browser because this browser has no native web sockets support.

Read more of this post

Development progress update

Hi all, welcome to this quick progress update post. As in an earlier post (Upcoming updates/changes in the server and client) i have posted some trivial and critical changes to be made to the server and client. Below is a list of things that have changed/modified or added:

Server changes:

  1. All client messaging is now done via json,
  2. Added a http json entry point (http port),
  3. Added a websocket service (http ws port),
  4. Added a settings page where client approval/debug mode can be toggled,
  5. Added the possibility to change the used timezone via the web interface,
  6. Internal changes to client handling,
  7. Added support for XBMC media players.

Client changes:

  1. We have updated the client to support all the server’s to clients exposed services:
    1. Plugins like the media plugin,
    2. Messaging between clients.
  2. The client overhauled to do full json rpc messaging
  3. A lot of internal client changes:
    1. Windows where images, are now dynamic,
    2. Better touch support (including bigger touch points),
    3. Some speed and memory improvements,
    4. Better messaging if the client is approved, denied by the server,
    5. Client now honors the server time, instead of the time of the host computer.
  4. And some improvements which causes us to develop new features faster and better.

The above changes took a while to implement, especially the JSON-RPC engine and the adding of the media plugin took more time then i anticipated.

Upcoming changes

Below i reflect the next upcoming changes and added features in order of planning.

System state

While the above changes are done, some changes have not made the previous post but are quite important to change as early as possible. When i started this project i made a little thinking error. I created server states which where relying on the time of day. This in our opinion is not the way to go. A state based on a time of day can not include (as designed now) the user presence, which of course is a big thing in home automation.

The change we will be making is that system states will be migrated to user presence like “At home”, “Away”, “Vacation”, etc. and the time of day/day parts will be a server property.

Web interface

This is a big hog. When i started with the web interface (which was meant as just a prototype, allowing quick and dirty actions) i only used Chrome for my testing (because of the integrated tools etc.). This caused programming which do not comply to all modern browsers to be supported. The biggest example would be the usage of an older jquery version causing problems in modern browsers. Also the web interface is not able to real time reflect changes happening in the background (for example device value changes or an other media item playing). And because the trigger and device xml pages are quite complex we need to change this appearance so the web pages should expose an easier to understand interface.

Supporting only one browser we can be sure of out of the box is a big no no, so a web interface overhaul is planned earlier then expected. So now having websockets available the web interface will be changed to html5.

Hardware support

The server internals do already support a couple of things like i2c, Arduino based boards via xml(untested), universal com (via usb) devices (untested), raspberry pi serial pins (deactivated) and HID devices (not complete). But not all these are exposed an should be finished and made available.

I was planning to add Modbus support immediately after adding the media plugin. The above paragraphs are having a larger and more complex impact on system internals and user interaction, so these have been pushed up into the list. The Modbus support will for example relies on the universal comport interface just like the arduino based boards. We need this interface because there are a lot of Modbus controllers out there.

Mobile device support

Now that the client core is getting to a point where we want it to be, we are looking in how to implement this core with Android. The core is about receiving broadcasts from the server indicating presence and authentication. It will be likely that the there will be an html5 version created for displaying on mobile devices in sync with the overhaul of the web management interface.


There are a lot of priority changes made, but we think this is for the better, improving the basics before adding new features. The above changes are the changes/additions we will be working on in order from now on (besides solving bugs if found and the mobile support).

So until the web interface has been updated, only use chrome for the best results.

PiDome Relay board


It’s been a while since I had my last posting here. I’ve been very busy with my work and other stuff. I will try to get more postings and hardware added to the system in the near future. This is just a small posting as what this board is about and an in-depth description of the technical stuff will be added soon.

As you may have seen we can now make our own PCB’s for our projects and the first one that we created was for the relay board. It is a board that has two switching relays and is controlled through USB, but without the need to have a dedicated USB-chip onboard. Next to that there is the possibility to connect up to three sensors to the board.

I have created this board in order to make it possible for me to switch my heating system on and off, but of course you can use it for any purpose, like lighting, as the relays used are capable of switching high voltage (7A at 230V).

USB and some sensors

The whole USB-part of the device relies on the HID-protocol (Human Interface Device) which is widely used in simple computer products like keyboards and mice. We choose for this because nowadays a lot of devices have an USB-connection and our board itself consumes very little power that we can take from the USB-connector, besides that it is relatively easy to implement this software-wise. I’ve used to HID-Serial library (from Ray Wang at that he adapted from V-USB for the Arduino-platform.
As identification of the device relies on ID’s (Vendor- and Device-ID) there is a small catch. Because we are too small and don’t have the resources to get our own ID’s we needed to find a solution for this. Luckily there are some people out there that are willing to share their ID’s with others. The only thing that we need to do is comply with the rules that they have set up for this, which aren’t that harsh. In the library-software you can set your Vendor-ID and Device-ID (in our case of, which is from the nice guy sharing his ID’s with others. After that we set up the serial so we can discriminate our devices from others. The format used is domain:product-id:version-id. This format is mandatory because of the shared use policy. We have chosen to use a 3-position product-ID and a 2-position version-ID. This gives us enough space to create whatever we want.

At the moment of writing this article the PiDome server doesn’t understand this serial yet, but will be shortly implemented. The device is ready for use within the PiDome project though.

Because I had a lot of pins just doing nothing on my AVR I decided that I better make use of them anyway. So I added the option of attaching three sensors. These sensors are very basic and simple and just need one wire for transferring their ‘data’, which in fact are just a voltage between 0 and 5V. The sensors (in my case LDR and proximity) capture the pulses on my gas-, electricity- and watermeters. The AVR checks for those pulses and parses them back to the PiDome-servers so it can be handled and stored for a variety of things. Of course you can also trigger stuff based on incoming pulses, but that’s all up to what someone wants.

More to come

I know this is our first PCB-project and there will surely come some revisions as it’s not perfect yet. But hey, it’s a start.

John asked me to make more generic devices in the near future and I’ll try to do that. Besides that I’d like to know what kind of devices you would like to see added to the system/platform. And not only devices as such, but maybe also protocols or data-transportation layers that you might want or need. Without any input I can only guess and it’s likely that I cannot guess all of your wishes.

Until next time, have fun with the PiDome project and send in your ideas as a comment on this posting.

PS. Don’t forget to like us on Facebook ( ;)

Marcel Wiebenga

Added media support to the server and client (XBMC)

We have added the first basic media control functionalities to the server and the client. This also means the first plugin to be supported has also been added and is XBMC version 12.3 (Frodo release). While developing the media controls we took XBMC as base for the media plugin support. Which means you can browse through XBMC, play and pause movies and audio, and control the volume. There is also playlist support limited to the now playing data.

Adding XBMC Media control

Adding XBMC to the system is easy, but before you want to add an instance you need to let XBMC now that a remote device wants to control it. So you need to enable this first. Go to the XBMC wiki page and enable “Remote Control”.

When enabled open the server’s web interface and go to “Settings > Plugins > Media”. Click on the “Add new media” button. A popup will be shown where you can select the XBMC plugin. After selection the correct XBMC version you need to fill in the details of your XBMC installation.

Add XBMC version 12


Give it a name (which will appear in the client), a description and the location.

The ip address is the ip address of the device where you have XBMC installed. The telnet port is the raw socket connection is made to from the plugin, and by default is port 9090. The http port is the port you use to access XBMC via the web interface. I think this is default set to 8080.

When these details have been entered click “Ok” to save. The plugin immediately goes to work and tries to connect with the XBMC instance, and tries to discover what is playing and what the current playlist is.

Control your XBMC media via the web interface

To control your just added instance go to the web interface and browse to: “Control > Media”. On this page you will see a list of added media instances which you can control. Click on one of the items to open the control window.

Control XBMC media


In the screenshot above my XBMC instance is playing audio with the current playlist shown. You can control XBMC with the most left buttons to go to Home, Back or show the OSD. The buttons next to it is to browse and go up,down,left, right and select the current selected item.
Next to this is shown which item is currently playing. Bacuase it is a song it will try to show the song name, artist and album. If these are not known the song will display the file name. Below this information you can control the currently playing item, by pause it, play it again, go to the next or previous song. The stop button will stop plaing the media and clears the playlist. At the most right you can change the volume and mute/unmute XBMC.

The playlist controls are play the current item or remove the item from the playlist.

Control XBMC from the client

Usage in full mode

If you have downloaded the client and run it in widget or full mode, you can access all your media appliances by clicking/pressing the “Media list” folder. You will get a window with a list showing all the added instances. In this list you will have real time updates on which instance is running what and how big the playlist is. So when a new song is started you will see the list updated.
When clicking/pressing an instance it will give a popup providing the same interface as the web interface with the same controls which act exact the same way. The only difference is that if there is an other item playing it will be reflected real time. The web interface needs a page refresh. Also when the player is showing, you need to press “Show playlist” to see the playlist.



Control XBMC from the client


Usage in widget mode

Usage in widget mode is exactly the same as in full mode. The windows are exactly the same.



This is the first basic implementation of controlling an XBMC instance. There will be features added in the future like browsing the media libraries and add items to the playlist. Also when there are more appliances support added to the server they will share the exact same interface depending on the capabilities of that appliance. Only the supported controls will be displayed.

Also, this plugin version only supports audio and video, images or xbmc plugins are not supported, unless you can use the browsing buttons in them.

Implement your own control.

Raw socket

Since the server supports JSON-RPC you can write your own client to control multiple XBMC instances from a single point. To write your own client please read this first on how to connect and sign in on the server and then head to the Plugins RPC page on how to retrieve the plugins and the Media Plugin RPC page for methods of controlling media via the server.

You only need to know the server’s RPC methods and it does not matter which media plugin you are talking to, the server does the translation to the instance for you.


All the RPC methods are also available via the http socket (GET and POST), you will only not get any real time update unless you do a request requesting what is playing now.

XBMC note:

XBMC uses different playlist id’s for audio/video etc, you do not need to address them via the RPC. The server does this for you, but to maintain portability of the RPC methods when you receive a playlist, every item contains a playlist id. Although it looks like this id is the playlist id in XBMC (which it is), you can not rely on this id to be the XBMC id and you should even forget about XBMC playlist id’s while communicating with the server. So please, do not hard code plyslist id 0 for audio and playlist id 1 for video and use the id supplied with the audio/video item if available. If there is no playlist id, then there is no playlist id. Trust the server.

The above is for future compatibility in case of mixing of playlist types is possible for example with other appliances/software.

Started with the JSON-RPC structure

This is not a large post but merely a status update.

A coupe of days ago i started with the JSON-RPC engine and it is going quite well. The engine is being developed in such a way i only need to add an interface, a wrapper and functional specifications to expose methods. Of course only “public” methods are exposed like commands to be send to devices, devices adding/removing and editing. Eventually all these kinds of methods will be exposed and available through raw socket and an http endpoint.

At the moment of writing a couple of methods have been exposed and are available via the JSON-RPC specifications page. I wished i already had some more methods exposed earlier but while creating the engine i realized i have to change things in the server internally, but now with a lot of things changed i hope to get more and more methods done very soon.

Until i have also changed to client to follow this new RPC approach the server will be broadcasting the two different broadcast structures which are the “old” and the new RPC method.

I hope that when the server will be going to be reaching a beta stage others will be encouraged to also build a client if they think our client is not applicable or want to target other platforms.

The JSON-RPC specification pages will grow as methods and/or broadcasts are added to the server.

Upcoming updates/changes in the server and client

A few days ago Marcel and i sat together over thinking some stuff happening with the server and the client. And came to the conclusion some things have to change, implemented and still need to be thought of.

The client

Since Marcel is the only one who has a touch enabled device we could finally test the touch gestures implemented. First conclusions was that it works very well and is very responsive, but also that there are some things that have to be changed or improved. So in the next days we will be focusing on improving the client’s handling and visuals.

Now, to close a screen you have to hold a window and a popup menu shows the option to close the window. We thought it would be better just to have a close button. Well this one was obvious and already on the to do list, but now a little bit prioritized. I’m also thinking about minimizing windows, but i’m not entirely done thinking about the gestures to use for it.

Some other stuff where that window titles are just a little bit to small and some gestures could be improved like tapping a window instead of only the title should bring it to front. We where also thinking that a cool feature would be when dragging a window icon from the task bar should trigger a window to close and task bar window icons should be highlighted with the current window which has the focus.

While we where tapping, pressing and moving windows around i noticed that the icon task bar was showing icons correctly which did not fit in the bar and thus where not visible, so I will add an option that the window icons in the task bar can be scrolled.

Error messages are automatically closed within a couple of seconds, but a “natural” thing i did without thinking about it is tapping the message to close it. So this will also be added.

Some other stuff that will be changed/added are:

  • Better button visuals and more consistent,
  • Bigger default text sizes,
  • Removing images as window visuals (this one was really bad but just used to test the visual look).
  • And of course now since there is a media plugin present, add the media controls to the client.

A full list of changes we want to implement asap are mentioned here: Also if you have tried the client and encountered a bug, or want to see something cool in it you can also put a feature request or bug report on that page.

The Server

Server communication is done through a “telnet” like socket connection and commands and results are text based and are using dividers. It works, but is very old school and the semantics are becoming a trouble to maintain. The “problem”/feature is to communicate with the server with different methods. Like said there is a “telnet” based method, but there also is a web interface. They both use different methods to finally use the same method, example: Telnet: “device::commandgroup::handle::true”, Web interface: “device=device&commandgroup=group&handle=handle&exec=true” (even the commands send by the web interface can be different).

When i started the project i wanted to use JSON for the structure, but the above (which is not JSON at all) is a result of quick tries which grew to this now, i must say, horrible situation. In the beginning i never thought of creating a system to be thrown in the public domain. But now it is, it has really become time to implement something that is more consistent, more portable and better/easier to understand. So, to reach this goal we have decided to implement a general protocol to communicate with the server. All the server communication will be going to be jsonrpc 2.0.

The above will be a big change but will make sure that in the future when implemented, it does not matter how you communicate with the server it will always be the same.

Also a change will be made with how the server’s configuration is handled. This is now an xml file and is going to be changed to a name value pair properties file.

A list of all changes or features to implement is available here: This is also the page where you can do a feature request or a bug report.

Hardware support

Yes, a big thing. We really need to implement the above changes first, especially the server part, and the media support in the client. But the next in line for supporting is modbus (as also is in the list above). This means that the next thing we will be supporting are modbus based devices (next to devices Marcel creates).

Mobile devices support

Yes, we eventually will start with developing a mobile/tablet based application based on android. When this will be available i can not say, we want to get the core done first in the normal client. This core will be the same (as much as possible) in the mobile client so there hopefully is a single core to maintain with different visuals. But this has to be investigated further first to determine which part of the core can be the same and what can not.


The next updates will mostly be functional updates than added functionalities, but these updates are important so we can create new functionalities quicker. I think it will take a week or two before these updates are done and i can continue to add functionalities.

If you have any remarks, thoughts or idea’s, share them :)


Client connection authorization now available via the web interface

Today a next big little thing is available in the server, client connection authorizations. This means that when in the configuration file client authorization is turned on, you now not only can use the console to authorize clients, it is also available on the web interface. This option can be found at:  Server > Manage clients.

Previously when this option was turned on, there could be only one device authorized at a time. It is now possible to connect multiple clients at the same time. You will be represented with a list of clients which needs authorizing.

Below a screenshot of a client which needs authorization

List of clients to approve

List of clients to approve

By clicking the button Approve the client will be able to further connect with the server enabling receiving statuses and sending commands, thus resuming normal operations. When clicking disapprove a message will be send to the client showing it has been disallowed to connect and the connection is closed.

The next screenshot shows the above client approved

List of connected clients

List of connected clients

Within this list you can send a message to a client, or disconnect it. Messages are not yet displayed in the client because the client does not have the messaging routines ready yet.

The above list will not only show clients which has been approved, but all the connected clients. So when the needs approval option has been set to false, connected clients are shown.

The option to approve connecting clients can be found in the file: <server-install-dir>/config/system.xml, follow the xml tags until you are at: config > clients > displayclients > needsAuth. By default this is “false”. When set to “true” you will have authorization enabled. You will need to restart the server to let it work.

The manual page with more information is available here: Client Management


Get every new post delivered to your Inbox.

Join 102 other followers

%d bloggers like this: