Thinking towards a feature freeze (Even raw USB-HID is ready, but hence not visible), so what do you want to have frozen?

It is time  to go work towards a feature freeze and work towards version 0.1 for the server as for the client. But as with every feature freeze announcement there should also be a list of features you can expect when frozen. So here a little explanation, and, what do you expect or want? Read more of this post

Upcomming client changes

With the addition of utility measurements in the server there will also be additions made in the client to reflect these measurements. Not only are these added, we also change some other stuff and make some other additions. In this post i want to show the additions and changes which are made. I hope to have these changes rounded up and uploaded by Monday evening (21:00 GMT+1) or otherwise Tuesday evening the day after (My son has a lot of soccer stuff going on this weekend).

Some of you maybe already have spotted the upcoming changes on our Twitter or Facebook page and this post will then go in some more detail. Read more of this post

Development progress update (Utility measuring)

Marcel did not had a lot of time the last couple of months which put a little hold on device development. Two weeks ago we finally sat together again and where thinking about quick and dirty devices to develop. We got a couple of ideas like specific simple devices like light intensity measurements, temperature readings, and last but not least, utility usage measuring. He decided to firstly go for the utility measurement, and thinking this was a very good idea, i followed this idea.

Preparations

Hardware

We both have analog meters which measure electricity, gas and water usage. This means we have rotation disks which are reflective at the moment a specific amount of usage has passed (bu our electric disks are just the opposite of this). So we will be using Optical detector/Phototransistor‘s for capturing the “pulses”. We decided to use I²C for reading an Arduino  which has the sensors connected. But because of the I²C interface on the Raspberry Pi is master only we can only read the amount of “pulses” when the pi initiates this. So the software on the Arduino will be counting pulses, and when read from the pi, reset it’s internal counters for depending on the sensor data which is requested. So this is what we are going to do. P.S. There is a big change it will be go to an attiny, depending on our personal needs.

Software

Next to the hardware is of course the software needed to read these data in the server. Well, the device software is easy does it with the already done I²C driver interface. The question is now, am i going to use this device to also do the calculations concerning transforming these pulses to wattages, kwh, liters/gallons and gas m³ or kwh. We deciced it would be better to use a plugin for this. Well here are the catches coming.

The decision to create a plugin also means we should not only support a “every so much time read amount of pulses”, but also an “accept every pulse instantly” method. Also not all the meters are the same in measurement calculations. Some use so many rotations on a analog device is on kW/h, others use kh as measurement. And we are not done yet, there are differences in calculating gas usage, some use so much kW/h used and others use m³. And of course there are liters and gallons. And still we are not done.

When having a plugin you should keep users as happy as possible to deliver them options, but not too much or, even worse, too less. One of the features we thought off in the preparation is: “On what kind of information do we make suggestions on what is good usage, over usage or just: “Damn, that’s a lot!””. And this is something we/I still are not thought out of yet. Do we let users fill in they’re usage of last year and create a day normal, high, a lot, scale or based on the amount of house members? House members is the easiest way for everyone, but filling in amounts give more precise control because some use gas for heating, and some air coolers. When filling in the amount of house members there will be unfair messages, but using absolute data gives complexity.

So when the plugin comes available it will support continuous and timed pulse amount readings, and users will need to fill out they’re last years usage (or an user targeted usage) to create a fair usage scale. Costs calculations will not yet be in the first version of the plugin but will certainly come. Also the plugin will be able to hook to any device the user wants which measures usages pulses.

Current work in progress

Hardware

Currently we are using an Arduino board with an Atmega328p to do the measurements which counts the amount of pulses, and the server reads the device every minute. Below pictures of the sensors and current setup

Analog readers

Analog readers

The above shows the sensors we are using to detect the dark spot on the electricity meters, and the below photo shows the current test setup

Current power utility usage test setup.

Current power utility usage test setup.

The above photo shows the Arduino used with three leds. The red led lids up as long as the non reflective part of the disk of the analog electricity usage is in front of the sensor. This makes it very easy to debug any reading errors. The yellow and blue do the exact opposite. They lit up when the reflectors come before the sensor and respectively for gas and blue for water usage. The white wire are 16 meters of I²C and the black ones are going to the sensors.

Software

Currently the software can be configured to select a specific device and which sensor of that device is reading the specific utility usage, and what calculation has to be applied. Currenltly only analog disks are supported. See the screenshot below

Unfinished utility settings page

Unfinished utility settings page

The interface is still unfinished because some fields are missing like last years/goal measurements. And also high/low usage times and prices. Some extra fields will come where you fill in kwh/kh and liters/gallons. Gas will stay m³ but a possibility to supply a kW/h measurement unit is being thought of.

Current tests show a nice flow in usages which are being made with this plugin and is also already being graphed on the server’s web interface as seen below

Utility views

Utility views

The above is the beginning of the utility usage views in the web interface. IT supports real time gauge updates. The graphs can show the history for a day, week, month and year. More graphs and information will ofcourse follow as the plugin configuration get’s shape so more information can be shown.

In line with the above is the development of the client which of course should display this data. The screenshot below shows the clients progress

Utility gauges and little graphic updates

Utility gauges and little graphic updates

This screenshot is taken at a different moment, but shows the gauges of the current usages. Also in the client it will be possible to attach a gauge to the top bar. All the gauges are also in real time updated, and will of course be extended with graphs and information as soon it is available. The current shows work in progress and this was a first test with gauges in the upper task bar. Clicking on the task-bar widget opens up the bigger gauge.

Conclusion

The above is the current work in progress and when any feature has been build in it will be uploaded to the alpha version on the download page, and in between hot builds are available on the build server. Be aware that the build server is not always available because it is a workstation. I hop the above shines a little light on what we are doing right now. If you have any suggestion please let us know in the form field below, on twitter or facebook. The latter two will show more news and screenshots.

Hardware source code

Of course when stable, the source code for the hardware used will be available on bitbucket for download.

I’m tired, good night,
John.

Web interface overhaul is done (including a change log).

Finally, i have completely replaced the old web interface with a new one. This was quite a lot of work because i did not only replaced current functionalities but while I was replacing stuff, a lot of new functionalities have been added and some nasty little beasts have been squashed. A very imporant notice is that the new interface is focused on HTML 5 and CSS 3. This means that older browsers are not supported (as the old interface did not support new browsers).

Browser support listing should be:

  • Internet Explorer: 9+,
  • Chrome: current,
  • Opera: current,
  • Safari: current,
  • Firefox: current.

In other words, JQuery 2.0 compatible browsers.

Some examples of additions that have been done:

  • Server localization has been added with modifiable timezone settings which includes a map you can use to just click on the exact location of your server.
  • Server state (what’s in a name) has been replaced with day parts and presence properties,
  • Graphs,
  • Initial tablet and phone interfaces,
  • A real time web interface, without refreshing pages the last known device/system/presence/etc.. is known.
  • A bunch load of RPC methods have been added (will be added to the manual pages asap).
  • Added floor locations to the locations itself.

Some examples of changes:

  • A far more faster web interface,
  • The trigger and macro system has been redesigned from the ground up and are by far more faster,

The above are just some examples, The list of changes is far more greater, The full change log is below: Read more of this post

How the trigger system works

Hi all,

This will be a quick post about the trigger system introduced in the new web interface. The old interface already had a trigger system but this was restricted to when device this has that value do a device command. There was also no easy way to follow the rule steps.

Below is a screenshot of the new trigger system where the rest of the post will try to explain it and also talks about how the system works.

An example trigger entry

An example trigger entry

Read more of this post

Web interface overhaul progress update

Hi all, I thought it was time to post a little web interface overhaul update. It is not going as fast as i hoped it would go, but some personal things got in the way and of course i have a family to take care of.

The changes being made are a bit more extensive then only the web design overhaul. This big change graphical change also caused some other stuff to change within the server code, add more RPC methods so others can also completely control the server with they’re own interface (if there is a need for it).

Overhauling and adding

Not only is the web interface getting an overhaul but in sync with the web interface there will also be a web interface for mobile devices. So there will be three interfaces in total. The web interface overhauled and a new interface for mobile phone and tablet devices. The main difference between the browser and the mobile versions is that the mobile version will only be able to control devices, media, etc. and the web interface does the maintenance next to the controlling.

Noticeable features added

Some noticeable new features are:

  1. An easier method to set the location of the server/your home.
    The server uses latitude and longitude to calculate sunrise and sunset. The web interface now has the possibility to ask the browser for location data which is reflected on a map. On this map you can adjust your location to exactly pinpoint where your home/server is. (This information is never ever or whatever send to any or whatever some online service or something else you can think of but stored in a local configuration file).
  2. A visual trigger editor. Triggers are used with input from devices which can, if a certain condition is met, execute macro’s or device commands.
    Here you can add/edit triggers with just some mouse clicks and you can exactly see how the trigger works.
  3. Real time web interface updates.
    The web interface in the browser as well on mobile devices supports web sockets. This means that if you and someone else are looking at the same device where one is on the computer and the other on a tablet. And the tablet user for example switches a light on, the web interface user will see it change on the web page without refreshing. This way you are always sure you see the latest device statuses.
  4. Graphs have been added for devices which have a numeric output.
    When a device is configured to have a numeric output which is or logarithmic or series the web interface will notice the user of the availability of a specific graph.

We are also posting screenshots of pages which are done on the web screenshots page.

Current change-log

Below is a list of changes currently implemented in the latest alpha release with the exception of the graphical trigger editor and not all the RPC changes are included. This list is a list i keep privately so i know what i have done next to the todo list. Because we occasionally renew the alpha release the current release shows two different web interfaces in the browser. If a page has not been updated you will be redirected to the old page and vice verse. Also the web sockets on phones and tablets is currently not supported in the default android browser because this browser has no native web sockets support.

Read more of this post

Development progress update

Hi all, welcome to this quick progress update post. As in an earlier post (Upcoming updates/changes in the server and client) i have posted some trivial and critical changes to be made to the server and client. Below is a list of things that have changed/modified or added:

Server changes:

  1. All client messaging is now done via json,
  2. Added a http json entry point (http port),
  3. Added a websocket service (http ws port),
  4. Added a settings page where client approval/debug mode can be toggled,
  5. Added the possibility to change the used timezone via the web interface,
  6. Internal changes to client handling,
  7. Added support for XBMC media players.

Client changes:

  1. We have updated the client to support all the server’s to clients exposed services:
    1. Plugins like the media plugin,
    2. Messaging between clients.
  2. The client overhauled to do full json rpc messaging
  3. A lot of internal client changes:
    1. Windows where images, are now dynamic,
    2. Better touch support (including bigger touch points),
    3. Some speed and memory improvements,
    4. Better messaging if the client is approved, denied by the server,
    5. Client now honors the server time, instead of the time of the host computer.
  4. And some improvements which causes us to develop new features faster and better.

The above changes took a while to implement, especially the JSON-RPC engine and the adding of the media plugin took more time then i anticipated.

Upcoming changes

Below i reflect the next upcoming changes and added features in order of planning.

System state

While the above changes are done, some changes have not made the previous post but are quite important to change as early as possible. When i started this project i made a little thinking error. I created server states which where relying on the time of day. This in our opinion is not the way to go. A state based on a time of day can not include (as designed now) the user presence, which of course is a big thing in home automation.

The change we will be making is that system states will be migrated to user presence like “At home”, “Away”, “Vacation”, etc. and the time of day/day parts will be a server property.

Web interface

This is a big hog. When i started with the web interface (which was meant as just a prototype, allowing quick and dirty actions) i only used Chrome for my testing (because of the integrated tools etc.). This caused programming which do not comply to all modern browsers to be supported. The biggest example would be the usage of an older jquery version causing problems in modern browsers. Also the web interface is not able to real time reflect changes happening in the background (for example device value changes or an other media item playing). And because the trigger and device xml pages are quite complex we need to change this appearance so the web pages should expose an easier to understand interface.

Supporting only one browser we can be sure of out of the box is a big no no, so a web interface overhaul is planned earlier then expected. So now having websockets available the web interface will be changed to html5.

Hardware support

The server internals do already support a couple of things like i2c, Arduino based boards via xml(untested), universal com (via usb) devices (untested), raspberry pi serial pins (deactivated) and HID devices (not complete). But not all these are exposed an should be finished and made available.

I was planning to add Modbus support immediately after adding the media plugin. The above paragraphs are having a larger and more complex impact on system internals and user interaction, so these have been pushed up into the list. The Modbus support will for example relies on the universal comport interface just like the arduino based boards. We need this interface because there are a lot of Modbus controllers out there.

Mobile device support

Now that the client core is getting to a point where we want it to be, we are looking in how to implement this core with Android. The core is about receiving broadcasts from the server indicating presence and authentication. It will be likely that the there will be an html5 version created for displaying on mobile devices in sync with the overhaul of the web management interface.

Conclusion

There are a lot of priority changes made, but we think this is for the better, improving the basics before adding new features. The above changes are the changes/additions we will be working on in order from now on (besides solving bugs if found and the mobile support).

So until the web interface has been updated, only use chrome for the best results.

PiDome Relay board

Hi,

It’s been a while since I had my last posting here. I’ve been very busy with my work and other stuff. I will try to get more postings and hardware added to the system in the near future. This is just a small posting as what this board is about and an in-depth description of the technical stuff will be added soon.

As you may have seen we can now make our own PCB’s for our projects and the first one that we created was for the relay board. It is a board that has two switching relays and is controlled through USB, but without the need to have a dedicated USB-chip onboard. Next to that there is the possibility to connect up to three sensors to the board.

I have created this board in order to make it possible for me to switch my heating system on and off, but of course you can use it for any purpose, like lighting, as the relays used are capable of switching high voltage (7A at 230V).

USB and some sensors

The whole USB-part of the device relies on the HID-protocol (Human Interface Device) which is widely used in simple computer products like keyboards and mice. We choose for this because nowadays a lot of devices have an USB-connection and our board itself consumes very little power that we can take from the USB-connector, besides that it is relatively easy to implement this software-wise. I’ve used to HID-Serial library (from Ray Wang at github.com/rayshobby/hid-serial) that he adapted from V-USB for the Arduino-platform.
As identification of the device relies on ID’s (Vendor- and Device-ID) there is a small catch. Because we are too small and don’t have the resources to get our own ID’s we needed to find a solution for this. Luckily there are some people out there that are willing to share their ID’s with others. The only thing that we need to do is comply with the rules that they have set up for this, which aren’t that harsh. In the library-software you can set your Vendor-ID and Device-ID (in our case of voti.nl), which is from the nice guy sharing his ID’s with others. After that we set up the serial so we can discriminate our devices from others. The format used is domain:product-id:version-id. This format is mandatory because of the shared use policy. We have chosen to use a 3-position product-ID and a 2-position version-ID. This gives us enough space to create whatever we want.

At the moment of writing this article the PiDome server doesn’t understand this serial yet, but will be shortly implemented. The device is ready for use within the PiDome project though.

Because I had a lot of pins just doing nothing on my AVR I decided that I better make use of them anyway. So I added the option of attaching three sensors. These sensors are very basic and simple and just need one wire for transferring their ‘data’, which in fact are just a voltage between 0 and 5V. The sensors (in my case LDR and proximity) capture the pulses on my gas-, electricity- and watermeters. The AVR checks for those pulses and parses them back to the PiDome-servers so it can be handled and stored for a variety of things. Of course you can also trigger stuff based on incoming pulses, but that’s all up to what someone wants.

More to come

I know this is our first PCB-project and there will surely come some revisions as it’s not perfect yet. But hey, it’s a start.

John asked me to make more generic devices in the near future and I’ll try to do that. Besides that I’d like to know what kind of devices you would like to see added to the system/platform. And not only devices as such, but maybe also protocols or data-transportation layers that you might want or need. Without any input I can only guess and it’s likely that I cannot guess all of your wishes.

Until next time, have fun with the PiDome project and send in your ideas as a comment on this posting.

PS. Don’t forget to like us on Facebook (facebook.com/PiDomeDomotica) ;)

Marcel Wiebenga

Added media support to the server and client (XBMC)

We have added the first basic media control functionalities to the server and the client. This also means the first plugin to be supported has also been added and is XBMC version 12.3 (Frodo release). While developing the media controls we took XBMC as base for the media plugin support. Which means you can browse through XBMC, play and pause movies and audio, and control the volume. There is also playlist support limited to the now playing data.

Adding XBMC Media control

Adding XBMC to the system is easy, but before you want to add an instance you need to let XBMC now that a remote device wants to control it. So you need to enable this first. Go to the XBMC wiki page and enable “Remote Control”.

When enabled open the server’s web interface and go to “Settings > Plugins > Media”. Click on the “Add new media” button. A popup will be shown where you can select the XBMC plugin. After selection the correct XBMC version you need to fill in the details of your XBMC installation.

Add XBMC version 12

 

Give it a name (which will appear in the client), a description and the location.

The ip address is the ip address of the device where you have XBMC installed. The telnet port is the raw socket connection is made to from the plugin, and by default is port 9090. The http port is the port you use to access XBMC via the web interface. I think this is default set to 8080.

When these details have been entered click “Ok” to save. The plugin immediately goes to work and tries to connect with the XBMC instance, and tries to discover what is playing and what the current playlist is.

Control your XBMC media via the web interface

To control your just added instance go to the web interface and browse to: “Control > Media”. On this page you will see a list of added media instances which you can control. Click on one of the items to open the control window.

Control XBMC media

 

In the screenshot above my XBMC instance is playing audio with the current playlist shown. You can control XBMC with the most left buttons to go to Home, Back or show the OSD. The buttons next to it is to browse and go up,down,left, right and select the current selected item.
Next to this is shown which item is currently playing. Bacuase it is a song it will try to show the song name, artist and album. If these are not known the song will display the file name. Below this information you can control the currently playing item, by pause it, play it again, go to the next or previous song. The stop button will stop plaing the media and clears the playlist. At the most right you can change the volume and mute/unmute XBMC.

The playlist controls are play the current item or remove the item from the playlist.

Control XBMC from the client

Usage in full mode

If you have downloaded the client and run it in widget or full mode, you can access all your media appliances by clicking/pressing the “Media list” folder. You will get a window with a list showing all the added instances. In this list you will have real time updates on which instance is running what and how big the playlist is. So when a new song is started you will see the list updated.
When clicking/pressing an instance it will give a popup providing the same interface as the web interface with the same controls which act exact the same way. The only difference is that if there is an other item playing it will be reflected real time. The web interface needs a page refresh. Also when the player is showing, you need to press “Show playlist” to see the playlist.

 

 

Control XBMC from the client

 

Usage in widget mode

Usage in widget mode is exactly the same as in full mode. The windows are exactly the same.

 

Conclusion

This is the first basic implementation of controlling an XBMC instance. There will be features added in the future like browsing the media libraries and add items to the playlist. Also when there are more appliances support added to the server they will share the exact same interface depending on the capabilities of that appliance. Only the supported controls will be displayed.

Also, this plugin version only supports audio and video, images or xbmc plugins are not supported, unless you can use the browsing buttons in them.

Implement your own control.

Raw socket

Since the server supports JSON-RPC you can write your own client to control multiple XBMC instances from a single point. To write your own client please read this first on how to connect and sign in on the server and then head to the Plugins RPC page on how to retrieve the plugins and the Media Plugin RPC page for methods of controlling media via the server.

You only need to know the server’s RPC methods and it does not matter which media plugin you are talking to, the server does the translation to the instance for you.

Web RPC

All the RPC methods are also available via the http socket (GET and POST), you will only not get any real time update unless you do a request requesting what is playing now.

XBMC note:

XBMC uses different playlist id’s for audio/video etc, you do not need to address them via the RPC. The server does this for you, but to maintain portability of the RPC methods when you receive a playlist, every item contains a playlist id. Although it looks like this id is the playlist id in XBMC (which it is), you can not rely on this id to be the XBMC id and you should even forget about XBMC playlist id’s while communicating with the server. So please, do not hard code plyslist id 0 for audio and playlist id 1 for video and use the id supplied with the audio/video item if available. If there is no playlist id, then there is no playlist id. Trust the server.

The above is for future compatibility in case of mixing of playlist types is possible for example with other appliances/software.

Started with the JSON-RPC structure

This is not a large post but merely a status update.

A coupe of days ago i started with the JSON-RPC engine and it is going quite well. The engine is being developed in such a way i only need to add an interface, a wrapper and functional specifications to expose methods. Of course only “public” methods are exposed like commands to be send to devices, devices adding/removing and editing. Eventually all these kinds of methods will be exposed and available through raw socket and an http endpoint.

At the moment of writing a couple of methods have been exposed and are available via the JSON-RPC specifications page. I wished i already had some more methods exposed earlier but while creating the engine i realized i have to change things in the server internally, but now with a lot of things changed i hope to get more and more methods done very soon.

Until i have also changed to client to follow this new RPC approach the server will be broadcasting the two different broadcast structures which are the “old” and the new RPC method.

I hope that when the server will be going to be reaching a beta stage others will be encouraged to also build a client if they think our client is not applicable or want to target other platforms.

The JSON-RPC specification pages will grow as methods and/or broadcasts are added to the server.

Follow

Get every new post delivered to your Inbox.

Join 109 other followers

%d bloggers like this: