Posted: Monday Feb 8th, 2016 09:13 pm |
|
1st Post |
cab123
Member
Joined: | Saturday Oct 3rd, 2009 |
Location: | Coimbra, Portugal |
Posts: | 25 |
Status: |
Offline
|
back to top
|
Hi, i would like to share with the community my simple driver implementation of comfort to MQTT.
I have this running for a couple of months and never registered any failure with the communication. MQTT is amazingly light and mature. And my driver seems to be stable enough for the job, had some issues with strange USB resets, but now it gracefully handles every USB quirks i found, including reconnecting the UCM.
This approach provides a simple and flexible way to interact with comfort from/to multiple sources. Personally i have comfort ULTRA, individual power monitoring, solar inverter statistics all feeding data to a MQTT broker. Then I process some of that data in NodeRED and present/control everything in OpenHAB. It works really well.
I do not use the alarm section of comfort, only home automation with push buttons, relays, infrared and some KNX stuff.
Used hardware:
- UCM USB
- A x86 server with Debian Jessie installed (could be a raspberry Pi)
(1)
Connect the UCM/USB to the server and make the USB connection deterministic:
(1.1)
Find the Device / ID / Serial Number of the device:
root@debian:/# lsusb
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 002: ID 0403:6001 Future Technology Devices International, Ltd FT232 USB-Serial (UART) IC
Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
root@debian:/# lsusb -d 0403:6001 -v | grep iSerial
iSerial 3 A7007iyv
|
(1.2)
From the output above we get:
- idVendor: 0403
- idProduct: 6001
- Serial Number: A7007iyv
Your data will probabily be the same, except for the unique serial number of your UCM.
Next create a file like the one below and add the right info:
nano /etc/udev/rules.d/99-usb-serial.rules |
SUBSYSTEM=="tty", ATTRS{idVendor}=="0403", ATTRS{idProduct}=="6001", ATTRS{serial}=="A7007iyv", SYMLINK+="comfort" |
Now everytime the USB UCM is connected th the server it will be uniquely available at /dev/comfort, regardless of other USB peripherals in the system.
(2)
Install MQTT broker and clients
http://mosquitto.org/2013/01/mosquitto-debian-repository/
for the clients:
sudo apt-get install mosquitto-clients
anoher good reference with some testing:
https://pallavichaurasia94.wordpress.com/2014/10/14/mosquitto-in-debian/
(3)
the script to establish a bridge between Comfort and MQTT:
(3.1)
First install the folloing perl modules:
cpan Net::MQTT::Simple
cpan IO::Select
cpan Time::HiRes |
(3.2)
Find a appropriate directory, for example /opt and copy the contents of the script to a new file, for example:
nano simple_mqtt_serial_select.pl |
#!/usr/bin/perl
use Net::MQTT::Simple;
use IO::Select;
use Time::HiRes qw(time);
use strict;
#############################################################
### by:APR ###
### ###
### Perform the following system commands: ###
### lsusb ###
### lsusb -d 0403:6001 -v | grep iSerial ###
### nano /etc/udev/rules.d/99-usb-serial.rules ###
### SUBSYSTEM=="tty", ATTRS{idVendor}=="0403", ATTRS{idProduct}=="6001", ATTRS{serial}=="A7007iyv", SYMLINK+="comfort"
### ###
### Change the following lines to suit your configuration ###
### ###
#############################################################
my $comfort_port = '/dev/comfort';
my $mqtt_hostname = 'localhost';
my $mqtt_to_comfort = "comfort/to";
my $mqtt_from_comfort = "comfort/from";
#############################################################
my $serial_location = "off";
my $serie = IO::Select->new();
my $mqtt = Net::MQTT::Simple->new($mqtt_hostname);
$mqtt->subscribe($mqtt_to_comfort,\&callback);
my $ts = time();
while(1) {
if ( time() - $ts > 10 ) { # every 10 secs
$ts = time();
my $SL = `ls -al /dev/comfort 2>&1`;
if( $SL =~ /(comfort -> .+)/ ) { # isconnected
$SL = $1;
if ( $serial_location eq "off" ) { # was plugged
print localtime(time) . " $SL\n";
$serial_location = $SL;
open(SERIAL, "+<$comfort_port") or die "Can't open serial port: $!\n";
$serie->add(\*SERIAL);
}
elsif ( $SL ne $serial_location) { # was restarted
print localtime(time) . " $SL\n";
$serial_location = $SL;
$serie->remove(\*SERIAL);
close(SERIAL);
open(SERIAL, "+<$comfort_port") or die "Can't open serial port: $!\n";
$serie->add(\*SERIAL);
}
}
else { # is disconnected
if($serial_location ne "off") { # was removed
print localtime(time) . "UCM disconnected\n";
$serial_location = "off";
$serie->remove(\*SERIAL);
close(SERIAL);
}
}
}
$mqtt->tick(0.1);
if($serial_location ne "off") {
if ($serie->can_read(.01)) {
my $recv = <SERIAL>;
$recv =~ tr/\x03\x0D\x0A//d;
$mqtt->publish($mqtt_from_comfort => $recv);
}
}
}
sub callback {
my ($topic, $message) = @_;
if($serial_location ne "off") { print SERIAL chr(0x03) . "$message\r\n"; }
else { $mqtt->publish($mqtt_from_comfort => "OFFLINE"); }
} |
To test:
WINDOW_1: run the script
root@debian:/# perl simple_mqtt_serial_select.pl |
WINDOW_2: listen for MQTT events
root@debian:/# mosquitto_sub -v -t comfort/# |
WINDOW_3: post something to comfort topic MQTT
root@debian:/# mosquitto_pub -t comfort/to -m 'LI1234' |
On the WINDOW_2 you should see something like:
comfort/to LI1234
comfort/from LU01 |
That´s it! Now you can subscribe (listen) or post commands to the MQTT broker and interact with comfort.
Make the script boot at startup by editing the file just before "exit 0"
root@debian:/# nano /etc/rc.local |
/usr/bin/perl /opt/simple_mqtt_serial_select.pl & |
And now comfort can easily be integrated with other bits of amazing software, like openHAB, NodeRED, etc.
If there is any interest in this i can share some of my NodeRED and OpenHAB integration configuration.
Have fun!
|
Posted: Wednesday Feb 10th, 2016 07:23 am |
|
2nd Post |
tech07
Administrator
Joined: | Thursday Jan 29th, 2009 |
Location: | Singapore |
Posts: | 106 |
Status: |
Offline
|
back to top
|
yes pleasecan this work with UCM/ETh03?
How can this allow working with openhab etc
|
Posted: Wednesday Feb 10th, 2016 07:31 pm |
|
3rd Post |
cab123
Member
Joined: | Saturday Oct 3rd, 2009 |
Location: | Coimbra, Portugal |
Posts: | 25 |
Status: |
Offline
|
back to top
|
Hi, i'm not sure how the UCM/ETH03 works, since i don't have one, but i suppose everything is possible .
I'll gather some info and post here later a working example of OpenHAB controlling comfort.
|
Posted: Saturday Feb 13th, 2016 06:58 am |
|
4th Post |
cab123
Member
Joined: | Saturday Oct 3rd, 2009 |
Location: | Coimbra, Portugal |
Posts: | 25 |
Status: |
Offline
|
back to top
|
So the next step after having a MQTT broker interacting with comfort is to install NodeRED
http://nodered.org/
It is really easy to set up stuff in logical blocks, however some javascript is needed for the functions.
This is my processing of comfort stuff:
The first "flow" updates comfort's time periodically. Every 10 minutes a timestamp is generated, passed through the "Build Date Command" and sent to the MQTT topic comfort is listening to. The function is something like this:
At the bottom of the first image, every group of received messages from Comfort are handled. For instance, i have temperature sensors, one KNX and another from a SCS that are transformed to MQTT also and sent to the broker. This way i can later read the sensors and display them on openHab.
The "split function is configured as shown:
and the Sensor function decodes comfort sensor messages like this:
var string = msg.payload;
var re = /^s[r\?](..)(..)(..)/; // regex to extract LSB, MSB
var sensor = msg.payload.replace(re, "$1"); //get sensor id
var hex = msg.payload.replace(re, "$3$2"); //Store MSB LSB
var dec = parseInt(hex,16); // convert to Decimal
if ((dec & 0x8000) > 0) { // handle negative numbers
dec = dec - 0x10000;
}
switch(sensor) {
case "00":
msg.topic = "house/floor2/livingroom/temperature";
break;
case "09":
msg.topic = "house/floor2/bedroom/temperature";
break;
default:
msg.topic = "unspecified";
}
msg.payload = dec;return msg; |
All processing can be done here. Imagination is the limit, everything can be done with small bits of code and simple "flows".
Later i will try to show some of my OpenHab setup that takes advantage of all this data.
Fell free to ask any questions.
|
Posted: Wednesday Aug 10th, 2016 07:34 pm |
|
5th Post |
cab123
Member
Joined: | Saturday Oct 3rd, 2009 |
Location: | Coimbra, Portugal |
Posts: | 25 |
Status: |
Offline
|
back to top
|
Hi guys sorry for the late update, i didn't want to show the openhab examples in portuguese so i kept delaying a update. Now i have changed my dashboards to show you what is the final result.
for setting up openhab to control comfort you just need to interact with the mqtt broker, still it is a bit slow learning curve but it is totally worth it!
and for example inside the TV control you can find the actual control:
The buttons listen for updates, so if i turn off my living room TV locally, i am actually pressing a KNX button that sends a telegram to the BUS that is captured by the UCM and passed on to comfort that updates a counter that is sent though serial connection that is captured by the perl driver and sent to MQTT, that broadcasts it to any listening device, where openhab is waiting, and then applies some rules to update the button status that is binded to the TV, so next time i toggle the TV i will perform the right action! Easy!
Last edited on Wednesday Aug 10th, 2016 07:36 pm by cab123
|
Posted: Friday Aug 12th, 2016 08:58 am |
|
6th Post |
slychiu
Administrator
Joined: | Saturday Apr 29th, 2006 |
Location: | Singapore |
Posts: | 5650 |
Status: |
Offline
|
back to top
|
Thats nice. Its good to see Comfort with Openhab.
We will try to test it
|
Posted: Saturday Aug 13th, 2016 01:43 pm |
|
7th Post |
slychiu
Administrator
Joined: | Saturday Apr 29th, 2006 |
Location: | Singapore |
Posts: | 5650 |
Status: |
Offline
|
back to top
|
How does the Comfort program go into this setup?
|
Posted: Friday Sep 9th, 2016 04:40 pm |
|
8th Post |
Posted: Thursday Oct 6th, 2016 04:45 pm |
|
9th Post |
meep
Member
Joined: | Saturday Sep 6th, 2014 |
Location: | |
Posts: | 11 |
Status: |
Offline
|
back to top
|
Hi
Is there any chance you could post more of your Node-Red javascript?
I need to do exactly this as I want to start using my old comfort as essentially an i/o block for openHab.
The more reference code I can find the better.
As an aside - any thoughts on how one could go about accessing Comfort voicemail recordings from openHab?
Many thanks
Peter
|
Posted: Sunday Oct 9th, 2016 01:32 am |
|
10th Post |
idheath
Member
Joined: | Saturday Aug 11th, 2007 |
Location: | United Kingdom |
Posts: | 87 |
Status: |
Offline
|
back to top
|
I'm so pleased that I've seen this thread :-)
I have just started with openHAB and am just about to have a play with MQTT (for other reasons). I've been wondering for ages how I could connect Comfort to openHAB and wondered myself about MQTT as a sensible option.
I have a UCM/ETH02 which I intend to keep for Comfigurator.
I also have a UCM/232 which I plan to connect to my Raspberry Pi 3 (which is running openHAB and will also be running MQTT when I install it). I've got a tiny USB to serial converter so I should be able to link my UCM/232 to the Raspberry Pi that way.
|
Posted: Sunday Oct 9th, 2016 02:01 am |
|
11th Post |
meep
Member
Joined: | Saturday Sep 6th, 2014 |
Location: | |
Posts: | 11 |
Status: |
Offline
|
back to top
|
I managed to get comfort talking to Mqtt and node-red via ucm/232 and a usb/serial convertor. Works great both to and from.
It's on Windows though so I needed to completely re-write the perl driver to work. I'll share it here when I've got it refined.Last edited on Sunday Oct 9th, 2016 02:02 am by meep
|
Posted: Sunday Oct 9th, 2016 11:23 pm |
|
12th Post |
idheath
Member
Joined: | Saturday Aug 11th, 2007 |
Location: | United Kingdom |
Posts: | 87 |
Status: |
Offline
|
back to top
|
meep wrote:
I managed to get comfort talking to Mqtt and node-red via ucm/232 and a usb/serial convertor. Works great both to and from.
It's on Windows though so I needed to completely re-write the perl driver to work. I'll share it here when I've got it refined.
Thanks v.much
|
Posted: Monday Oct 10th, 2016 03:39 am |
|
13th Post |
meep
Member
Joined: | Saturday Sep 6th, 2014 |
Location: | |
Posts: | 11 |
Status: |
Offline
|
back to top
|
Well, I managed to get this working on windows with Win32::SerialPort in Perl. (just couldn't hack the above code to work so started afresh).
It's a long way from cab123's original driver in terms of code but owes a lot to his concept, structure and techniques. It's also got a LOT less error checking implemented so is really just a proof of concept at this stage and would require a few more additions to make it more robust.
While this does send and receive between comfort and MQTT, and you can check these messages in Node-Red, I added a Trace function to allow key messages be printed to the console - really helps with debugging.
!/usr/bin/perl
use Net::MQTT::Simple;
use Win32::SerialPort;
use strict;
use warnings;
#hello
print STDERR "\n".localtime()."\nOpening Comfort Communications... \n";
#Set up & open port
my $port = Win32::SerialPort->new("COM5") or die "Failed to open port: $!\n";
print STDERR "Port established: $port \n";
$port->baudrate(9600);
$port->databits(8);
$port->parity("none");
$port->stopbits(1);
$port->handshake("xoff");
$port->write_settings;
#set up mqtt & polling for inbound messages
my $mqtt_hostname = '192.168.1.199';
my $mqtt_to_comfort = "comfort/to";
my $mqtt_from_comfort = "comfort/from";
my $mqtt = Net::MQTT::Simple->new($mqtt_hostname);
$mqtt->subscribe($mqtt_to_comfort,&callback);
#additional control vars
my $serial_stream = ""; #container for serial input from comfort
my $ts = time(); #current time
my $defaultreadspeed = 1; #port read interval
my $trace = 1; #enable debug output to console
#setup complete
print STDERR "Starting to listen...\n";
while (1) { #main loop
if ( time() - $ts > $defaultreadspeed ) { #use $defaultreadspeed to throttle
$ts = time(); #set target time for next loop
$serial_stream = $port->read(200); #read up to 200 bytes from Comfort
if ($serial_stream) { #check if there's data read
doTrace ("received from Comfort: $serial_stream");
#split the data into an array based on STX or EOL
#remove those extraneous characters from the resulting array elements
my @reports = split /(?:\x03|\x0D|\x0A)/, $serial_stream;
for my $rep (@reports) { #loop through each of the returned strings
if (length($rep)){ #only process non-empty strings
doTrace ("sending report to MQTT: $rep");
$mqtt->publish($mqtt_from_comfort => $rep); #send to MQTT
}else{
doTrace("empty string, not sending to MQTT");
}
}
$port->lookclear; #needed to prevent blocking (?)
} #end check if data read
$mqtt->tick(1); #force query MQTT input with 1 second timeout
} #time check
}#while loop
################
##SUBROUTINES##
##############
sub callback { #process inputs from MQTT and send to Comfort
my ($topic, $message) = @_;
doTrace ("got message from MQTT: $message");
$port->lookclear; #needed to prevent blocking (?)
my $str = chr(0x03).$message."\r\n"; #prepend STX char and add CR
$port->write($str); #send to Comfort
doTrace ("sent to Comfort: $str");
}
sub doTrace { #write trace logs to console
my ($traceString) = @_;
if ($trace){ #only implement if enabled
my $timestamp = getLoggingTime(); #get timestamp
print STDERR "$timestamp $traceString \n"; #print to console
}
}
sub getLoggingTime { #compose a decent looking timestamp
my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst)=localtime(time);
my $nice_timestamp = sprintf ( "%04d%02d%02d %02d:%02d:%02d",
$year+1900,$mon+1,$mday,$hour,$min,$sec);
return $nice_timestamp;
} |
This can be tested with something like the command below at the command line. Obviously change your own Mosquito server and port details. (note, you must first issue the login (LI) command to Comfort, with a user code, or it will keep returning 'NA' as the response);
mosquito_pub -h 192.168.1.199 -p 1883 -t comfort/to -m "z?" |
Attached is an example of this running. You can see the initial receipt of the "z?" query from MQTT and the response from comfort sent back. Then you can see Comfort reporting activity Input 05 (that's the result of me opening and closing a window with a contact sensor).
NB: For some reason, the forum is stripping backslashes from the code segments above. I've tried to add them back in but if you're trying to run this and it's failing, it mey be due to a missing '\' somewhere.
Attachment: Capture.PNG (Downloaded 117 times) Last edited on Monday Oct 10th, 2016 01:09 pm by meep
|
Posted: Tuesday Oct 11th, 2016 03:15 am |
|
14th Post |
cab123
Member
Joined: | Saturday Oct 3rd, 2009 |
Location: | Coimbra, Portugal |
Posts: | 25 |
Status: |
Offline
|
back to top
|
Hey! I'm very happy to see you are having fun with this!
I don't have much more javascript examples, aside from what i already shared, it's pretty basic stuff.
i would recommend you to switch to a linux box for these server-side protocols - they will run forever, will never complain, and be readily available to serve you anytime :-)
A small raspberry Pi is cheap to buy/run and has enough horsepower to do this and much more.
|
Posted: Tuesday Oct 11th, 2016 03:49 am |
|
15th Post |
meep
Member
Joined: | Saturday Sep 6th, 2014 |
Location: | |
Posts: | 11 |
Status: |
Offline
|
back to top
|
Cheers, it's more getting to grips with node-red and regex to be honest, getting there slowly.
I run all my stuff on a single unraid server through a combination of dockers and vms.
I went with a Windows vm as I needed to pass through the usb/serial adapter and as I'll likely need to run comfigurator for a while, thought I'd go with Windows as I wouldn't need to keep swapping connections etc.
Might switch back to a Linux vm as I get more comfortable with it all.
Thanks for the inspiration on this and taking the time to document it here, really helped me realise what I needed to do was possible.
|
Posted: Thursday Jul 11th, 2019 02:35 pm |
|
16th Post |
jelockwood
Guest
Joined: | |
Location: | |
Posts: | |
Status: |
Offline
|
back to top
|
@cab123
If you're still working on this any plans to implement an MQTT broker on the UCM/Pi module?
Obviously one could use the UCM/Eth03 and then link to a standard external Raspberry Pi but would seem to make sense to do the Comfort side of this using the dedicated Pi module.
|
Posted: Friday Jul 12th, 2019 12:25 pm |
|
17th Post |
mattbrain
UCM Pi Users
Joined: | Tuesday Jun 12th, 2007 |
Location: | United Kingdom |
Posts: | 185 |
Status: |
Offline
|
back to top
|
The alphaWerk components uses MQTT as their backend & Mosquitto is installed on the CM3. It is locked to localhost only by default but can be opened up to external devices easily.
There is also nothing stopping you from installing different software, hardware notes are available to use your own tools to talk to comfort and manage the watchdog timer.
|
|