Comfort in Meltdown!

[Pgordon]

This is turning into a descent through the 7 circles of hell...

I have numerous recurring problems now with Comfort that really must get resolved...

Firstly, I *still* am getting incessant instances of keypad text failures... on all 3 keypads I will constantly get \"Wrong text, press * 1 together\" I have also noticed that on at least one of them I have also seen on several occasions \"No text data, download aborted\" (or something very close to that). Moreover, on 2 of the keypads (ID\'s 2 & 3), when they are showing the \"Press * 1\" message, I can do so, and the text downloads, however, on ID 1, if I try to press \"* 1\" instead of a text download, I get the message \"Engineer Sign-in option is OFF!\" - why??? and why only on this keypad?

When a Keypad is showing the \"No text data, download aborted\" message, there\'s seemingly nothing I can press at the keypad to get out of that... I have to go & reset Comfort (is there perhaps a key combination that does a hardware reset at the keypad?)

In a previous thread I was advised to run the bus monitor to capture the data to look for (as has been suggested as a cause of this) data corruption on the bus... all well and good, but I have observed in doing this, that whenever I quit the bus monitor, I then cannot get a response from the UCM-ETH in Comfigurator, so I have no way out other than to go into the panel & hardware reset the UCM - Why does running the bus monitor leave the UCM in an unusable state when I exit it?

so right now, I have all 3 keypads that are unusable, and no way of resetting comfort without going into the panel to press the hardware reset buttons... this will not do!

So, dealing with this problem first.... I now have an extensive bus monitor capture file over the last few days which certainly covers numerous instances of this KP problem occurring... it doesn\'t make a lot of sense just looking at it, so what should I do with it? As a related note, is there any documentation/instructions on using the bus monitor tool? - I searched the forum and the Comfigurator help file, and could find nothing... there are various buttons & entry fields in this tool that I have no understanding of...

Also, I would like someone to explain the rationale between this behaviour and the suggestion of corruption on the comms bus... - I have had not a *single* instance of a communications failure alarm during this entire time? - surely data corruption ought to trigger a communications failure shouldn\'t it?. Secondly, I\'d like to understand the keypad text mechanism, because without that I\'m sceptical as to how/why a data corruption should/would cause this error... a keypad downloads the text from Comfort... I get that... but surely once it is downloaded & stored in the keypad, *WHY* would it subsequently get lost or corrupted? - presumably the KP stores the text locally? - I can visibly see it downloading a decent amount of data... where in the KP is this text data stored? - is it in volatile memory? - thus I would *expect* it to need to download again after a KP power-cycle, but none of my KP\'s have been, or are being power-cycled, so even if the above is true, I see no reason at all why a KP would need to download the text file again.

Presumably the text file is checksummed, and verified upon download to a keypad, and when verified is stored locally in the KP... surely once it\'s there, it\'s there?... what effect would a comms corruption on the bus have on data safely stored locally within a keypad?

Or, is it that the keypad reports this error if text to display is received from comfort that does not match the text stored locally in the keypad? *this* I could understand then triggering the behaviour, but still I have to wonder why I have never seen any wrong text displayed?... Comfort doesn\'t send actual text strings to the KP\'s does it? - I am assuming that the actual data on the bus would be the numbers from the wordlist? - so for example, if Comfort wants to tell all KP\'s to display, say \"Master Bedroom Movement\" - all words from the wordlist, it will not broadcast that actual string to the keypads, rather the *numbers* of those words from the wordlist, for instance: 199,37,6,121,255

That being the case, IF I were getting frequent corruptions of data, surely I would expect to see, at least occasionally, corruptions of the VALUES of those word numbers? - in which case I might expect to see some amusingly unexpected combinations of words, - incorrect, but VALID words, appearing on the LCD??... - I HAVE NEVER SEEN THIS!

Also, If I\'m having data corruptions on my RS485 bus so frequently that my keypads are going mental several times a day, why do I not see any other manifestations of this in other problems? - I never have a write of a full config file PC>Comfort fail for example? I never have a firmware upgrade fail for example (and I\'ve done firmware upgrades just recently). Nothing else that one would expect to be \'sensitive\' to data transmission integrity is showing any problems...

I\'ve seen enough extremely weird cause & effect combinations over my years in I.T, so I\'m wise enough to discount no possibility... but I just can\'t get my head around the logic relating the cause & effect in this situation... it may well be because my understanding of the communications mechanism to the keypads is way inadequate to make any informed determinations, but right now, it the suggestion of comms corruption just doesn\'t sit well with me...

Anyhoo, I do now have about 3 days worth of bus monitor data saved to file, so how may we proceed to analyse this to establish the voracity of this data corruption hypothesis once and for all?

I have one other problem I\'ve seen occur a couple of times just recently, but \"divide & conquer\" as the saying goes, so just getting this one sorted out is where I\'d like to concentrate first.

Thanks for listening!

Paul G.

 

 

[Ingo]

Paul,

I feel your pain, it\'s frustrating as hell, I know, but it\'s no joke.

To put it simply, the RS485 corruption was probably the most logical cause of the problem with the data you supplied previously. As you now provided more extensive data it points to perhaps a different cause but still doesn\'t rule out possible bus issues.

I never had issues like this and can only speculate but I would go to the second suggestion made in the previous post and remove all keypads except one, probably leave the closest one to Comfort and see what happens.

In the absence of evidence pointing to a specific problem I would turn this all around and say remove all external devices and add them back one by one. If you can manage once device per day I am sure you might find the culprit within a few days. But yes, if the bus has a problem it would manifest in the other connected devices as well.

Attach your capture so we can have a look and give us times/dates when particular things happened so we can try and figure out what actually happens on the inside.

Just to comment on the Bus Monitor, yes the code is sometimes a bit \'flakey\' and Cytech is probably going to flame me for saying this but it does work. When I have Bus Monitor running for a long while it sortoff slows down meaning data on the bus is captured up to a minute later after the action. Also, when you quit Bus Monitor it doesn\'t always exit on the UCM side. With this I mean a code is sent to the UCM to activate/deactivate the packet capture mode and sometimes it doesn\'t exit that mode. In a case like that you just reset the UCM. I normally use my UCM/USB for captures and UCM/Eth for control, or is that vice-versa, anyway you get the message.

Ingo

[Pgordon]

Just spent a little time scanning through the output from the bus monitor... incidentally, I note the file started only today, but I had left the tool running from at least yesterday, if not further back... - does the monitor restart the log at midnight each day? - and just \'dump\' any existing data captured from the previous day?

Anyway... there IS a section which seems to show something going haywire with one of the keypads... here:

11: 00: 00: 635 ----< 03 22 3F 32 36 39 44 44 39 02 <--->\"?269DD9
               The analog vale of Slave 02\'s input 26is 9D
11: 00: 05: 105 ----< 03 41 4F 30 30 36 44 02 <--->AO006D
               Request Keypad 01 backlight off
11: 00: 08: 154 ----< 03 41 39 38 33 02 <--->A983
                Request Reset for Keypad 01
11: 00: 08: 524 ----< 03 13 63 30 35 33 38 30 30 30 31 30 31 34 38 02 <--->c053800010148
               Send data to Ucm 03 RS232 port
11: 00: 08: 574 ----< 03 13 71 32 45 30 30 34 42 02 <--->q2E004B
               Ucm 03 reports counter update with counter 2E to value 00
11: 00: 08: 584 ----< 03 00 53 32 30 31 36 30 31 30 37 31 31 30 30 30 30 30 30 35 42 02 <--->/S20160107110000005B
               Set date and time
11: 00: 08: 654 ----< 03 00 67 46 45 30 37 35 44 32 32 34 30 30 31 30 30 44 31 02 <--->/gFE075D22400100D1
                Version information sent
11: 00: 08: 714 ----< 03 00 64 32 45 30 30 30 30 31 33 35 38 02 <--->/d2E00001358
               All Modules(or Mainboard)sets counter 2E\'s value
11: 00: 08: 879 ----< 03 41 7D 30 36 30 36 30 41 32 39 02 <--->A}06060A29
               Identity UCM command
11: 00: 08: 912 ----< 03 42 82 33 39 02 <--->B‚39
               Keypad 02 queries IVS info
11: 00: 09: 044 ----< 03 42 82 30 35 46 46 32 34 30 31 30 32 30 33 46 46 46 46 30 44 02 <--->B‚05FF24010203FFFF0D
               IVS info sent
11: 00: 09: 106 ----< 03 43 82 33 38 02 <--->C‚38
               Keypad 03 queries IVS info
11: 00: 09: 226 ----< 03 43 82 30 35 46 46 32 34 30 31 30 32 30 33 46 46 46 46 30 43 02 <--->C‚05FF24010203FFFF0C
               IVS info sent
11: 00: 09: 786 ----< 03 41 82 33 41 02 <--->A‚3A
               Keypad 01 queries IVS info
11: 00: 09: 896 ----< 03 41 82 30 35 46 46 32 34 30 31 30 32 30 33 46 46 46 46 30 45 02 <--->A‚05FF24010203FFFF0E
               IVS info sent
11: 00: 10: 226 ----< 03 41 3C 38 30 02 <--->A<80
                Request current security mode
11: 00: 10: 236 ----< 03 41 3C 30 30 30 32 37 45 02 <--->A<00027E
                Change to  security off by Keypad 01
11: 00: 10: 536 ----< 03 41 78 30 31 34 33 02 <--->Ax0143
               Download zone words command
11: 00: 10: 556 ----< 03 00 78 30 31 31 36 42 41 46 46 31 42 37 39 46 46 31 42 33 32 31 39 41 45 46 46 30 46 02 <--->/x0116BAFF1B79FF1B3219AEFF0F
               Download zone words command
11: 00: 10: 676 ----< 03 43 78 30 34 33 45 02 <--->Cx043E
               Download zone words command
11: 00: 10: 696 ----< 03 00 78 30 34 32 42 37 39 46 46 35 43 35 41 32 46 32 45 46 46 43 43 02 <--->/x042B79FF5C5A2F2EFFCC
               Download zone words command
11: 00: 10: 971 ----< 03 41 78 30 36 33 45 02 <--->Ax063E
               Download zone words command
11: 00: 11: 017 ----< 03 00 78 30 36 32 30 37 39 46 46 34 35 37 39 35 43 35 41 46 46 37 34 02 <--->/x062079FF45795C5AFF74
               Download zone words command
11: 00: 11: 111 ----< 03 43 78 30 38 33 41 02 <--->Cx083A
               Download zone words command
11: 00: 11: 131 ----< 03 00 78 30 38 46 46 30 45 32 45 37 39 46 46 30 32 42 41 46 46 30 46 02 <--->/x08FF0E2E79FF02BAFF0F
               Download zone words command
11: 00: 11: 351 ----< 03 22 3F 32 36 39 45 44 38 02 <--->\"?269ED8
               The analog vale of Slave 02\'s input 26is 9E
11: 00: 11: 431 ----< 03 41 78 30 42 33 39 02 <--->Ax0B39
               Download zone words command
11: 00: 11: 441 ----< 03 00 78 30 42 46 46 46 46 46 46 46 46 46 46 46 46 38 30 02 <--->/x0BFFFFFFFFFFFF80
               Download zone words command
11: 00: 11: 551 ----< 03 43 78 31 31 33 31 02 <--->Cx1131
               Download zone words command
11: 00: 11: 571 ----< 03 00 78 31 31 34 39 37 39 35 43 35 41 46 46 46 46 38 31 32 45 37 39 46 46 44 37 02 <--->/x1149795C5AFFFF812E79FFD7
               Download zone words command
11: 00: 11: 852 ----< 03 41 78 31 34 33 30 02 <--->Ax1430
               Download zone words command
11: 00: 11: 882 ----< 03 00 78 31 34 33 34 32 45 37 39 46 46 46 46 46 46 46 46 46 46 39 42 02 <--->/x14342E79FFFFFFFFFF9B
               Download zone words command
11: 00: 11: 983 ----< 03 43 78 31 39 32 39 02 <--->Cx1929
               Download zone words command
11: 00: 12: 003 ----< 03 00 78 31 39 46 46 46 46 46 46 46 46 46 46 46 46 46 46 46 46 37 34 02 <--->/x19FFFFFFFFFFFFFFFF74
               Download zone words command
11: 00: 12: 303 ----< 03 41 78 32 31 32 33 02 <--->Ax2123
               Download zone words command
11: 00: 12: 313 ----< 03 00 78 32 31 43 34 35 41 32 46 32 45 46 46 30 36 39 32 37 39 46 46 44 41 02 <--->/x21C45A2F2EFF069279FFDA
               Download zone words command
11: 00: 12: 423 ----< 03 43 78 32 33 31 46 02 <--->Cx231F
               Download zone words command
11: 00: 12: 453 ----< 03 00 78 32 33 30 36 39 34 37 39 46 46 34 33 37 39 46 46 32 35 30 36 37 39 46 46 46 32 02 <--->/x23069479FF4379FF250679FFF2
               Download zone words command
11: 00: 12: 753 ----< 03 41 78 32 36 31 45 02 <--->Ax261E
               Download zone words command
11: 00: 12: 763 ----< 03 00 78 32 36 46 46 30 36 39 33 37 39 46 46 34 33 33 32 31 39 41 45 46 46 31 34 02 <--->/x26FF069379FF433219AEFF14
               Download zone words command
11: 00: 12: 874 ----< 03 43 78 32 39 31 39 02 <--->Cx2919
               Download zone words command
11: 00: 12: 910 ----< 03 00 78 32 39 46 46 46 46 46 46 46 46 46 46 46 46 46 46 46 46 36 34 02 <--->/x29FFFFFFFFFFFFFFFF64
               Download zone words command
11: 00: 13: 253 ----< 03 41 78 33 31 31 33 02 <--->Ax3113
               Download zone words command
11: 00: 13: 283 ----< 03 00 78 33 31 46 46 46 46 46 46 46 46 46 46 46 46 46 46 46 46 35 43 02 <--->/x31FFFFFFFFFFFFFFFF5C
               Download zone words command
11: 00: 13: 393 ----< 03 43 78 33 39 30 39 02 <--->Cx3909
               Download zone words command
11: 00: 13: 403 ----< 03 00 78 33 39 46 46 46 46 46 46 46 46 46 46 46 46 46 46 46 46 35 34 02 <--->/x39FFFFFFFFFFFFFFFF54
               Download zone words command
11: 00: 13: 753 ----< 03 41 78 34 31 30 33 02 <--->Ax4103
               Download zone words command
11: 00: 13: 783 ----< 03 00 78 34 31 46 46 46 46 46 46 46 46 46 46 46 46 46 46 46 46 34 43 02 <--->/x41FFFFFFFFFFFFFFFF4C
               Download zone words command
11: 00: 13: 903 ----< 03 43 78 34 39 46 39 02 <--->Cx49F9
               Download zone words command
11: 00: 13: 923 ----< 03 00 78 34 39 46 46 46 46 46 46 46 46 46 46 46 46 46 46 46 46 34 34 02 <--->/x49FFFFFFFFFFFFFFFF44
               Download zone words command
11: 00: 14: 203 ----< 03 41 78 35 31 46 33 02 <--->Ax51F3
               Download zone words command
11: 00: 14: 223 ----< 03 00 78 35 31 46 46 46 46 46 46 46 46 46 46 46 46 46 46 46 46 33 43 02 <--->/x51FFFFFFFFFFFFFFFF3C
               Download zone words command
11: 00: 14: 363 ----< 03 43 78 35 39 45 39 02 <--->Cx59E9
               Download zone words command
11: 00: 14: 373 ----< 03 00 78 35 39 46 46 46 46 46 46 46 46 46 46 46 46 46 46 46 46 33 34 02 <--->/x59FFFFFFFFFFFFFFFF34
               Download zone words command
11: 00: 14: 653 ----< 03 41 78 36 31 45 33 02 <--->Ax61E3
               Download zone words command
11: 00: 14: 673 ----< 03 00 78 38 35 02 <--->/x85
               Download zone words command
11: 00: 14: 763 ----< 03 43 78 36 31 45 31 02 <--->Cx61E1
               Download zone words command
11: 00: 14: 783 ----< 03 00 78 38 35 02 <--->/x85
               Download zone words command
11: 00: 15: 034 ----< 03 41 7E 33 45 02 <--->A~3E
               Software error kindly note down string data and contact with us
11: 00: 15: 042 ----< 03 41 7E 30 30 33 45 02 <--->A~003E
               Engineer code disabled
11: 00: 15: 340 ----< 03 41 41 37 42 02 <--->AA7B
                Request Keypad 01
11: 00: 15: 360 ----< 03 41 41 30 30 30 31 30 30 30 30 37 41 02 <--->AA000100007A
                Request Keypad 01 turn off LED1,turn on LED2, turn off LED3, turn off LED4
11: 00: 15: 670 ----< 03 41 8F 32 44 02 <--->A2D
               Keypad 01 requests disable alarm info status
11: 00: 15: 680 ----< 03 41 8F 30 30 32 44 02 <--->A002D
               CM request to restore alarm information
11: 00: 16: 000 ----< 03 41 67 35 35 02 <--->Ag55
               Keypad 01 requests version information
11: 00: 16: 010 ----< 03 41 67 46 45 30 37 35 44 32 32 34 30 30 31 30 30 39 30 02 <--->AgFE075D2240010090
                Version information sent

Does that mean anything useful? -

11: 00: 15: 034 ----< 03 41 7E 33 45 02 <--->A~3E
               Software error kindly note down string data and contact with us


Thanks

Paul G.

 

[Ingo]

Doesn\'t look right. Let\'s see what Cytech says as I\'ve not seen this before. The keypad in question is ID 1, perhaps remove that one first and see if it helps.

The command looks to be corrupted though, if I read this correctly there is no reason to enable Engineer code and if so the value is wrong. Something obviously happened at this point.

[Pgordon]

Thanks Ingo, useful to know it\'s not just me with the bus monitor UCM lockup thing... Perhaps I too should look at getting an additional UCM so I don\'t get bitten by it... with only a single UCM-ETH that I use for all connection to comfort, it does leave me completely stymied if/when it locks up... - At least it\'s a self-installed panel, so I can go & remove the lid & hardware reset... :-)

In the last week or two, I have added additional devices into the environment: 1 new keypad in the master bedroom - this is the furthest away from the main panel... it takes 12V/COM, KA/KB from slave panel 2, - and since voice/mic aren\'t available at a slave, these two are routed all the way back to the master panel on a different floor... keypad to slave 2 cable run is probably about 15 metres (ish), and the additional run for voice/mic back to the master panel is perhaps another 8-10 metres. The voice quality on this remotest keypad is fine (I doubled up 2-cores for the voice line to minimise losses). Since this is the furthest away, and most recent addition, I will indeed disconnect that keypad tomorrow, and run for a few days & observe...

The other recent addition is a Zwave UCM. that\'s inside the main panel, connected by the short 4-way white cable.

I\'ll remove only one device at a time I think... and observe behaviour after each change...

Will report observations back in this thread in the near future!

Cheers.

Paul G.

 

 

[Ingo]

Cool, keep us posted as this is an interesting fault.

[Pgordon]

Hmm...

Perhaps worth mentioning that keypad 1 is the main one by the main entry/exit door & as such is the one used for arming/disarming to/from away mode. This was the first KP installed, is very close to the master panel (about 4 - 5 metre cable run) and is using good quality foil screened cable that does not travel close to any obvious source of interference.

Even back when that was first installed, and was the only keypad in the system, I did see an occasional problem with the LCD just going completely blank, and for that reason alone, my hourly response includes a \"reset Keypad 1\" which is the activity you can see in that snip from the bus monitor...

The other possibly slightly uncommon activity related to this keypad is that I have an ON response for a nearby PIR that turns the LCD backlight on, as I found that I could not read the LCD, even in broad daylight, without the backlight, and having seen a few occurrences of the aforementioned blank LCD problem, I was minded to keep an eye on it every time I walked past...

 

[Pgordon]

Just for completeness, here\'s a grab of my keypad info from the modules page in Comfigurator:

 

[Pgordon]

In other news, I\'ve just observed something else relating to one of the other problems I alluded to above, but have not described yet... but because I just had cause to hardware reset the UCM-ETH, I think I have narrowed down the cause....

Ever since Dec 31st, my Comfort system *INSISTS* that the date is one day behind reality.. i.e. right now, I can see the following:

> DT
< DT2016010719173400


See, it says the date today is the 7th... well, it isn\'t, its the 8th... this has been going on ever since the new year... at first, I suspected invalid time crossing over from CBUS, as I have experienced that before, - although it was only ever the hour of DST that was \'corrupted\' by CBUS, never the actual date... Also, I have always taken steps to prevent CBUS timekeeping bleeding over into Comfort (disabled clock/timekeeping setting at the CBUS UCM). for the last few days I\'ve been selectively isolating parts of CBUS from Comfort (powered down the Wiser for a day or two, unhooked the CBUS cable from the UCM etc.)

I can reset the time from the PC in Comfigurator, and it does seem to \'stick\' - the last time I corrected the date in Comfort was 2-3 days ago and it has been correct ever since... However, I have a keypad right next to the panel where the UCM-ETH is housed... I just had to go & hardware reset that UCM... and *IMMEDIATELY* I observed the date displayed on the keypad LCD changed in front of my eyes from the 8th to the 7th...

This is completely reproducible...

> DT20160108192942


The time is now correct in Comfort and displayed on all keypads correctly... Wait one minute whilst I now walk to the panel & reset the UCM-ETH via the button...

....

Bingo!

> DT
< DT2016010719345800


The UCM-ETH is screwing up Comfort\'s date by exactly 1 day.... the time remains correct....

below are the SNTP settings on the ETH module:

Note that the \"local time\" shown in the ETH module remains correct at all times...

I increased the frequency of the SNTP update to Comfort to 10 minutes a few days ago as part of t-shooting this problem...

Note that at no time after the UCM makes the date wrong does it correct itself, it will remain steadfastly 1 day behind reality until I manually reset it through Comfigurator...

What gives?

Why is the UCM-ETH giving the wrong date to Comfort, despite the fact that Server Manager shows the correct date received from SNTP? - Why does it not then correct it 10 minutes later as the settings would seem to suggest it should do?

Paul G.

 


 

[Ingo]

Leap year issue, upgrade to the latest Eth03 firmware.