homematic mqtt | Ingmar's Retroblog

A smarthome is no longer a rarity today and is very widespread. There are countless systems on the market that make your own home “smart”. The digital voice assistants from Google, Amazon and co. in conjunction with smart light bulbs are among the systems that are easy and quick to install. But there are also complex smart home systems, in which actuators for every lamp and socket are installed in the house distributors. The windows and doors are equipped with signaling contacts and secure the home or report if once forgotten to close the windows after shock ventilation. It goes without saying that these systems also contribute to energy optimization when programmed sensibly. I also operate Smarthome components from various manufacturers.

For years, this has included the HomeMatic system, which communicates with its actuators and sensors both wired and via the Bidcos protocol. The HUE system from Phillips talks to its smart lamps and sockets via ZigBee. The gateways of these systems are connected to a LAN network and each system brings its own web server, through which it can then be controlled and set. An inverter of photovoltaic systems can provide its data via different interfaces (RS485, CAN, RS232). To bring all of them to a central display level, I decided to use the NodeRed system. The necessary NodeRed server runs on a Raspberry PI. (On the CCU3 with the Raspbian image is still enough space to run the NodeRed server – it is even available as a separate plugin for the CCU and is called “RedMatic”). With this configuration you can “slay” almost everything in the field of home automation. With ESP32 and Raspberry you can easily transfer status information via MQTT (Message Queueing Telemetry Transport). I use this for example with the small feed-in inverters of a balcony PV system, as well as with the PV inverters of an offgrid system. Here the data is received via different bus systems in the Raspberry or ESP32 and converted into the MQTT protocol. The MQTT broker collects the data from the individual devices and via NodeRed they can then be written to a database, visualized in the browser or on the smartphone and also easily processed in the HomeMatic system, as required.

Thus, it is possible to network almost all systems with each other smartly and, importantly for me, to visualize them on ONE platform. One single system was missing until now. That is my old Neura heating heat pump. The company Neura has not existed for several years and the web server “webidalog” developed by “b.i.t.” has never been updated. So the heat pump has a web server on a small with Linux computer onboard and builds the web application with an ancient Java version. For the operation a Java Runtime must be installed on the PC, which runs only with some tricks on a current Windows computer (keyword: virtualization). For the operation via a smartphone an html – version with limited functionality is available. My plan now was to find an interface, with which I can read out the data of the heat pump at least once, in order to have flow- return temperatures of the floor heating, boiler temperature, etc. also available in my NodeRed system. But since there is almost no documentation for the system and reverse engineering is a bit critical if the system should continue to run, I came up with the following idea:

With a “headles browser” it should be possible to parse the html version of the Neura WebDialog website and find the relevant data and turn it into MQTT topics via variables. And here I have to give a special thanks to my colleague Mario Wehr, who built the software structure to parse the website. The software is written in PHP and finally runs on a Raspberry PI. All you need is a php8-cli runtime and a few modules. The way the software works is that every time the heat pump website is called, a login is executed, then the data is parsed and sent to MQTT broker. The continuous calling of the php script I then simply solved with a cronjob that is executed every minute.