Since I’ve been working on home automation, I’ve naturally wanted to optimize and simplify as much as possible and adapt and implement it in line with the new buzzwords “green electronics”, “sustainability”, “energy-saving” … and so on. For example, my appliances switch off when they are not used or ignored, stand-by energy consumption is largely avoided and IOT technology also prevents human forgetfulness (leaving windows open in winter or forgetting to switch lights off). As readers of the blog already know, I use systems such as HomeMatic, NodeRed and, for some time now, Homeassistant with ESPHome, Zigbee2Mqtt etc. Of course, the aim is also to keep all systems cloud-free. I don’t want the data to take a detour via some server in the Far East to switch a light on and off in my home. So, if possible, everything should take place within my own network and not “phone” to the outside and also work if I cut the data line.
For a long time now, various suppliers have been offering an extremely practical device for comfort in the parents’ quiet room. I’m talking about a space-saving way of accommodating the flicker box (nowadays also known as a flat-screen TV) in the room. I’m just mentioning terms like:
Speaka Professional TV ceiling mount electric motorized (1439178) or MyWall HL46ML … etc. Some of these devices can be controlled with a wireless remote control, others via the Tuya CloudApp. You can bypass the Tuya app via the Tuya IOT development environment and bring these devices into your home assistant via the “TuyaLocal” integration – it works – but it’s more of a “ONLY” solution. In my opinion, the ideal solution is to integrate these devices into the ESPHome system. Using the Speaka Professional TV ceiling mount as an example, I will show you how it can be integrated into the ESPHome network and thus into the Home Assistant with a small extension. This version of the SpeaKa part has no Internet connection and is only controlled via a wireless remote control.
TV ceiling mount with open cover
With a little reverse engineering, we (my colleague Werner and myself) analyzed the existing appliance factory. The system is structured something like this:
Circuit board in the ceiling bracket
Systemdiagramm
The system diagram above shows how the circuit board is constructed. The power supply comes from a plug-in power supply with DC 24V output at 1.5A. On the board you can still see an unpopulated area whose solder pads are wired with +3V3, GND and RX, TX lines suitable for an ESP8266. A USB socket can also be seen. These two interfaces are not included in the diagram. We examined the RX/TX lines that are routed from the unpopulated solder pads (ESP8266) to the microcontroller (1301 X 016B). However, no signals could be measured here. (Presumably the interface is not activated in the flashed program version).
“Debug” wires on the RX/TX and on the RF chip
So this does not take us any further. In the next step, we looked at where the control signals of the radio remote control come from and how they are subsequently implemented. The RF receiver chip has 16 pins and unfortunately no labeling. Or has it been removed? The supply voltage of the RF chip is connected to pin 1 and pin 16, pin 2 and pin 3 are connected to a crystal and a line is routed from pin 9 to the microcontroller. So this must be the data output. Using the “PulseView” software from Sigrok and a Far East logic analyzer, we sniffed this output. And lo and behold, data packets with a duration of 10.3ms were revealed here. The PulseView software was able to recognize the protocol as an RS232 protocol after a few attempts with different analyzed data rates. It was then easy to log the received and decoded control commands to the microcontroller.
RF chip with connected “sniffer” cable
The baud rate of the RS232 port on the RF chip output is 9600 at 8N1. 10 bytes are received in HEX for each command sent. Here is the list of commands: (missing bytes follow…maybe sometime)
Befehl
Byte0
Byte1
Byte2
Byte3
Byte4
Byte5
Byte6
Byte7
Byte8
Byte9
UP
0xAA
0x06
0x04
0x25
0x03
0xD5
0x01
0x00
0x02
0x55
DOWN
0xAA
0x06
0x04
0x25
0x03
0xD5
0x00
0x10
0x11
0x55
LEFT
0xAA
0x06
0x04
0x25
0x03
0xD5
0x55
RIGHT
0xAA
0x06
0x04
0x25
0x03
0xD5
0x55
BUTTON1
0xAA
0x06
0x04
0x25
0x03
0xD5
0x55
BUTTON2
0xAA
0x06
0x04
0x25
0x03
0xD5
0x00
0x08
0x09
0x55
MEM1
0xAA
0x06
0x04
0x25
0x03
0xD5
0x55
MEM2
0xAA
0x06
0x04
0x25
0x03
0xD5
0x55
OK
0xAA
0x06
0x04
0x25
0x03
0xD5
0x00
0x40
0x41
0x55
SET
xx
xx
xx
xx
xx
xx
xx
xx
xx
xx
Once the data protocol had been found using the logic analyzer, we tried to send the data to the microcontroller using a terminal program and a USB to TTL232 converter. The RF chip was removed for this purpose. It pulled the level to VCC in the idle state and prevented parallel operation of the “RS232 transmitter”.
RF-Chip removedBoard without chip with debug line
USB UART for sending commands
The control commands from the table above could be successfully sent via the terminal program. Now only an ESP32 board had to take over this task. An ESP32 NodeMCU board from the pool was equipped with a basic ESPHome image and integrated into the Homeassistant network. The ESPHome node now only had to be taught to send the byte sequence via the TX pin of the ESP32 when the corresponding trigger was activated in the Homeassistant. To do this, the ESP32 board was attached to the PCB and the VCC3V3, GND and TX lines were soldered to PIN9 of the former RF chip.
ESP32 on the board of the Speaka ceiling bracket
Re-installed in the ceiling bracket
The following esphome script must now be added to the ESPHome web environment.
esphome:
name: tvhalterung
friendly_name: TVHalterung
esp32:
board: esp32dev
framework:
type: arduino
# Enable logging
logger:
# Enable Home Assistant API
api:
encryption:
key: "hier dein key beim Anlegen des device"
ota:
password: "hier dein ota password"
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
# Enable fallback hotspot (captive portal) in case wifi connection fails
ap:
ssid: "Tvhalterung Fallback Hotspot"
password: "hier wieder deins"
captive_portal:
uart:
tx_pin: 4
rx_pin: 5
baud_rate: 9600
# Example button configuration
button:
- platform: template
name: TV Halterung UP
id: tv_up
icon: "mdi:arrow-up-bold-outline"
on_press:
- logger.log: "Button pressed TV Up"
- uart.write: [0xAA,0x06,0x04,0x25,0x03,0xD5,0x01,0x00,0x02,0x55]
- platform: template
name: TV Halterung OK
id: tv_ok
icon: "mdi:stop-circle-outline"
on_press:
- logger.log: "Button pressed TV OK"
- uart.write: [0xAA,0x06,0x04,0x25,0x03,0xD5,0x00,0x40,0x41,0x55]
- platform: template
name: TV Halterung DOWN
id: tv_down
icon: "mdi:arrow-down-bold-outline"
on_press:
- logger.log: "Button pressed TV Down"
- uart.write: [0xAA,0x06,0x04,0x25,0x03,0xD5,0x00,0x10,0x11,0x55]
- platform: template
name: TV Halterung Button1
id: tv_button1
icon: "mdi:numeric-1-circle-outline"
on_press:
- logger.log: "Button pressed TV Button1"
- uart.write: [0xAA,0x06,0x04,0x25,0x03,0xD5,0x00,0x20,0x21,0x55]
- platform: template
name: TV Halterung Button2
id: tv_button2
icon: "mdi:numeric-2-circle-outline"
on_press:
- logger.log: "Button pressed TV Button2"
- uart.write: [0xAA,0x06,0x04,0x25,0x03,0xD5,0x00,0x08,0x09,0x55]
- platform: template
name: TV Halterung Left
id: tv_left
icon: "mdi:arrow-left-bold-outline"
on_press:
- logger.log: "Button pressed TV Left"
- uart.write: [0xAA,0x06,0x04,0x25,0x03,0xD5,0x00,0x20,0x21,0x55]
- platform: template
name: TV Halterung Right
id: tv_right
icon: "mdi:arrow-right-bold-outline"
on_press:
- logger.log: "Button pressed TV Right"
- uart.write: [0xAA,0x06,0x04,0x25,0x03,0xD5,0x00,0x20,0x21,0x55]
- platform: template
name: TV Halterung MEM1
id: tv_mem1
icon: "mdi:alpha-m-circle-outline"
on_press:
- logger.log: "Button pressed TV MEM1"
- uart.write: [0xAA,0x06,0x04,0x25,0x03,0xD5,0x00,0x01,0x02,0x55]
- platform: template
name: TV Halterung MEM2
id: tv_mem2
icon: "mdi:alpha-m-circle-outline"
on_press:
- logger.log: "Button pressed TV MEM2"
- uart.write: [0xAA,0x06,0x04,0x25,0x03,0xD5,0x00,0x01,0x02,0x55]
Once the esphomescript has been compiled and uploaded to the ESP, there is a new ESPHome device with the name TV holder in the Home Assistant environment. The buttons for the control are now listed here as entities. If everything went well, you should now be able to control the TV mount via the Home Assistant.
(Not all control commands have been implemented correctly yet – the correct codes will be added to the table)
In the article “Pylontech PV battery status in HomeAssistant”, I had improved the project “Pylontech battery monitoring” of the following GitHub links and drew a circuit board to make the whole construct a little more compact and professional. https://github.com/irekzielinski/Pylontech-Battery-Monitoring https://github.com/hidaba/PylontechMonitoring
All battery data of the Pylontech battery modules are displayed in the Homeassistant. Great! But when I take a look at the list of devices registered in my wifi networks, I almost feel sick – there are now far too many wireless devices, especially from the smart home sector, sharing the channel bandwidth. So my current plan is to bring some of the smart home devices onto the wired LAN network.
The self-made devices based on the ESPs are ideal for this. These are devices such as the OpenDTU interface, the EVU smart home interface or, as here, the interface from the serial console of the Pylontech battery to the MQTT server in the Home Assistant.
I have done some tests with boards such as the OLIMEX ESP32-PoE and the WT32-ETH01. The Olimex board would have the great advantage of also being able to be supplied with power via PoE. However, the power supply for PoE operation is so “poor” that the required standards of external boards are not met. Here I can mention the NRF24L01 radio module, for example. I did some tests with it and decided to disregard the PoE functionality for the time being. This led to the plan to use the WT32-ETH01 with an ESP32 to design a universal board with several interfaces. It should be able to do the following:
communicate with the PV inverters using OpenDTU and NRF24L01
communicate with the Smarthome system via the Pylontech Console using MQTT
have an optional CAN interface
be able to communicate via RS422/RS485 in addition to the RS232 interface
receive the power supply via 5V USB
and to have everything packed nicely small and compact in one housing
So I designed a circuit and drew a circuit board. I got the boards manufactured by a Far East PCB manufacturer. The assembly is also done quickly.
Circuit diagram of the Universal Lan Interface
The picture below shows the PCB layout before production.
The WT32-ETH01 board does not have a USB port for programming the controller. It is programmed via an external USB-UART adapter. To activate the programming mode, an IO pin must also be connected to GND. To simplify this somewhat, there is now a “PROG” jumper on the board. If this jumper is plugged in, the WT32 can receive the firmware files. I have provided a pin header slot “TO-FTDI” as a connection option for the USB-UART adapter.
The board is now designed that it can be used to operate different devices. If you connect an NRF24L01 module to the “NRF24L01+” pin header and flash the ESP32-OpenDTU image to the controller, the inverter data can be received and transmitted via the LAN network. I have created a suitable IO-config jason-file for the use of the WT32.
Another application is the use of the board with the serial output of the battery data of the Pylontech PV batteries. The batteries provide a “Console” port which represents an RS232 interface. The data is transferred to the WT32 controller via this port and is then available via LAN in the local network.
I have adapted the ESP8266 script from hidaba and irekzielinski for the ESP32 controller. (see code at the end of the article)
Once the code has been compiled and uploaded, the status of the Pylontec batteries should be visible under the set IP address after all connections have been made.
Board version with Pylontech setupVersion with OpenDTU and NRF24L01 setup
The device setups shown in the picture are equipped with a housing. I have published the “.stl” files created with FreeCad on thingiverse.
“Freshly mopped floors without having to vacuum beforehand: The FC 7 Cordless hard floor cleaner removes all types of dry and damp everyday dirt in one step.” (Original text kaercher.com)
You get this product promise on the manufacturer’s website if you are interested in the FC7 electric hard floor cleaner. However, when this promise is no longer kept, I find out about the existence of these appliances. Because then I am asked to check why something is no longer working as it should. This is also the case here. The brushes (rollers – whatever these parts are called) no longer rotate, according to the problem description. Or to be more precise, they only turn sometimes when the bottom part of the moving handle is in a certain position. And since the handle (in which all the electronics such as batteries, BMS and operating elements are housed) can be moved within a wide range, it is reasonable to assume that there is a cable break or similar contact problem.
This is not exactly a complex problem, but perhaps one or the other is interested in how the problem can be solved with more or less effort.
The first step is to remove the wastewater tank and the four cleaning rollers. The screws on the drive cover and battery cover can then be loosened and the covers removed.
Screws of the drive coverBattery/electronics cover screws
Once the covers have been loosened, they can be removed. The circuit board with the BMS and the control electronics of the device can be seen under the battery cover. The 18650 Li-Ion cells are located underneath. The outlets to the bottom drive, to the control unit in the handle, etc. are plugged in.
Circuit board with BMS and control unit
The eight-pin plug at the bottom left of the picture must be disconnected. It connects the brush drive to the electronics. Six of the eight pins of the plug are occupied. One red and one black wire are used to supply the DC motor (yes, only a DC brush motor has been installed here and not a brushless one …) and two brown wires are laid to the pins that form the resistance sensor for the water level in the dirty water tank. Two blue wires control the solenoid valve of the water inlet.
As the fault is in the motor drive (depending on the position of the handle, the motor may or may not turn), the fault may be in the cable connection from the circuit board to the motor. The fault was quickly discovered with the continuity test of the multimeter. The black cable to the motor was broken.
Broken cable (black wire to DC motor)
The breaking point is exactly in the area where the handle is movably attached to the floor unit. This is exactly where the wiring harness and the rubber hose for the water guide are inserted. Constantly moving the cable harness will inevitably damage and break the cables in the long term. Especially if the handle is used at very shallow angles, for example to clean the floor under boxes, chests of drawers, etc.
Wire soldered and insulated with heat-shrink tubing
I did the repair here by soldering the wire together and protecting it with heat-shrink tubing. I wrapped the damaged cable protection conduit with insulating tape. This should hold for some time. As the part will not last forever due to its design, the wiring harness should be completely replaced during the next repair. (as this is probably not available as a spare part, you will probably have to make one yourself – but then with more stable, highly flexible wires…)
The components could now be reassembled. Place the rollers in the green/blue colors on the drive hubs and screw everything back together.
Broken cable connections and torn toothed belts in the motor unit are obviously the most common faults with this appliance.
II am always fascinated by the topic of radioactivity. More precisely, it is the measurement or detection of this ionizing radiation, which is produced by the decay and of atomic nuclei with the release of energy. A basic distinction is made between the energy (alpha and beta particles) emitted by the movement of the decaying particles (i.e. particle radiation) and the radiation energy that is transported as an electromagnetic wave (gamma radiation and also X-rays). These types of radiation have different energy densities and ranges. Depending on the type, they are more or less easy to shield. Alpha radiation is particle radiation that is strongly slowed down by matter (air, water) and no longer penetrates a sheet of paper. However, these particles give off the energy over their very short distance. This is particularly dangerous if these particles are inhaled or radiate on the upper layers of the skin. Gamma radiation in turn penetrates matter very easily like a radio wave and can be shielded most effectively with lead. It goes without saying that this type of radiation is anything but harmless.
You cannot see, smell, taste or otherwise perceive this radiation directly, but the danger is still there. With relatively simple techniques, however, these decay processes can be made visible or audible and counted.
This has been done for a long time with a so-called counter tube or, thanks to modern technology, with semiconductors. A P-N junction is operated in reverse direction and the very small reverse current is measured with the exclusion of light (i.e. darkened). If high-energy radiation hits this P-N transition, the current flow is increased for a short time and can be detected.
Whenever the opportunity arises to get a detector very cheaply, I of course take it. So this time too. I had to look at a simple kit based on detection using a counter tube. The kit comes from the Far East and consists of a base board, an attached Arduino Nano and an LC display that is also attached.
All components required for detection are on the mainboard. This includes, among other things, the generation of high voltage for the counter tube, which is implemented using a simple boost converter circuit driven by a 555. To attach the counter tube to the mainboard, the designer of this board chose simple glass tube fuse holders. They don’t fit exactly, but they can be stretched so that they hold the counter tube firmly in place. Incidentally, the counter tube is a J305. It is approx. 90mm long and has a diameter of almost one centimeter.
The counter tube works with an anode voltage of 350V to 480V. Below I have listed the specifications from the data sheet:
Anode voltage: 350 v bis 480 V
Type: J305 Geiger-counter tube
Cathode material: tin oxide
Wall density: 50 ± 10 cg/cm²
Operating temperature range: -40 °C bis 50 °C
Diameter: 10 mm (±0,5 mm)
Length: 90 mm (±2 mm)
Self-background radiation: 0,2 pulses/s
Sensivity to γ-radiation: 0,1 MeV
Current consumption: 0,015 mA bis 0,02 mA
Working voltage: 380 V bis 450 V
γ-radiation: 20mR/h ~ 120mR/h
β-radiation: 100 ~ 1800 Pulse/min.
100 ~ 1800 pulses/min.
The signal detection and processing of the signal also takes place on the mainboard. The recognized impulses are reproduced via a small piezo loudspeaker. In order to be able to count them, you don’t have to sit in front of the loudspeaker with a stopwatch and count the beeps every minute – no – that is done by a microcontroller, which, as is common today, consists of a finished board. Here the designer has chosen an Arduino Nano (or nano replica). In turn, a program runs on it that counts the impulses and also shows them nicely on a two-line LC display and ideally also converts them into µSievert / h. To transfer the pulses to the Arduino, the level of the signal is brought to TTL level and switched to the interrupt input of the Arduino. The LC display uses the I2C output of the Arduino. The lines for this are only led from the socket strip into which the Arduino is plugged via the mainboard to the socket strip for the display. To supply the whole system with voltage, the 5V from the USB port of the Arduino are used directly. Optionally, the 5V can also be connected to the mainboard via a connector strip.
Once everything has been assembled and the USB supply is connected, there is first of all a short waiting time during which the high voltage is built up. Here the programmer has come up with an animation that shows “Boot …” on the display.
And then it starts. The Geiger counter is ready for use and begins to count. As a test I only have an old clock with hands painted with radium paint. There is at least a clear change in the number of detected counting pulses when the watch is brought near the counter tube.
Anyone who has installed a photovoltaic system in their own home may even use an energy storage system. In this example, it is an off-grid system equipped with two modules from the manufacturer Pylontech. The Pylontech US3000C batteries have an output voltage of 48V. The nominal capacity is 3500Wh. The installed cells are LiFePO4 cells and the usable capacity is specified as 3374Wh according to the data sheet. The batteries are designed to be connected in parallel with other Pylontech batteries. The internally installed BMS (battery management system) communicates with the other Pylontech battery modules via a so-called “link” interface. A battery configured as a “master” handles the data exchange with the inverter. Here, Pylontech provides the CAN or RS485 bus as an interface. However, if you want information about the individual cells (voltages, currents, charges, temperatures, etc.), there is another interface on each module called “Console”. This is an RS232 interface via which you can communicate directly with the battery’s BMS. This port is also used to update the firmware of the BMS. However, I STRONGLY advise against playing around with firmware updates and flash software. This is reserved for the manufacturer or the liable party.
However, as this interface also provides a lot of information about the cells installed in the battery, this is an interesting approach. Initially, I had a laptop connected to a terminal and was able to discover and monitor the individual cell voltages and, above all, the possibly different charge status of the modules connected in parallel. So I thought it would be a good idea to have this information available in my home automation system, where it could be visualized and used for control purposes.
As we geeks and technology enthusiasts are quite familiar with terms such as Homeassistant, Docker, Proxmox, HomeMatic, NodeRed etc., I thought that this data should also become entities in the Homeassistant. So a small new project was quickly created. My plan was to read the data from the serial interface and send it to the Home Assistant via MQTT.
But before I start disassembling the data strings that come out via the serial port, I’ll have a look at the search engines. Perhaps someone else has already dealt with this topic. And that’s exactly what happened. I found what I was looking for on GitHub under the term “pylontec2mqtt”. A project is hosted at https://github.com/irekzielinski/Pylontech-Battery-Monitoring that uses ESP8266 to collect the serial data from the port and sends it to the Homeassistant server via MQTT and Wifi. A fork with a further development of this project can be found at https://github.com/hidaba/PylontechMonitoring.
Why am I publishing the project here on the blog despite the simple replica? I have optimized the circuit a little and packed it into a layout and adapted the code a little. I would like to share the result here. It was important to me to have a sensible structure on a circuit board that is connected with a USB A-B cable for the power supply and a LAN-RJ45 cable for the data connection. I wanted to use a “solid” USB connector (not the fragile mini or micro USB connectors)
On a breadboard and with the usual development boards, I quickly “knitted together” a functional model so that I could adapt the software to it.
Functional sample on perforated grid
So I first created a circuit diagram from the sketches in the Git project. There is a “real” RS232 level at the “Console” interface, which is converted to a 5V TTL via the MAX3232 IC. The BSS123 FET is used to realize a level converter to 3.3V for each of the RX and TX signals.
pylontec2mqtt schematic
The ESP8266 processes this 3.3V TTL level in the form of the Wemos D1Mini or WemosD1Pro development board, which is plugged onto the circuit board. I then packed the entire construction into a small plastic housing, which can be conveniently connected to the Pylontec and a USB power source via the LAN and USB cables.
Layout preview in designtool
The layout design is shown in the picture above. The circuit board and the position of the components were checked again with the preview before production and then ordered from a trusted manufacturer.
Preview of the circuit board before production
After barely two weeks of waiting, I had the empty circuit boards in my hands and was able to fit them with the components.
fully assembled circuit board
The picture above shows the fully assembled board. The only thing missing here is the Wemos board with the ESP.
Comparison between functional model and first “production model”
In the end, I plugged in a WemosD1 Pro, as this offers the option of connecting an external WiFi antenna and thus getting a reasonable wireless range.
After flashing the software and commissioning, the Wemos web server can be accessed at the IP address specified in the code. Here you can also check whether the Pylontech battery is communicating with the Wemos. The result then looks like this.
Webseite of the WEMOS ESP
Here you can see that both battery modules are recognized correctly. The next step is to check whether messages are being sent via the MQTT protocol. The IP address of the MQTT broker must also be specified in the Wemo code. In my setup, I have set up the MQTT Explorer in the Home Assistant to be able to check the MQTT functions quickly and easily.
MQTT Explorer
The image above shows that the data also arrives correctly via MQTT. Now it is only necessary to create a sensor yaml file in the home assistant to make the topics available as entities. I have added the following code to configuration.yaml for this purpose:
On the Homeassistant website, the visualization could then look like this, for example:
Last but not least, I am posting the customized code below. The libraries required for compilation and further information can be found in the GitHub links above.
#include <ESP8266WiFi.h>
#include <ESP8266mDNS.h>
#include <ArduinoOTA.h>
#include <ESP8266WebServer.h>
#include <circular_log.h>
#include <ArduinoJson.h>
#include <NTPClient.h>
#include <ESP8266TimerInterrupt.h>
//+++ START CONFIGURATION +++
//IMPORTANT: Specify your WIFI settings:
#define WIFI_SSID "wifiname"
#define WIFI_PASS deinpasswort1234"
#define WIFI_HOSTNAME "mppsolar-pylontec"
//Uncomment for static ip configuration
#define STATIC_IP
IPAddress local_IP(192, 168, xxx, yyy);
IPAddress subnet(255, 255, 255, 0);
IPAddress gateway(192, 168, xxx, zzz);
IPAddress primaryDNS(192, 168, xxx, zzz);
//Uncomment for authentication page
//#define AUTHENTICATION
//set http Authentication
const char* www_username = "admin";
const char* www_password = "password";
//IMPORTANT: Uncomment this line if you want to enable MQTT (and fill correct MQTT_ values below):
#define ENABLE_MQTT
// Set offset time in seconds to adjust for your timezone, for example:
// GMT +1 = 3600
// GMT +1 = 7200
// GMT +8 = 28800
// GMT -1 = -3600
// GMT 0 = 0
#define GMT 3600
//NOTE 1: if you want to change what is pushed via MQTT - edit function: pushBatteryDataToMqtt.
//NOTE 2: MQTT_TOPIC_ROOT is where battery will push MQTT topics. For example "soc" will be pushed to: "home/grid_battery/soc"
#define MQTT_SERVER "192.168.xx.broker"
#define MQTT_PORT 1883
#define MQTT_USER ""
#define MQTT_PASSWORD ""
#define MQTT_TOPIC_ROOT "ingmarsretro/pylontec/" //this is where mqtt data will be pushed
#define MQTT_PUSH_FREQ_SEC 2 //maximum mqtt update frequency in seconds
//+++ END CONFIGURATION +++
#ifdef ENABLE_MQTT
#include <PubSubClient.h>
WiFiClient espClient;
PubSubClient mqttClient(espClient);
#endif //ENABLE_MQTT
//text response
char g_szRecvBuff[7000];
const long utcOffsetInSeconds = GMT;
char daysOfTheWeek[7][12] = {"Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"};
// Define NTP Client to get time
WiFiUDP ntpUDP;
NTPClient timeClient(ntpUDP, "pool.ntp.org", utcOffsetInSeconds);
ESP8266WebServer server(80);
circular_log<7000> g_log;
bool ntpTimeReceived = false;
int g_baudRate = 0;
void Log(const char* msg)
{
g_log.Log(msg);
}
//Define Interrupt Timer to Calculate Power meter every second (kWh)
#define USING_TIM_DIV1 true // for shortest and most accurate timer
ESP8266Timer ITimer;
bool setInterval(unsigned long interval, timer_callback callback); // interval (in microseconds)
#define TIMER_INTERVAL_MS 1000
//Global Variables for the Power Meter - accessible from the calculating interrupt und from main
unsigned long powerIN = 0; //WS gone in to the BAttery
unsigned long powerOUT = 0; //WS gone out of the Battery
//Global Variables for the Power Meter - Überlauf
unsigned long powerINWh = 0; //WS gone in to the BAttery
unsigned long powerOUTWh = 0; //WS gone out of the Battery
void setup() {
memset(g_szRecvBuff, 0, sizeof(g_szRecvBuff)); //clean variable
pinMode(LED_BUILTIN, OUTPUT);
digitalWrite(LED_BUILTIN, HIGH);//high is off
// put your setup code here, to run once:
WiFi.mode(WIFI_STA);
WiFi.persistent(false); //our credentialss are hardcoded, so we don't need ESP saving those each boot (will save on flash wear)
WiFi.hostname(WIFI_HOSTNAME);
#ifdef STATIC_IP
WiFi.config(local_IP, gateway, subnet, primaryDNS);
#endif
WiFi.begin(WIFI_SSID, WIFI_PASS);
for(int ix=0; ix<10; ix++)
{
Log("Wait for WIFI Connection");
if(WiFi.status() == WL_CONNECTED)
{
break;
}
delay(1000);
}
ArduinoOTA.setHostname(WIFI_HOSTNAME);
ArduinoOTA.begin();
server.on("/", handleRoot);
server.on("/log", handleLog);
server.on("/req", handleReq);
server.on("/jsonOut", handleJsonOut);
server.on("/reboot", [](){
#ifdef AUTHENTICATION
if (!server.authenticate(www_username, www_password)) {
return server.requestAuthentication();
}
#endif
ESP.restart();
});
server.begin();
timeClient.begin();
#ifdef ENABLE_MQTT
mqttClient.setServer(MQTT_SERVER, MQTT_PORT);
#endif
Log("Boot event");
}
void handleLog()
{
#ifdef AUTHENTICATION
if (!server.authenticate(www_username, www_password)) {
return server.requestAuthentication();
}
#endif
server.send(200, "text/html", g_log.c_str());
}
void switchBaud(int newRate)
{
if(g_baudRate == newRate)
{
return;
}
if(g_baudRate != 0)
{
Serial.flush();
delay(20);
Serial.end();
delay(20);
}
char szMsg[50];
snprintf(szMsg, sizeof(szMsg)-1, "New baud: %d", newRate);
Log(szMsg);
Serial.begin(newRate);
g_baudRate = newRate;
delay(20);
}
void waitForSerial()
{
for(int ix=0; ix<150;ix++)
{
if(Serial.available()) break;
delay(10);
}
}
int readFromSerial()
{
memset(g_szRecvBuff, 0, sizeof(g_szRecvBuff));
int recvBuffLen = 0;
bool foundTerminator = true;
waitForSerial();
while(Serial.available())
{
char szResponse[256] = "";
const int readNow = Serial.readBytesUntil('>', szResponse, sizeof(szResponse)-1); //all commands terminate with "$$\r\n\rpylon>" (no new line at the end)
if(readNow > 0 &&
szResponse[0] != '\0')
{
if(readNow + recvBuffLen + 1 >= (int)(sizeof(g_szRecvBuff)))
{
Log("WARNING: Read too much data on the console!");
break;
}
strcat(g_szRecvBuff, szResponse);
recvBuffLen += readNow;
if(strstr(g_szRecvBuff, "$$\r\n\rpylon"))
{
strcat(g_szRecvBuff, ">"); //readBytesUntil will skip this, so re-add
foundTerminator = true;
break; //found end of the string
}
if(strstr(g_szRecvBuff, "Press [Enter] to be continued,other key to exit"))
{
//we need to send new line character so battery continues the output
Serial.write("\r");
}
waitForSerial();
}
}
if(recvBuffLen > 0 )
{
if(foundTerminator == false)
{
Log("Failed to find pylon> terminator");
}
}
return recvBuffLen;
}
bool readFromSerialAndSendResponse()
{
const int recvBuffLen = readFromSerial();
if(recvBuffLen > 0)
{
server.sendContent(g_szRecvBuff);
return true;
}
return false;
}
bool sendCommandAndReadSerialResponse(const char* pszCommand)
{
switchBaud(115200);
if(pszCommand[0] != '\0')
{
Serial.write(pszCommand);
}
Serial.write("\n");
const int recvBuffLen = readFromSerial();
if(recvBuffLen > 0)
{
return true;
}
//wake up console and try again:
wakeUpConsole();
if(pszCommand[0] != '\0')
{
Serial.write(pszCommand);
}
Serial.write("\n");
return readFromSerial() > 0;
}
void handleReq()
{
#ifdef AUTHENTICATION
if (!server.authenticate(www_username, www_password)) {
return server.requestAuthentication();
}
#endif
bool respOK;
if(server.hasArg("code") == false)
{
respOK = sendCommandAndReadSerialResponse("");
}
else
{
respOK = sendCommandAndReadSerialResponse(server.arg("code").c_str());
}
handleRoot();
}
void handleJsonOut()
{
#ifdef AUTHENTICATION
if (!server.authenticate(www_username, www_password)) {
return server.requestAuthentication();
}
#endif
if(sendCommandAndReadSerialResponse("pwr") == false)
{
server.send(500, "text/plain", "Failed to get response to 'pwr' command");
return;
}
parsePwrResponse(g_szRecvBuff);
prepareJsonOutput(g_szRecvBuff, sizeof(g_szRecvBuff));
server.send(200, "application/json", g_szRecvBuff);
}
void handleRoot() {
#ifdef AUTHENTICATION
if (!server.authenticate(www_username, www_password)) {
return server.requestAuthentication();
}
#endif
timeClient.update(); //get ntp datetime
unsigned long days = 0, hours = 0, minutes = 0;
unsigned long val = os_getCurrentTimeSec();
days = val / (3600*24);
val -= days * (3600*24);
hours = val / 3600;
val -= hours * 3600;
minutes = val / 60;
val -= minutes*60;
time_t epochTime = timeClient.getEpochTime();
String formattedTime = timeClient.getFormattedTime();
//Get a time structure
struct tm *ptm = gmtime ((time_t *)&epochTime);
int currentMonth = ptm->tm_mon+1;
static char szTmp[9500] = "";
long timezone= GMT / 3600;
snprintf(szTmp, sizeof(szTmp)-1, "<html><b>Pylontech Battery</b><br>Time GMT: %s (%s %d)<br>Uptime: %02d:%02d:%02d.%02d<br><br>free heap: %u<br>Wifi RSSI: %d<BR>Wifi SSID: %s",
formattedTime, "GMT ", timezone,
(int)days, (int)hours, (int)minutes, (int)val,
ESP.getFreeHeap(), WiFi.RSSI(), WiFi.SSID().c_str());
strncat(szTmp, "<BR><a href='/log'>Runtime log</a><HR>", sizeof(szTmp)-1);
strncat(szTmp, "<form action='/req' method='get'>Command:<input type='text' name='code'/><input type='submit'> <a href='/req?code=pwr'>PWR</a> | <a href='/req?code=pwr%201'>Power 1</a> | <a href='/req?code=pwr%202'>Power 2</a> | <a href='/req?code=pwr%203'>Power 3</a> | <a href='/req?code=pwr%204'>Power 4</a> | <a href='/req?code=help'>Help</a> | <a href='/req?code=log'>Event Log</a> | <a href='/req?code=time'>Time</a><br>", sizeof(szTmp)-1);
//strncat(szTmp, "<form action='/req' method='get'>Command:<input type='text' name='code'/><input type='submit'><a href='/req?code=pwr'>Power</a> | <a href='/req?code=help'>Help</a> | <a href='/req?code=log'>Event Log</a> | <a href='/req?code=time'>Time</a><br>", sizeof(szTmp)-1);
strncat(szTmp, "<textarea rows='80' cols='180'>", sizeof(szTmp)-1);
//strncat(szTmp, "<textarea rows='45' cols='180'>", sizeof(szTmp)-1);
strncat(szTmp, g_szRecvBuff, sizeof(szTmp)-1);
strncat(szTmp, "</textarea></form>", sizeof(szTmp)-1);
strncat(szTmp, "</html>", sizeof(szTmp)-1);
//send page
server.send(200, "text/html", szTmp);
}
unsigned long os_getCurrentTimeSec()
{
static unsigned int wrapCnt = 0;
static unsigned long lastVal = 0;
unsigned long currentVal = millis();
if(currentVal < lastVal)
{
wrapCnt++;
}
lastVal = currentVal;
unsigned long seconds = currentVal/1000;
//millis will wrap each 50 days, as we are interested only in seconds, let's keep the wrap counter
return (wrapCnt*4294967) + seconds;
}
void wakeUpConsole()
{
switchBaud(1200);
//byte wakeUpBuff[] = {0x7E, 0x32, 0x30, 0x30, 0x31, 0x34, 0x36, 0x38, 0x32, 0x43, 0x30, 0x30, 0x34, 0x38, 0x35, 0x32, 0x30, 0x46, 0x43, 0x43, 0x33, 0x0D};
//Serial.write(wakeUpBuff, sizeof(wakeUpBuff));
Serial.write("~20014682C0048520FCC3\r");
delay(1000);
byte newLineBuff[] = {0x0E, 0x0A};
switchBaud(115200);
for(int ix=0; ix<10; ix++)
{
Serial.write(newLineBuff, sizeof(newLineBuff));
delay(1000);
if(Serial.available())
{
while(Serial.available())
{
Serial.read();
}
break;
}
}
}
#define MAX_PYLON_BATTERIES 8
struct pylonBattery
{
bool isPresent;
long soc; //Coulomb in %
long voltage; //in mW
long current; //in mA, negative value is discharge
long tempr; //temp of case or BMS?
long cellTempLow;
long cellTempHigh;
long cellVoltLow;
long cellVoltHigh;
char baseState[9]; //Charge | Dischg | Idle
char voltageState[9]; //Normal
char currentState[9]; //Normal
char tempState[9]; //Normal
char time[20]; //2019-06-08 04:00:29
char b_v_st[9]; //Normal (battery voltage?)
char b_t_st[9]; //Normal (battery temperature?)
bool isCharging() const { return strcmp(baseState, "Charge") == 0; }
bool isDischarging() const { return strcmp(baseState, "Dischg") == 0; }
bool isIdle() const { return strcmp(baseState, "Idle") == 0; }
bool isBalancing() const { return strcmp(baseState, "Balance") == 0; }
bool isNormal() const
{
if(isCharging() == false &&
isDischarging() == false &&
isIdle() == false &&
isBalancing() == false)
{
return false; //base state looks wrong!
}
return strcmp(voltageState, "Normal") == 0 &&
strcmp(currentState, "Normal") == 0 &&
strcmp(tempState, "Normal") == 0 &&
strcmp(b_v_st, "Normal") == 0 &&
strcmp(b_t_st, "Normal") == 0 ;
}
};
struct batteryStack
{
int batteryCount;
int soc; //in %, if charging: average SOC, otherwise: lowest SOC
int temp; //in mC, if highest temp is > 15C, this will show the highest temp, otherwise the lowest
long currentDC; //mAh current going in or out of the battery
long avgVoltage; //in mV
char baseState[9]; //Charge | Dischg | Idle | Balance | Alarm!
pylonBattery batts[MAX_PYLON_BATTERIES];
bool isNormal() const
{
for(int ix=0; ix<MAX_PYLON_BATTERIES; ix++)
{
if(batts[ix].isPresent &&
batts[ix].isNormal() == false)
{
return false;
}
}
return true;
}
//in Wh
long getPowerDC() const
{
return (long)(((double)currentDC/1000.0)*((double)avgVoltage/1000.0));
}
// power in Wh in charge
float powerIN() const
{
if (currentDC > 0) {
return (float)(((double)currentDC/1000.0)*((double)avgVoltage/1000.0));
} else {
return (float)(0);
}
}
// power in Wh in discharge
float powerOUT() const
{
if (currentDC < 0) {
return (float)(((double)currentDC/1000.0)*((double)avgVoltage/1000.0)*-1);
} else {
return (float)(0);
}
}
//Wh estimated current on AC side (taking into account Sofar ME3000SP losses)
long getEstPowerAc() const
{
double powerDC = (double)getPowerDC();
if(powerDC == 0)
{
return 0;
}
else if(powerDC < 0)
{
//we are discharging, on AC side we will see less power due to losses
if(powerDC < -1000)
{
return (long)(powerDC*0.94);
}
else if(powerDC < -600)
{
return (long)(powerDC*0.90);
}
else
{
return (long)(powerDC*0.87);
}
}
else
{
//we are charging, on AC side we will have more power due to losses
if(powerDC > 1000)
{
return (long)(powerDC*1.06);
}
else if(powerDC > 600)
{
return (long)(powerDC*1.1);
}
else
{
return (long)(powerDC*1.13);
}
}
}
};
batteryStack g_stack;
long extractInt(const char* pStr, int pos)
{
return atol(pStr+pos);
}
void extractStr(const char* pStr, int pos, char* strOut, int strOutSize)
{
strOut[strOutSize-1] = '\0';
strncpy(strOut, pStr+pos, strOutSize-1);
strOutSize--;
//trim right
while(strOutSize > 0)
{
if(isspace(strOut[strOutSize-1]))
{
strOut[strOutSize-1] = '\0';
}
else
{
break;
}
strOutSize--;
}
}
/* Output has mixed \r and \r\n
pwr
@
Power Volt Curr Tempr Tlow Thigh Vlow Vhigh Base.St Volt.St Curr.St Temp.St Coulomb Time B.V.St B.T.St
1 49735 -1440 22000 19000 19000 3315 3317 Dischg Normal Normal Normal 93% 2019-06-08 04:00:30 Normal Normal
....
8 - - - - - - - Absent - - - - - - -
Command completed successfully
$$
pylon
*/
bool parsePwrResponse(const char* pStr)
{
if(strstr(pStr, "Command completed successfully") == NULL)
{
return false;
}
int chargeCnt = 0;
int dischargeCnt = 0;
int idleCnt = 0;
int alarmCnt = 0;
int socAvg = 0;
int socLow = 0;
int tempHigh = 0;
int tempLow = 0;
memset(&g_stack, 0, sizeof(g_stack));
for(int ix=0; ix<MAX_PYLON_BATTERIES; ix++)
{
char szToFind[32] = "";
snprintf(szToFind, sizeof(szToFind)-1, "\r\r\n%d ", ix+1);
const char* pLineStart = strstr(pStr, szToFind);
if(pLineStart == NULL)
{
return false;
}
pLineStart += 3; //move past \r\r\n
extractStr(pLineStart, 55, g_stack.batts[ix].baseState, sizeof(g_stack.batts[ix].baseState));
if(strcmp(g_stack.batts[ix].baseState, "Absent") == 0)
{
g_stack.batts[ix].isPresent = false;
}
else
{
g_stack.batts[ix].isPresent = true;
extractStr(pLineStart, 64, g_stack.batts[ix].voltageState, sizeof(g_stack.batts[ix].voltageState));
extractStr(pLineStart, 73, g_stack.batts[ix].currentState, sizeof(g_stack.batts[ix].currentState));
extractStr(pLineStart, 82, g_stack.batts[ix].tempState, sizeof(g_stack.batts[ix].tempState));
extractStr(pLineStart, 100, g_stack.batts[ix].time, sizeof(g_stack.batts[ix].time));
extractStr(pLineStart, 121, g_stack.batts[ix].b_v_st, sizeof(g_stack.batts[ix].b_v_st));
extractStr(pLineStart, 130, g_stack.batts[ix].b_t_st, sizeof(g_stack.batts[ix].b_t_st));
g_stack.batts[ix].voltage = extractInt(pLineStart, 6);
g_stack.batts[ix].current = extractInt(pLineStart, 13);
g_stack.batts[ix].tempr = extractInt(pLineStart, 20);
g_stack.batts[ix].cellTempLow = extractInt(pLineStart, 27);
g_stack.batts[ix].cellTempHigh = extractInt(pLineStart, 34);
g_stack.batts[ix].cellVoltLow = extractInt(pLineStart, 41);
g_stack.batts[ix].cellVoltHigh = extractInt(pLineStart, 48);
g_stack.batts[ix].soc = extractInt(pLineStart, 91);
//////////////////////////////// Post-process ////////////////////////
g_stack.batteryCount++;
g_stack.currentDC += g_stack.batts[ix].current;
g_stack.avgVoltage += g_stack.batts[ix].voltage;
socAvg += g_stack.batts[ix].soc;
if(g_stack.batts[ix].isNormal() == false){ alarmCnt++; }
else if(g_stack.batts[ix].isCharging()){chargeCnt++;}
else if(g_stack.batts[ix].isDischarging()){dischargeCnt++;}
else if(g_stack.batts[ix].isIdle()){idleCnt++;}
else{ alarmCnt++; } //should not really happen!
if(g_stack.batteryCount == 1)
{
socLow = g_stack.batts[ix].soc;
tempLow = g_stack.batts[ix].cellTempLow;
tempHigh = g_stack.batts[ix].cellTempHigh;
}
else
{
if(socLow > g_stack.batts[ix].soc){socLow = g_stack.batts[ix].soc;}
if(tempHigh < g_stack.batts[ix].cellTempHigh){tempHigh = g_stack.batts[ix].cellTempHigh;}
if(tempLow > g_stack.batts[ix].cellTempLow){tempLow = g_stack.batts[ix].cellTempLow;}
}
}
}
//now update stack state:
g_stack.avgVoltage /= g_stack.batteryCount;
g_stack.soc = socLow;
if(tempHigh > 15000) //15C
{
g_stack.temp = tempHigh; //in the summer we highlight the warmest cell
}
else
{
g_stack.temp = tempLow; //in the winter we focus on coldest cell
}
if(alarmCnt > 0)
{
strcpy(g_stack.baseState, "Alarm!");
}
else if(chargeCnt == g_stack.batteryCount)
{
strcpy(g_stack.baseState, "Charge");
g_stack.soc = (int)(socAvg / g_stack.batteryCount);
}
else if(dischargeCnt == g_stack.batteryCount)
{
strcpy(g_stack.baseState, "Dischg");
}
else if(idleCnt == g_stack.batteryCount)
{
strcpy(g_stack.baseState, "Idle");
}
else
{
strcpy(g_stack.baseState, "Balance");
}
return true;
}
void prepareJsonOutput(char* pBuff, int buffSize)
{
memset(pBuff, 0, buffSize);
snprintf(pBuff, buffSize-1, "{\"soc\": %d, \"temp\": %d, \"currentDC\": %ld, \"avgVoltage\": %ld, \"baseState\": \"%s\", \"batteryCount\": %d, \"powerDC\": %ld, \"estPowerAC\": %ld, \"isNormal\": %s}", g_stack.soc,
g_stack.temp,
g_stack.currentDC,
g_stack.avgVoltage,
g_stack.baseState,
g_stack.batteryCount,
g_stack.getPowerDC(),
g_stack.getEstPowerAc(),
g_stack.isNormal() ? "true" : "false");
}
void loop() {
#ifdef ENABLE_MQTT
mqttLoop();
#endif
ArduinoOTA.handle();
server.handleClient();
//if there are bytes availbe on serial here - it's unexpected
//when we send a command to battery, we read whole response
//if we get anything here anyways - we will log it
int bytesAv = Serial.available();
if(bytesAv > 0)
{
if(bytesAv > 63)
{
bytesAv = 63;
}
char buff[64+4] = "RCV:";
if(Serial.readBytes(buff+4, bytesAv) > 0)
{
digitalWrite(LED_BUILTIN, LOW);
delay(5);
digitalWrite(LED_BUILTIN, HIGH);//high is off
Log(buff);
}
}
}
#ifdef ENABLE_MQTT
#define ABS_DIFF(a, b) (a > b ? a-b : b-a)
void mqtt_publish_f(const char* topic, float newValue, float oldValue, float minDiff, bool force)
{
char szTmp[16] = "";
snprintf(szTmp, 15, "%.2f", newValue);
if(force || ABS_DIFF(newValue, oldValue) > minDiff)
{
mqttClient.publish(topic, szTmp, false);
}
}
void mqtt_publish_i(const char* topic, int newValue, int oldValue, int minDiff, bool force)
{
char szTmp[16] = "";
snprintf(szTmp, 15, "%d", newValue);
if(force || ABS_DIFF(newValue, oldValue) > minDiff)
{
mqttClient.publish(topic, szTmp, false);
}
}
void mqtt_publish_s(const char* topic, const char* newValue, const char* oldValue, bool force)
{
if(force || strcmp(newValue, oldValue) != 0)
{
mqttClient.publish(topic, newValue, false);
}
}
void pushBatteryDataToMqtt(const batteryStack& lastSentData, bool forceUpdate /* if true - we will send all data regardless if it's the same */)
{
mqtt_publish_f(MQTT_TOPIC_ROOT "soc", g_stack.soc, lastSentData.soc, 0, forceUpdate);
mqtt_publish_f(MQTT_TOPIC_ROOT "temp", (float)g_stack.temp/1000.0, (float)lastSentData.temp/1000.0, 0.1, forceUpdate);
mqtt_publish_i(MQTT_TOPIC_ROOT "currentDC", g_stack.currentDC, lastSentData.currentDC, 1, forceUpdate);
mqtt_publish_i(MQTT_TOPIC_ROOT "estPowerAC", g_stack.getEstPowerAc(), lastSentData.getEstPowerAc(), 10, forceUpdate);
mqtt_publish_i(MQTT_TOPIC_ROOT "battery_count",g_stack.batteryCount, lastSentData.batteryCount, 0, forceUpdate);
mqtt_publish_s(MQTT_TOPIC_ROOT "base_state", g_stack.baseState, lastSentData.baseState , forceUpdate);
mqtt_publish_i(MQTT_TOPIC_ROOT "is_normal", g_stack.isNormal() ? 1:0, lastSentData.isNormal() ? 1:0, 0, forceUpdate);
mqtt_publish_i(MQTT_TOPIC_ROOT "getPowerDC", g_stack.getPowerDC(), lastSentData.getPowerDC(), 1, forceUpdate);
mqtt_publish_i(MQTT_TOPIC_ROOT "powerIN", g_stack.powerIN(), lastSentData.powerIN(), 1, forceUpdate);
mqtt_publish_i(MQTT_TOPIC_ROOT "powerOUT", g_stack.powerOUT(), lastSentData.powerOUT(), 1, forceUpdate);
// publishing details
for (int ix = 0; ix < g_stack.batteryCount; ix++) {
char ixBuff[50];
String ixBattStr = MQTT_TOPIC_ROOT + String(ix) + "/voltage";
ixBattStr.toCharArray(ixBuff, 50);
mqtt_publish_f(ixBuff, g_stack.batts[ix].voltage / 1000.0, lastSentData.batts[ix].voltage / 1000.0, 0, forceUpdate);
ixBattStr = MQTT_TOPIC_ROOT + String(ix) + "/current";
ixBattStr.toCharArray(ixBuff, 50);
mqtt_publish_f(ixBuff, g_stack.batts[ix].current / 1000.0, lastSentData.batts[ix].current / 1000.0, 0, forceUpdate);
ixBattStr = MQTT_TOPIC_ROOT + String(ix) + "/soc";
ixBattStr.toCharArray(ixBuff, 50);
mqtt_publish_i(ixBuff, g_stack.batts[ix].soc, lastSentData.batts[ix].soc, 0, forceUpdate);
ixBattStr = MQTT_TOPIC_ROOT + String(ix) + "/charging";
ixBattStr.toCharArray(ixBuff, 50);
mqtt_publish_i(ixBuff, g_stack.batts[ix].isCharging()?1:0, lastSentData.batts[ix].isCharging()?1:0, 0, forceUpdate);
ixBattStr = MQTT_TOPIC_ROOT + String(ix) + "/discharging";
ixBattStr.toCharArray(ixBuff, 50);
mqtt_publish_i(ixBuff, g_stack.batts[ix].isDischarging()?1:0, lastSentData.batts[ix].isDischarging()?1:0, 0, forceUpdate);
ixBattStr = MQTT_TOPIC_ROOT + String(ix) + "/idle";
ixBattStr.toCharArray(ixBuff, 50);
mqtt_publish_i(ixBuff, g_stack.batts[ix].isIdle()?1:0, lastSentData.batts[ix].isIdle()?1:0, 0, forceUpdate);
ixBattStr = MQTT_TOPIC_ROOT + String(ix) + "/state";
ixBattStr.toCharArray(ixBuff, 50);
mqtt_publish_s(ixBuff, g_stack.batts[ix].isIdle()?"Idle":g_stack.batts[ix].isCharging()?"Charging":g_stack.batts[ix].isDischarging()?"Discharging":"", lastSentData.batts[ix].isIdle()?"Idle":lastSentData.batts[ix].isCharging()?"Charging":lastSentData.batts[ix].isDischarging()?"Discharging":"", forceUpdate);
ixBattStr = MQTT_TOPIC_ROOT + String(ix) + "/temp";
ixBattStr.toCharArray(ixBuff, 50);
mqtt_publish_f(ixBuff, (float)g_stack.batts[ix].tempr/1000.0, (float)lastSentData.batts[ix].tempr/1000.0, 0.1, forceUpdate);
}
}
void mqttLoop()
{
//if we have problems with connecting to mqtt server, we will attempt to re-estabish connection each 1minute (not more than that)
static unsigned long g_lastConnectionAttempt = 0;
//first: let's make sure we are connected to mqtt
const char* topicLastWill = MQTT_TOPIC_ROOT "availability";
if (!mqttClient.connected() && (g_lastConnectionAttempt == 0 || os_getCurrentTimeSec() - g_lastConnectionAttempt > 60)) {
if(mqttClient.connect(WIFI_HOSTNAME, MQTT_USER, MQTT_PASSWORD, topicLastWill, 1, true, "offline"))
{
Log("Connected to MQTT server: " MQTT_SERVER);
mqttClient.publish(topicLastWill, "online", true);
}
else
{
Log("Failed to connect to MQTT server.");
}
g_lastConnectionAttempt = os_getCurrentTimeSec();
}
//next: read data from battery and send via MQTT (but only once per MQTT_PUSH_FREQ_SEC seconds)
static unsigned long g_lastDataSent = 0;
if(mqttClient.connected() &&
os_getCurrentTimeSec() - g_lastDataSent > MQTT_PUSH_FREQ_SEC &&
sendCommandAndReadSerialResponse("pwr") == true)
{
static batteryStack lastSentData; //this is the last state we sent to MQTT, used to prevent sending the same data over and over again
static unsigned int callCnt = 0;
parsePwrResponse(g_szRecvBuff);
bool forceUpdate = (callCnt % 20 == 0); //push all the data every 20th call
pushBatteryDataToMqtt(lastSentData, forceUpdate);
callCnt++;
g_lastDataSent = os_getCurrentTimeSec();
memcpy(&lastSentData, &g_stack, sizeof(batteryStack));
}
mqttClient.loop();
}
#endif //ENABLE_MQTT
edit 7.11.24
In the meantime, I have also layouted an interface board with a USB type B socket for the 5V supply. (see layout below). Because as small and fine as the micro USB plugs are, I need something more robust.
new board version with USB type B socket for power supply
As I am asked more and more often for the production data, I am making the Gerber data of the circuit boards available for download:
In the article entitled: “Reading energy supply company smart meters with ESP32 and sending data via MQTT” (link), I described how the energy supply companies’ smart meters can be read out via the customer interface. The measurement data is then available as topics via the mqtt broker and can be further processed in various home automation systems (HomeMatic, Homeassistant, etc.). All you need is an ESP32 board and a few small parts to establish the connection to the smart meter. As a small update, I have now embellished the structure (back then with pin headers on a breadboard) a little and made a circuit board.
Layout preview in designtool
The associated circuit diagram essentially corresponds to the sketch in the previous article. To make things a little more convenient with the new circuit board, the connection to the customer interface of the smart meter can be plugged in via an RJ socket. I have also implemented the power supply via a USB socket.
Once the ESP32 circuit board had been fitted and plugged in, the device was given a small housing and is now doing its job in the electrical distribution cabinet.
The hardware is therefore ready and functional. I have also considered changing something about the software. Until now, the ESP was running a program that decrypted the data from the smart meter and then sent it to the IP address of the broker via MQTT. However, as I am now also a user of the ESPHome integration in my HomeAssistant environment, I have flashed the ESP with an ESPHome base image. On GitHub there is the repository of Andre-Schuiki, where he publishes a version for ISKRA and SIEMENS Smartmeter for use with ESPHome. The installation instructions can be found under the following link: https://github.com/Andre-Schuiki/esphome_im350/tree/main/esp_home
The script for the ESPHome device looks like this:
esphome:
name: kelagsmartmeter
friendly_name: KelagSmartmeter
libraries:
- "Crypto" # !IMPORTANT! we need this library for decryption!
esp32:
board: esp32dev
framework:
type: arduino
# Enable logging
logger:
# Enable Home Assistant API
api:
encryption:
key: "da kommt der key rein des neu angelegten ESPHome Gerätes rein"
ota:
password: "das automatisch generierte ota passwort"
wifi:
ssid: !secret wifi_ssid
password: !secret wifi_password
# Enable fallback hotspot (captive portal) in case wifi connection fails
ap:
ssid: "Kelagsmartmeter Fallback Hotspot"
password: "das automatisch generierte password"
captive_portal:
external_components:
- source:
type: local
path: custom_esphome
sensor:
- platform: siemens_im350
update_interval: 5s
trigger_pin: 26 # this pin goes to pin 2 of the customer interface and will be set to high before we try to read the data from the rx pin
rx_pin: 16 # this pin goes to pin 5 of the customer interface
tx_pin: 17 # not connected at the moment, i added it just in case we need it in the future..
decryption_key: "00AA01BB02CC03DD04EE05FF06AA07BB" # you get the key from your provider!
use_test_data: false # that was just for debugging, if you set it to true data are not read from serial and the test_data string is used
test_data: "7EA077CF022313BB45E6E700DB0849534B697460B6FA5F200005C8606F536D06C32A190761E80A97E895CECA358D0A0EFD7E9C47A005C0F65B810D37FB0DA2AD6AB95F7F372F2AB11560E2971B914A5F8BFF5E06D3AEFBCD95B244A373C5DBDA78592ED2C1731488D50C0EC295E9056B306F4394CDA7D0FC7E0000"
delay_before_reading_data: 1000 # this is needed because we have to wait for the interface to power up, you can try to lower this value but 1 sec was ok for me
max_wait_time_for_reading_data: 1100 # maximum time to read the 123 Bytes (just in case we get no data)
ntp_server: "pool.ntp.org" #if no ntp is specified pool.ntp.org is used
ntp_gmt_offset: 3600
ntp_daylight_offset: 3600
counter_reading_p_in:
name: reading_p_in
filters:
- lambda: return x / 1000;
unit_of_measurement: kWh
accuracy_decimals: 3
device_class: energy
counter_reading_p_out:
name: reading_p_out
filters:
- lambda: return x / 1000;
unit_of_measurement: kWh
accuracy_decimals: 3
device_class: energy
counter_reading_q_in:
name: reading_q_in
filters:
- lambda: return x / 1000;
unit_of_measurement: kvarh
device_class: energy
counter_reading_q_out:
name: reading_q_out
filters:
- lambda: return x / 1000;
unit_of_measurement: kvarh
device_class: energy
current_power_usage_in:
name: power_usage_in
filters:
- lambda: return x / 1000;
unit_of_measurement: kW
accuracy_decimals: 3
device_class: energy
current_power_usage_out:
name: power_usage_out
filters:
- lambda: return x / 1000;
unit_of_measurement: kW
accuracy_decimals: 3
device_class: energy
# Extra sensor to keep track of uptime
- platform: uptime
name: IM350_Uptime Sensor
switch:
- platform: restart
name: IM350_Restart
This post has absolutely nothing to do with retro this time. My colleague from the Multimedia major has been working on the topic of DeepFakes and has put a lot of effort into creating a great video. Here is the link to the video:
An interesting problem has arisen with the measurement technology in the laboratories at my workplace. I use the term “measurement technology” to describe the equipment of a laboratory workstation for basic training. There are a total of nineteen laboratory workstations, each equipped with two laboratory power supplies, two desktop multimeters, a Keysight signal generator and a Keysight (Agilent) oscilloscope of the Infiniivision DSO-X 20xx series. All devices are network-compatible and are connected to the corresponding workstation computer via LAN. This means that the measuring devices can be accessed using different software (Agilent VEE, Matlab, LabVIEW etc.). The devices were purchased around three years ago and replace the almost twenty-year-old laboratory equipment.
However, it has now happened that the DSO-X2012A oscilloscope at one workstation no longer showed any signs of life. It occasionally happens that during laboratory exercises or when working freely in the laboratories, a student presses the emergency stop switch of the workstation and thus de-energizes it. But this was not the case. All the devices connected to the workstation worked, with the exception of the DSO. Voltage could also be measured at the end of the IEC plug. So the problem could only be with the oscilloscope itself. The rear panel is quickly unscrewed, a shield plate removed and the power supply unit is exposed. The first visual inspection immediately revealed the large filter capacitor with its upwardly curved cap. But first things first.
Power supply unit of the Infiniivision
The mains voltage was measured at the AC pins of the mains input, but no DC voltage was measured at any of the outputs of the power supply unit. Regardless of whether the power switch of the device was switched on or off. This suggests that the power supply unit is defective.
Input fusing
First, the power supply unit was removed and examined, starting with the AC input side. The print fuse in the area of the mains filter was the first defective component to be noticed. It is a slow-blow 6.3A/250V fuse. As a blown fuse always has a reason to switch off, the search continued. The mains rectifiers were OK, but the 100uF / 420V electrolytic capacitor, which is used to smooth the DC voltage on the primary side, had already suffered some thermal damage and was bloated.
original electrolytic capacitor 100uF /420V /105°C
Its capacity was also no longer within the nominal range. But even that was not the direct reason for the fuse blowing. This was quickly found. A mosfet used to control the transformer was low-resistance. More precisely, it had a short circuit between all the connections.
Mosfet STP12NM50
The following picture shows the installation positions of the components. These have been replaced. The mosfet was replaced with an original type and the power supply capacitor was replaced with a 100uF / 450V / 105°C type. Although it is about five millimeters higher, it fits easily into the power supply unit.
Installation position of the capacitor and the mosfet
Two SMD resistors on the back of the power supply board were defective in the area of the gate connection of the mosfet. These were an SMD resistor of size 0805 with 5.11 Ohm and an SMD resistor of size 1206 with 2.0 kOhm. The picture below also shows the installation position.
Mounting position of defective SMD resistors
After all the components mentioned had been replaced, a first functional test was carried out. However, this was sobering, as the power supply unit was still not working. The fuse remained intact and the DC voltage on the primary side was stable. But the gate of the mosfet was not activated – unfortunately. Because now came the time-consuming part of the repair. On the power supply board, installed upright, there is another board on which several controller ICs are installed. If you follow the gate line from the Mosfet, it ends at a pin on this control board. So this must be removed.
Controller board removed
To do this, the cooling plate had to be removed first. Then it became a bit tedious, because the controller board is not connected to the main board via a pin header or plug connection, but the contact pins are laid out and milled out. This means that the desoldering work has to be carried out very carefully so as not to damage the conductor tracks at the ends of the milled pins.
Mainboard without controller board
UC3842B
Once the removal was successfully completed, the controller board could be inspected. Lo and behold, the line routed from the gate of the mosfet ends at pin 6 of a small IC. This is a UC3842B VD1R2G. The housing of this IC was blown up. In addition to the controller IC, a SOT23 PNP transistor (PMBT 2907A) was also dead and had a low resistance on all pins.
Installation position of the defective components
After replacing the defective components, the power supply unit was reassembled and a function test was carried out. The oscilloscope started up again and the power supply unit did its job.
defekte BauteileResult after successful repair
It would now be interesting to find out why the power supply gave up the ghost after just three years. Especially as the oscilloscopes do not run continuously, but are only switched on during the relevant courses. We noticed the following: The oscilloscope is permanently connected to the power supply. However, the oscilloscope’s power switch does not switch off the AC supply, but only the controller control in the secondary area of the power supply unit. This means that the power supply unit operates in standby mode when it is switched off. And we have noticed that all oscilloscopes that are switched off have a power loss in standby that heats up the mosfets and especially the 100uF electrolytic capacitor. This would explain the bloated, dried-out electrolytic capacitor and the subsequent death of the power supply units. To verify this, the temperature of the components was measured on several devices that had not been switched on for days.
Thermal sensor on the electrolytic capacitor surface
The following could be determined here. Both the surface of the capacitor and the cooling plate of the mosfets measured temperatures of 56°C to almost 60°C when switched off. Should this be the case?
Temperature measurement on the electrolytic capacitor
A smarthome is no longer a rarity today and is very widespread. There are countless systems on the market that make your own home “smart”. The digital voice assistants from Google, Amazon and co. in conjunction with smart light bulbs are among the systems that are easy and quick to install. But there are also complex smart home systems, in which actuators for every lamp and socket are installed in the house distributors. The windows and doors are equipped with signaling contacts and secure the home or report if once forgotten to close the windows after shock ventilation. It goes without saying that these systems also contribute to energy optimization when programmed sensibly. I also operate Smarthome components from various manufacturers.
For years, this has included the HomeMatic system, which communicates with its actuators and sensors both wired and via the Bidcos protocol. The HUE system from Phillips talks to its smart lamps and sockets via ZigBee. The gateways of these systems are connected to a LAN network and each system brings its own web server, through which it can then be controlled and set. An inverter of photovoltaic systems can provide its data via different interfaces (RS485, CAN, RS232). To bring all of them to a central display level, I decided to use the NodeRed system. The necessary NodeRed server runs on a Raspberry PI. (On the CCU3 with the Raspbian image is still enough space to run the NodeRed server – it is even available as a separate plugin for the CCU and is called “RedMatic”). With this configuration you can “slay” almost everything in the field of home automation. With ESP32 and Raspberry you can easily transfer status information via MQTT (Message Queueing Telemetry Transport). I use this for example with the small feed-in inverters of a balcony PV system, as well as with the PV inverters of an offgrid system. Here the data is received via different bus systems in the Raspberry or ESP32 and converted into the MQTT protocol. The MQTT broker collects the data from the individual devices and via NodeRed they can then be written to a database, visualized in the browser or on the smartphone and also easily processed in the HomeMatic system, as required.
Example of a smart home network
Thus, it is possible to network almost all systems with each other smartly and, importantly for me, to visualize them on ONE platform. One single system was missing until now. That is my old Neura heating heat pump. The company Neura has not existed for several years and the web server “webidalog” developed by “b.i.t.” has never been updated. So the heat pump has a web server on a small with Linux computer onboard and builds the web application with an ancient Java version. For the operation a Java Runtime must be installed on the PC, which runs only with some tricks on a current Windows computer (keyword: virtualization). For the operation via a smartphone an html – version with limited functionality is available. My plan now was to find an interface, with which I can read out the data of the heat pump at least once, in order to have flow- return temperatures of the floor heating, boiler temperature, etc. also available in my NodeRed system. But since there is almost no documentation for the system and reverse engineering is a bit critical if the system should continue to run, I came up with the following idea:
With a “headles browser” it should be possible to parse the html version of the Neura WebDialog website and find the relevant data and turn it into MQTT topics via variables. And here I have to give a special thanks to my colleague Mario Wehr, who built the software structure to parse the website. The software is written in PHP and finally runs on a Raspberry PI. All you need is a php8-cli runtime and a few modules. The way the software works is that every time the heat pump website is called, a login is executed, then the data is parsed and sent to MQTT broker. The continuous calling of the php script I then simply solved with a cronjob that is executed every minute.
Rummaging through a box of my old crafts I found the box below. It dates from the time when I was still working with Amgia, but also with PCs – I guess around 1996. I labeled the box “DB50XG MIDI – Wavetable Processor”.
Das Fundstück aus der Kiste
Inside is a circuit board from Yamaha, which is called the DB50XG. This board was designed as a daughter board for PC sound cards with “Waveblaster” expansion port. She expanded the sound cards with a polyphonic MIDI wavetable sampler. In this way, the General Midi Standard and the Yamaha XG Standard could be re-established. Today nobody thinks about it anymore. At that time, if you wanted to generate sounds with a PC from midi data, then either external hardware was required, or a sound card with an onboard midi synthesizer or wavetable chipset. The PC then took over the control, the sending and receiving of the midi data via a sequencer software. Today, the midi sounds are generated directly on the PC and the samples and sound models are integrated into the software. At that time, the performance of the PC hardware was not sufficient. If someone is wondering what I’m palavering about here – what is Midi and why do you need it? – then let me put it briefly here: Midi is the abbreviation for “Musical Instrument Digital Interface” – i.e. a digital interface – a data protocol for musical instruments. Roughly explained, it serves to network and control electronic musical instruments with each other. For example, a large number of sound-generating devices can be controlled via a single keyboard. I will not explain here how the Midi standard works, what the data packets look like and how it looks electrically. As always, there is plenty of information on the web.
Inside the box
Back to the self-made box. At that time I packed the DB50XG in the plastic box and from the “Waveblaster” port, a 26-pin socket strip, led the necessary cables to the outside to start up the Midi board. And that was pretty simple. The board requires a power supply of +/-12V and +5V. There is a Midi-IN and a Midi-OUT (through) pin, a reset pin and two analog audio out pins – one per channel. The table below shows the connector pin assignment:
pin number
assignment
1
Digital ground
2
not connected
3
Digital ground
4
not connected
5
Digital ground
6
Supply +5V
7
Digital ground
8
not connected
9
Digital ground
10
Supply +5V
11
Digital ground
12
not connected
13
not connected
14
Supply +5V
15
Analoge ground
16
not connected
17
Analogue ground
18
Supply + 12V
19
Analogue ground
20
Audio out richt
21
Analogue ground
22
Supply -12V
23
Analogue ground
24
Audio out left
25
Analogue ground
26
reset
The whole structure was rather spartan back then. The power supply had to be established via one or more external power supplies. There was no galvanic signal isolation using optocouplers. So I had to rely on the proper setup of the Midi IO controller that I connected to the Amiga. Of course it couldn’t stay like this. And I can’t bring myself not to use the beautiful DB50XG board anymore or to throw it away in the electronic waste. The plan that emerged from this was to develop a new interface board – or to tinker, which should be as universally usable as possible.
DB50XG
It’s been a few years since this idea and I’ve always worked on it a little bit. I thought the interface board should fulfill the following points:
a simple power supply should supply the Yamaha board with energy. Ideally, there should be a USB port and, optionally, a connection for a universal power supply. All required voltages should be generated on the interface board from the 5VDC.
As in the past, the DB50XG should also be able to be plugged in as a “piggyback” circuit board
The midi-in signal should be able to be fed in via the 5-pin DIN socket and also via a pin header – of course nicely decoupled (this means that a microcontroller such as Arduino and co. can also be connected without any effort)
The sound, i.e. the audio signal, should be available for acceptance via a chinch socket and also as a 3.5mm jack socket and via a pin header per channel.
Word repetitions SHOULD be avoided, but I don’t care
This ultimately resulted in the following circuit diagram. The 5VDC supply of the USB source is routed directly to the 5V supply of the midi board. The +12V/-12V that are also required are generated by a DC/DC converter (TMR0522). This is supplied on the input side by the 5V mains. The optional “external” voltage input goes to a LM2596ADJ. This is a step-down voltage regulator that can work with input voltages up to 40V. The regulated output side is available in many areas. I have integrated the ADJ (Adjustable) type into the circuit here, as I have a few of them in the assortment box. The voltage source can be selected with a jumper on the board.
Based on this circuit diagram, I created a layout and initially produced it in my own etching bath. The result was the following circuit board, which served as a test setup. Technically, the board worked perfectly, but I didn’t like the arrangement of the components. I placed the step-down converter and coil on the back. The distance between the connection sockets was also too close together for me. And how you do it as a PCB layouter – you always do a second design. So also it is this time.
The test setup with a fitted Midiboard can be seen in the image below. The midi signal as a test source comes from the PC and is generated by a USB midi adapter from the Far East.
So sat down in front of the computer again and redrawn the layout. The following version came out. I then ordered this version from a circuit board manufacturer.
The finally manufactured printed circuit board then looks like this. Below she can be seen with the DB50XG board attached.