All posts by ingmarsretro

Pylontech PV battery status in the HomeAssistant

Loading

Anyone who has installed a photovoltaic system in their own home may even use an energy storage system. In this example, it is an off-grid system equipped with two modules from the manufacturer Pylontech. The Pylontech US3000C batteries have an output voltage of 48V. The nominal capacity is 3500Wh. The installed cells are LiFePO4 cells and the usable capacity is specified as 3374Wh according to the data sheet. The batteries are designed to be connected in parallel with other Pylontech batteries. The internally installed BMS (battery management system) communicates with the other Pylontech battery modules via a so-called “link” interface. A battery configured as a “master” handles the data exchange with the inverter. Here, Pylontech provides the CAN or RS485 bus as an interface. However, if you want information about the individual cells (voltages, currents, charges, temperatures, etc.), there is another interface on each module called “Console”. This is an RS232 interface via which you can communicate directly with the battery’s BMS. This port is also used to update the firmware of the BMS. However, I STRONGLY advise against playing around with firmware updates and flash software. This is reserved for the manufacturer or the liable party.

However, as this interface also provides a lot of information about the cells installed in the battery, this is an interesting approach. Initially, I had a laptop connected to a terminal and was able to discover and monitor the individual cell voltages and, above all, the possibly different charge status of the modules connected in parallel. So I thought it would be a good idea to have this information available in my home automation system, where it could be visualized and used for control purposes.

As we geeks and technology enthusiasts are quite familiar with terms such as Homeassistant, Docker, Proxmox, HomeMatic, NodeRed etc., I thought that this data should also become entities in the Homeassistant. So a small new project was quickly created. My plan was to read the data from the serial interface and send it to the Home Assistant via MQTT.

But before I start disassembling the data strings that come out via the serial port, I’ll have a look at the search engines. Perhaps someone else has already dealt with this topic. And that’s exactly what happened. I found what I was looking for on GitHub under the term “pylontec2mqtt”. A project is hosted at https://github.com/irekzielinski/Pylontech-Battery-Monitoring that uses ESP8266 to collect the serial data from the port and sends it to the Homeassistant server via MQTT and Wifi. A fork with a further development of this project can be found at https://github.com/hidaba/PylontechMonitoring.

Why am I publishing the project here on the blog despite the simple replica? I have optimized the circuit a little and packed it into a layout and adapted the code a little. I would like to share the result here. It was important to me to have a sensible structure on a circuit board that is connected with a USB A-B cable for the power supply and a LAN-RJ45 cable for the data connection. I wanted to use a “solid” USB connector (not the fragile mini or micro USB connectors)

On a breadboard and with the usual development boards, I quickly “knitted together” a functional model so that I could adapt the software to it.

Functional sample on perforated grid

So I first created a circuit diagram from the sketches in the Git project. There is a “real” RS232 level at the “Console” interface, which is converted to a 5V TTL via the MAX3232 IC. The BSS123 FET is used to realize a level converter to 3.3V for each of the RX and TX signals.

pylontec2mqtt schematic

The ESP8266 processes this 3.3V TTL level in the form of the Wemos D1Mini or WemosD1Pro development board, which is plugged onto the circuit board. I then packed the entire construction into a small plastic housing, which can be conveniently connected to the Pylontec and a USB power source via the LAN and USB cables.

Layout preview in designtool

The layout design is shown in the picture above. The circuit board and the position of the components were checked again with the preview before production and then ordered from a trusted manufacturer.

Preview of the circuit board before production

After barely two weeks of waiting, I had the empty circuit boards in my hands and was able to fit them with the components.

fully assembled circuit board

The picture above shows the fully assembled board. The only thing missing here is the Wemos board with the ESP.

Comparison between functional model and first “production model”

In the end, I plugged in a WemosD1 Pro, as this offers the option of connecting an external WiFi antenna and thus getting a reasonable wireless range.

After flashing the software and commissioning, the Wemos web server can be accessed at the IP address specified in the code. Here you can also check whether the Pylontech battery is communicating with the Wemos. The result then looks like this.

Webseite of the WEMOS ESP

Here you can see that both battery modules are recognized correctly. The next step is to check whether messages are being sent via the MQTT protocol. The IP address of the MQTT broker must also be specified in the Wemo code. In my setup, I have set up the MQTT Explorer in the Home Assistant to be able to check the MQTT functions quickly and easily.

MQTT Explorer

The image above shows that the data also arrives correctly via MQTT. Now it is only necessary to create a sensor yaml file in the home assistant to make the topics available as entities. I have added the following code to configuration.yaml for this purpose:

mqtt:
  sensor:
#Pylontec Akku Serial Readout (ESP32 192.168.xxx.yyy)
    - state_topic: "ingmarsretro/pylontec/ESP_WiFi_RSSI"
      name: "Pylontec_RSSI"
      unit_of_measurement: dB
      
    - state_topic: "ingmarsretro/pylontec/availability"
      name: "Pylontec_Status"
      
    - state_topic: "ingmarsretro/pylontec/currentDC"
      name: "DC-Strom"
      unit_of_measurement: "mA"
      
    - state_topic: "ingmarsretro/pylontec/getPowerDC"  
      name: "getPower DC"
      unit_of_measurement: "W"
      
    - state_topic: "ingmarsretro/pylontec/powerIN"  
      name: "Power IN"
      unit_of_measurement: "W"  
      
    - state_topic: "ingmarsretro/pylontec/estPowerAC"
      name: "Pylontec_estPowerAC"
      unit_of_measurement: Watt  
      
    - state_topic: "ingmarsretro/pylontec/soc"
      name: "Pylontec_SOC"
      unit_of_measurement: "%"  
      
    - state_topic: "ingmarsretro/pylontec/temp"
      name: "Pylontec_temperature"
      unit_of_measurement: "°C"
      
    - state_topic: "ingmarsretro/pylontec/battery_count"
      name: "Pylontec_BatteryCount"
      unit_of_measurement: "pcs"      
      
    - state_topic: "ingmarsretro/pylontec/base_state"
      name: "Pylontec_BaseState"
      
    - state_topic: "ingmarsretro/pylontec/is_normal"  
      name: "Pylontec_is_normal"

    - state_topic: "ingmarsretro/pylontec/powerOUT"
      name: "Pylontec_powerOUT"
      unit_of_measurement: Watt
      
# Pylontech battery module 0      
      
    - state_topic: "ingmarsretro/pylontec/0/current"
      name: "Pylontec_Battery0_current"
      unit_of_measurement: "A"

    - state_topic: "ingmarsretro/pylontec/0/voltage"
      name: "Pylontec_Battery0_voltage"
      unit_of_measurement: "V"
      
    - state_topic: "ingmarsretro/pylontec/0/soc"
      name: "Pylontec_Battery0_soc"
      unit_of_measurement: "%"
      
    - state_topic: "ingmarsretro/pylontec/0/charging"
      name: "Pylontec_Battery0_charging"
      
    - state_topic: "ingmarsretro/pylontec/0/discharging"
      name: "Pylontec_Battery0_discharging"
     
    - state_topic: "ingmarsretro/pylontec/0/idle"
      name: "Pylontec_Battery0_idle"
      
    - state_topic: "ingmarsretro/pylontec/0/state"
      name: "Pylontec_Battery0_state"
      
    - state_topic: "ingmarsretro/pylontec/0/temp"
      name: "Pylontec_Battery0_temp"
      unit_of_measurement: "°C"
      
# Pylontech battery module 1      
      
    - state_topic: "ingmarsretro/pylontec/1/current"
      name: "Pylontec_Battery1_current"
      unit_of_measurement: "A"

    - state_topic: "ingmarsretro/pylontec/1/voltage"
      name: "Pylontec_Battery1_voltage"
      unit_of_measurement: "V"
      
    - state_topic: "ingmarsretro/pylontec/1/soc"
      name: "Pylontec_Battery1_soc"
      unit_of_measurement: "%"
      
    - state_topic: "ingmarsretro/pylontec/1/charging"
      name: "Pylontec_Battery1_charging"
      
    - state_topic: "ingmarsretro/pylontec/1/discharging"
      name: "Pylontec_Battery1_discharging"
     
    - state_topic: "ingmarsretro/pylontec/1/idle"
      name: "Pylontec_Battery1_idle"
      
    - state_topic: "ingmarsretro/pylontec/1/state"
      name: "Pylontec_Battery1_state"
      
    - state_topic: "ingmarsretro/pylontec/1/temp"
      name: "Pylontec_Battery1_temp"
      unit_of_measurement: "°C"
      

On the Homeassistant website, the visualization could then look like this, for example:

Last but not least, I am posting the customized code below. The libraries required for compilation and further information can be found in the GitHub links above.

#include <ESP8266WiFi.h>
#include <ESP8266mDNS.h>
#include <ArduinoOTA.h>
#include <ESP8266WebServer.h>
#include <circular_log.h>
#include <ArduinoJson.h>
#include <NTPClient.h>
#include <ESP8266TimerInterrupt.h>

//+++ START CONFIGURATION +++

//IMPORTANT: Specify your WIFI settings:
#define WIFI_SSID "wifiname"
#define WIFI_PASS deinpasswort1234"
#define WIFI_HOSTNAME "mppsolar-pylontec"

//Uncomment for static ip configuration
#define STATIC_IP
  IPAddress local_IP(192, 168, xxx, yyy);
  IPAddress subnet(255, 255, 255, 0);
  IPAddress gateway(192, 168, xxx, zzz);
  IPAddress primaryDNS(192, 168, xxx, zzz);

//Uncomment for authentication page
//#define AUTHENTICATION
//set http Authentication
const char* www_username = "admin";
const char* www_password = "password";


//IMPORTANT: Uncomment this line if you want to enable MQTT (and fill correct MQTT_ values below):
#define ENABLE_MQTT

// Set offset time in seconds to adjust for your timezone, for example:
// GMT +1 = 3600
// GMT +1 = 7200
// GMT +8 = 28800
// GMT -1 = -3600
// GMT 0 = 0
#define GMT 3600

//NOTE 1: if you want to change what is pushed via MQTT - edit function: pushBatteryDataToMqtt.
//NOTE 2: MQTT_TOPIC_ROOT is where battery will push MQTT topics. For example "soc" will be pushed to: "home/grid_battery/soc"
#define MQTT_SERVER        "192.168.xx.broker"
#define MQTT_PORT          1883
#define MQTT_USER          ""
#define MQTT_PASSWORD      ""
#define MQTT_TOPIC_ROOT    "ingmarsretro/pylontec/"  //this is where mqtt data will be pushed
#define MQTT_PUSH_FREQ_SEC 2  //maximum mqtt update frequency in seconds

//+++   END CONFIGURATION +++

#ifdef ENABLE_MQTT
#include <PubSubClient.h>
WiFiClient espClient;
PubSubClient mqttClient(espClient);
#endif //ENABLE_MQTT

//text response
char g_szRecvBuff[7000];

const long utcOffsetInSeconds = GMT;
char daysOfTheWeek[7][12] = {"Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"};
// Define NTP Client to get time
WiFiUDP ntpUDP;
NTPClient timeClient(ntpUDP, "pool.ntp.org", utcOffsetInSeconds);


ESP8266WebServer server(80);
circular_log<7000> g_log;
bool ntpTimeReceived = false;
int g_baudRate = 0;

void Log(const char* msg)
{
  g_log.Log(msg);
}

//Define Interrupt Timer to Calculate Power meter every second (kWh)
#define USING_TIM_DIV1 true                                             // for shortest and most accurate timer
ESP8266Timer ITimer;
bool setInterval(unsigned long interval, timer_callback callback);      // interval (in microseconds)
#define TIMER_INTERVAL_MS 1000

//Global Variables for the Power Meter - accessible from the calculating interrupt und from main
unsigned long powerIN = 0;       //WS gone in to the BAttery
unsigned long powerOUT = 0;      //WS gone out of the Battery
//Global Variables for the Power Meter - Überlauf
unsigned long powerINWh = 0;       //WS gone in to the BAttery
unsigned long powerOUTWh = 0;      //WS gone out of the Battery

void setup() {
  
  memset(g_szRecvBuff, 0, sizeof(g_szRecvBuff)); //clean variable
  
  pinMode(LED_BUILTIN, OUTPUT); 
  digitalWrite(LED_BUILTIN, HIGH);//high is off
  
  // put your setup code here, to run once:
  WiFi.mode(WIFI_STA);
  WiFi.persistent(false); //our credentialss are hardcoded, so we don't need ESP saving those each boot (will save on flash wear)
  WiFi.hostname(WIFI_HOSTNAME);
  #ifdef STATIC_IP
     WiFi.config(local_IP, gateway, subnet, primaryDNS);
  #endif
  WiFi.begin(WIFI_SSID, WIFI_PASS);

  for(int ix=0; ix<10; ix++)
  {
    Log("Wait for WIFI Connection");
    if(WiFi.status() == WL_CONNECTED)
    {
      break;
    }

    delay(1000);
  }

  ArduinoOTA.setHostname(WIFI_HOSTNAME);
  ArduinoOTA.begin();
  server.on("/", handleRoot);
  server.on("/log", handleLog);
  server.on("/req", handleReq);
  server.on("/jsonOut", handleJsonOut);
  server.on("/reboot", [](){
    #ifdef AUTHENTICATION
    if (!server.authenticate(www_username, www_password)) {
      return server.requestAuthentication();
    }
    #endif
    ESP.restart();
  });
  
  server.begin(); 
  
  timeClient.begin();

  
#ifdef ENABLE_MQTT
  mqttClient.setServer(MQTT_SERVER, MQTT_PORT);
#endif

  Log("Boot event");
  
}

void handleLog()
{
  #ifdef AUTHENTICATION
  if (!server.authenticate(www_username, www_password)) {
    return server.requestAuthentication();
  } 
  #endif
  server.send(200, "text/html", g_log.c_str());
}

void switchBaud(int newRate)
{
  if(g_baudRate == newRate)
  {
    return;
  }
  
  if(g_baudRate != 0)
  {
    Serial.flush();
    delay(20);
    Serial.end();
    delay(20);
  }

  char szMsg[50];
  snprintf(szMsg, sizeof(szMsg)-1, "New baud: %d", newRate);
  Log(szMsg);
  
  Serial.begin(newRate);
  g_baudRate = newRate;

  delay(20);
}

void waitForSerial()
{
  for(int ix=0; ix<150;ix++)
  {
    if(Serial.available()) break;
    delay(10);
  }
}

int readFromSerial()
{
  memset(g_szRecvBuff, 0, sizeof(g_szRecvBuff));
  int recvBuffLen = 0;
  bool foundTerminator = true;
  
  waitForSerial();
  
  while(Serial.available())
  {
    char szResponse[256] = "";
    const int readNow = Serial.readBytesUntil('>', szResponse, sizeof(szResponse)-1); //all commands terminate with "$$\r\n\rpylon>" (no new line at the end)
    if(readNow > 0 && 
       szResponse[0] != '\0')
    {
      if(readNow + recvBuffLen + 1 >= (int)(sizeof(g_szRecvBuff)))
      {
        Log("WARNING: Read too much data on the console!");
        break;
      }
      
      strcat(g_szRecvBuff, szResponse);
      recvBuffLen += readNow;

      if(strstr(g_szRecvBuff, "$$\r\n\rpylon"))
      {
        strcat(g_szRecvBuff, ">"); //readBytesUntil will skip this, so re-add
        foundTerminator = true;
        break; //found end of the string
      }

      if(strstr(g_szRecvBuff, "Press [Enter] to be continued,other key to exit"))
      {
        //we need to send new line character so battery continues the output
        Serial.write("\r");
      }

      waitForSerial();
    }
  }

  if(recvBuffLen > 0 )
  {
    if(foundTerminator == false)
    {
      Log("Failed to find pylon> terminator");
    }
  }

  return recvBuffLen;
}

bool readFromSerialAndSendResponse()
{
  const int recvBuffLen = readFromSerial();
  if(recvBuffLen > 0)
  {
    server.sendContent(g_szRecvBuff);
    return true;
  }

  return false;
}

bool sendCommandAndReadSerialResponse(const char* pszCommand)
{
  switchBaud(115200);

  if(pszCommand[0] != '\0')
  {
    Serial.write(pszCommand);
  }
  Serial.write("\n");

  const int recvBuffLen = readFromSerial();
  if(recvBuffLen > 0)
  {
    return true;
  }

  //wake up console and try again:
  wakeUpConsole();

  if(pszCommand[0] != '\0')
  {
    Serial.write(pszCommand);
  }
  Serial.write("\n");

  return readFromSerial() > 0;
}

void handleReq()
{
  #ifdef AUTHENTICATION
  if (!server.authenticate(www_username, www_password)) {
    return server.requestAuthentication();
  }
  #endif
  bool respOK;
  if(server.hasArg("code") == false)
  {
    respOK = sendCommandAndReadSerialResponse("");
  }
  else
  {
    respOK = sendCommandAndReadSerialResponse(server.arg("code").c_str());
  }

  handleRoot();
}



void handleJsonOut()
{
  #ifdef AUTHENTICATION
  if (!server.authenticate(www_username, www_password)) {
    return server.requestAuthentication();
  }
  #endif
  if(sendCommandAndReadSerialResponse("pwr") == false)
  {
    server.send(500, "text/plain", "Failed to get response to 'pwr' command");
    return;
  }

  parsePwrResponse(g_szRecvBuff);
  prepareJsonOutput(g_szRecvBuff, sizeof(g_szRecvBuff));
  server.send(200, "application/json", g_szRecvBuff);
}

void handleRoot() {
  #ifdef AUTHENTICATION
  if (!server.authenticate(www_username, www_password)) {
    return server.requestAuthentication();
  }
  #endif
  timeClient.update(); //get ntp datetime
  unsigned long days = 0, hours = 0, minutes = 0;
  unsigned long val = os_getCurrentTimeSec();
  days = val / (3600*24);
  val -= days * (3600*24);
  hours = val / 3600;
  val -= hours * 3600;
  minutes = val / 60;
  val -= minutes*60;

  time_t epochTime = timeClient.getEpochTime();
  String formattedTime = timeClient.getFormattedTime();
  //Get a time structure
  struct tm *ptm = gmtime ((time_t *)&epochTime); 
  int currentMonth = ptm->tm_mon+1;

  static char szTmp[9500] = "";  
  long timezone= GMT / 3600;
  snprintf(szTmp, sizeof(szTmp)-1, "<html><b>Pylontech Battery</b><br>Time GMT: %s (%s %d)<br>Uptime: %02d:%02d:%02d.%02d<br><br>free heap: %u<br>Wifi RSSI: %d<BR>Wifi SSID: %s", 
            formattedTime, "GMT ", timezone,
            (int)days, (int)hours, (int)minutes, (int)val, 
            ESP.getFreeHeap(), WiFi.RSSI(), WiFi.SSID().c_str());

  strncat(szTmp, "<BR><a href='/log'>Runtime log</a><HR>", sizeof(szTmp)-1);
  strncat(szTmp, "<form action='/req' method='get'>Command:<input type='text' name='code'/><input type='submit'> <a href='/req?code=pwr'>PWR</a> | <a href='/req?code=pwr%201'>Power 1</a> |  <a href='/req?code=pwr%202'>Power 2</a> | <a href='/req?code=pwr%203'>Power 3</a> | <a href='/req?code=pwr%204'>Power 4</a> | <a href='/req?code=help'>Help</a> | <a href='/req?code=log'>Event Log</a> | <a href='/req?code=time'>Time</a><br>", sizeof(szTmp)-1);
  //strncat(szTmp, "<form action='/req' method='get'>Command:<input type='text' name='code'/><input type='submit'><a href='/req?code=pwr'>Power</a> | <a href='/req?code=help'>Help</a> | <a href='/req?code=log'>Event Log</a> | <a href='/req?code=time'>Time</a><br>", sizeof(szTmp)-1);
  strncat(szTmp, "<textarea rows='80' cols='180'>", sizeof(szTmp)-1);
  //strncat(szTmp, "<textarea rows='45' cols='180'>", sizeof(szTmp)-1);
  strncat(szTmp, g_szRecvBuff, sizeof(szTmp)-1);
  strncat(szTmp, "</textarea></form>", sizeof(szTmp)-1);
  strncat(szTmp, "</html>", sizeof(szTmp)-1);
  //send page
  server.send(200, "text/html", szTmp);
}

unsigned long os_getCurrentTimeSec()
{
  static unsigned int wrapCnt = 0;
  static unsigned long lastVal = 0;
  unsigned long currentVal = millis();

  if(currentVal < lastVal)
  {
    wrapCnt++;
  }

  lastVal = currentVal;
  unsigned long seconds = currentVal/1000;
  
  //millis will wrap each 50 days, as we are interested only in seconds, let's keep the wrap counter
  return (wrapCnt*4294967) + seconds;
}

void wakeUpConsole()
{
  switchBaud(1200);

  //byte wakeUpBuff[] = {0x7E, 0x32, 0x30, 0x30, 0x31, 0x34, 0x36, 0x38, 0x32, 0x43, 0x30, 0x30, 0x34, 0x38, 0x35, 0x32, 0x30, 0x46, 0x43, 0x43, 0x33, 0x0D};
  //Serial.write(wakeUpBuff, sizeof(wakeUpBuff));
  Serial.write("~20014682C0048520FCC3\r");
  delay(1000);

  byte newLineBuff[] = {0x0E, 0x0A};
  switchBaud(115200);
  
  for(int ix=0; ix<10; ix++)
  {
    Serial.write(newLineBuff, sizeof(newLineBuff));
    delay(1000);

    if(Serial.available())
    {
      while(Serial.available())
      {
        Serial.read();
      }
      
      break;
    }
  }
}

#define MAX_PYLON_BATTERIES 8

struct pylonBattery
{
  bool isPresent;
  long  soc;     //Coulomb in %
  long  voltage; //in mW
  long  current; //in mA, negative value is discharge
  long  tempr;   //temp of case or BMS?
  long  cellTempLow;
  long  cellTempHigh;
  long  cellVoltLow;
  long  cellVoltHigh;
  char baseState[9];    //Charge | Dischg | Idle
  char voltageState[9]; //Normal
  char currentState[9]; //Normal
  char tempState[9];    //Normal
  char time[20];        //2019-06-08 04:00:29
  char b_v_st[9];       //Normal  (battery voltage?)
  char b_t_st[9];       //Normal  (battery temperature?)

  bool isCharging()    const { return strcmp(baseState, "Charge")   == 0; }
  bool isDischarging() const { return strcmp(baseState, "Dischg")   == 0; }
  bool isIdle()        const { return strcmp(baseState, "Idle")     == 0; }
  bool isBalancing()   const { return strcmp(baseState, "Balance")  == 0; }
  

  bool isNormal() const
  {
    if(isCharging()    == false &&
       isDischarging() == false &&
       isIdle()        == false &&
       isBalancing()   == false)
    {
      return false; //base state looks wrong!
    }

    return  strcmp(voltageState, "Normal") == 0 &&
            strcmp(currentState, "Normal") == 0 &&
            strcmp(tempState,    "Normal") == 0 &&
            strcmp(b_v_st,       "Normal") == 0 &&
            strcmp(b_t_st,       "Normal") == 0 ;
  }
};

struct batteryStack
{
  int batteryCount;
  int soc;  //in %, if charging: average SOC, otherwise: lowest SOC
  int temp; //in mC, if highest temp is > 15C, this will show the highest temp, otherwise the lowest
  long currentDC;    //mAh current going in or out of the battery
  long avgVoltage;    //in mV
  char baseState[9];  //Charge | Dischg | Idle | Balance | Alarm!

  
  pylonBattery batts[MAX_PYLON_BATTERIES];

  bool isNormal() const
  {
    for(int ix=0; ix<MAX_PYLON_BATTERIES; ix++)
    {
      if(batts[ix].isPresent && 
         batts[ix].isNormal() == false)
      {
        return false;
      }
    }

    return true;
  }

  //in Wh
  long getPowerDC() const
  {
    return (long)(((double)currentDC/1000.0)*((double)avgVoltage/1000.0));
  }

  // power in Wh in charge
  float powerIN() const
  {
    if (currentDC > 0) {
       return (float)(((double)currentDC/1000.0)*((double)avgVoltage/1000.0));
    } else {
       return (float)(0);
    }
  }
  
  // power in Wh in discharge
  float powerOUT() const
  {
    if (currentDC < 0) {
       return (float)(((double)currentDC/1000.0)*((double)avgVoltage/1000.0)*-1);
    } else {
       return (float)(0);
    }
  }

  //Wh estimated current on AC side (taking into account Sofar ME3000SP losses)
  long getEstPowerAc() const
  {
    double powerDC = (double)getPowerDC();
    if(powerDC == 0)
    {
      return 0;
    }
    else if(powerDC < 0)
    {
      //we are discharging, on AC side we will see less power due to losses
      if(powerDC < -1000)
      {
        return (long)(powerDC*0.94);
      }
      else if(powerDC < -600)
      {
        return (long)(powerDC*0.90);
      }
      else
      {
        return (long)(powerDC*0.87);
      }
    }
    else
    {
      //we are charging, on AC side we will have more power due to losses
      if(powerDC > 1000)
      {
        return (long)(powerDC*1.06);
      }
      else if(powerDC > 600)
      {
        return (long)(powerDC*1.1);
      }
      else
      {
        return (long)(powerDC*1.13);
      }
    }
  }
};

batteryStack g_stack;


long extractInt(const char* pStr, int pos)
{
  return atol(pStr+pos);
}

void extractStr(const char* pStr, int pos, char* strOut, int strOutSize)
{
  strOut[strOutSize-1] = '\0';
  strncpy(strOut, pStr+pos, strOutSize-1);
  strOutSize--;
  
  
  //trim right
  while(strOutSize > 0)
  {
    if(isspace(strOut[strOutSize-1]))
    {
      strOut[strOutSize-1] = '\0';
    }
    else
    {
      break;
    }

    strOutSize--;
  }
}

/* Output has mixed \r and \r\n
pwr

@

Power Volt   Curr   Tempr  Tlow   Thigh  Vlow   Vhigh  Base.St  Volt.St  Curr.St  Temp.St  Coulomb  Time                 B.V.St   B.T.St  

1     49735  -1440  22000  19000  19000  3315   3317   Dischg   Normal   Normal   Normal   93%      2019-06-08 04:00:30  Normal   Normal  

....   

8     -      -      -      -      -      -      -      Absent   -        -        -        -        -                    -        -       

Command completed successfully

$$

pylon
*/
bool parsePwrResponse(const char* pStr)
{
  if(strstr(pStr, "Command completed successfully") == NULL)
  {
    return false;
  }
  
  int chargeCnt    = 0;
  int dischargeCnt = 0;
  int idleCnt      = 0;
  int alarmCnt     = 0;
  int socAvg       = 0;
  int socLow       = 0;
  int tempHigh     = 0;
  int tempLow      = 0;

  memset(&g_stack, 0, sizeof(g_stack));

  for(int ix=0; ix<MAX_PYLON_BATTERIES; ix++)
  {
    char szToFind[32] = "";
    snprintf(szToFind, sizeof(szToFind)-1, "\r\r\n%d     ", ix+1);

    const char* pLineStart = strstr(pStr, szToFind);
    if(pLineStart == NULL)
    {
      return false;
    }

    pLineStart += 3; //move past \r\r\n

    extractStr(pLineStart, 55, g_stack.batts[ix].baseState, sizeof(g_stack.batts[ix].baseState));
    if(strcmp(g_stack.batts[ix].baseState, "Absent") == 0)
    {
      g_stack.batts[ix].isPresent = false;
    }
    else
    {
      g_stack.batts[ix].isPresent = true;
      extractStr(pLineStart, 64, g_stack.batts[ix].voltageState, sizeof(g_stack.batts[ix].voltageState));
      extractStr(pLineStart, 73, g_stack.batts[ix].currentState, sizeof(g_stack.batts[ix].currentState));
      extractStr(pLineStart, 82, g_stack.batts[ix].tempState, sizeof(g_stack.batts[ix].tempState));
      extractStr(pLineStart, 100, g_stack.batts[ix].time, sizeof(g_stack.batts[ix].time));
      extractStr(pLineStart, 121, g_stack.batts[ix].b_v_st, sizeof(g_stack.batts[ix].b_v_st));
      extractStr(pLineStart, 130, g_stack.batts[ix].b_t_st, sizeof(g_stack.batts[ix].b_t_st));
      g_stack.batts[ix].voltage = extractInt(pLineStart, 6);
      g_stack.batts[ix].current = extractInt(pLineStart, 13);
      g_stack.batts[ix].tempr   = extractInt(pLineStart, 20);
      g_stack.batts[ix].cellTempLow    = extractInt(pLineStart, 27);
      g_stack.batts[ix].cellTempHigh   = extractInt(pLineStart, 34);
      g_stack.batts[ix].cellVoltLow    = extractInt(pLineStart, 41);
      g_stack.batts[ix].cellVoltHigh   = extractInt(pLineStart, 48);
      g_stack.batts[ix].soc            = extractInt(pLineStart, 91);

      //////////////////////////////// Post-process ////////////////////////
      g_stack.batteryCount++;
      g_stack.currentDC += g_stack.batts[ix].current;
      g_stack.avgVoltage += g_stack.batts[ix].voltage;
      socAvg += g_stack.batts[ix].soc;

      if(g_stack.batts[ix].isNormal() == false){ alarmCnt++; }
      else if(g_stack.batts[ix].isCharging()){chargeCnt++;}
      else if(g_stack.batts[ix].isDischarging()){dischargeCnt++;}
      else if(g_stack.batts[ix].isIdle()){idleCnt++;}
      else{ alarmCnt++; } //should not really happen!

      if(g_stack.batteryCount == 1)
      {
        socLow = g_stack.batts[ix].soc;
        tempLow  = g_stack.batts[ix].cellTempLow;
        tempHigh = g_stack.batts[ix].cellTempHigh;
      }
      else
      {
        if(socLow > g_stack.batts[ix].soc){socLow = g_stack.batts[ix].soc;}
        if(tempHigh < g_stack.batts[ix].cellTempHigh){tempHigh = g_stack.batts[ix].cellTempHigh;}
        if(tempLow > g_stack.batts[ix].cellTempLow){tempLow = g_stack.batts[ix].cellTempLow;}
      }
      
    }
  }

  //now update stack state:
  g_stack.avgVoltage /= g_stack.batteryCount;
  g_stack.soc = socLow;

  if(tempHigh > 15000) //15C
  {
    g_stack.temp = tempHigh; //in the summer we highlight the warmest cell
  }
  else
  {
    g_stack.temp = tempLow; //in the winter we focus on coldest cell
  }

  if(alarmCnt > 0)
  {
    strcpy(g_stack.baseState, "Alarm!");
  }
  else if(chargeCnt == g_stack.batteryCount)
  {
    strcpy(g_stack.baseState, "Charge");
    g_stack.soc = (int)(socAvg / g_stack.batteryCount);
  }
  else if(dischargeCnt == g_stack.batteryCount)
  {
    strcpy(g_stack.baseState, "Dischg");
  }
  else if(idleCnt == g_stack.batteryCount)
  {
    strcpy(g_stack.baseState, "Idle");
  }
  else
  {
    strcpy(g_stack.baseState, "Balance");
  }


  return true;
}

void prepareJsonOutput(char* pBuff, int buffSize)
{
  memset(pBuff, 0, buffSize);
  snprintf(pBuff, buffSize-1, "{\"soc\": %d, \"temp\": %d, \"currentDC\": %ld, \"avgVoltage\": %ld, \"baseState\": \"%s\", \"batteryCount\": %d, \"powerDC\": %ld, \"estPowerAC\": %ld, \"isNormal\": %s}", g_stack.soc, 
                                                                                                                                                                                                            g_stack.temp, 
                                                                                                                                                                                                            g_stack.currentDC, 
                                                                                                                                                                                                            g_stack.avgVoltage, 
                                                                                                                                                                                                            g_stack.baseState, 
                                                                                                                                                                                                            g_stack.batteryCount, 
                                                                                                                                                                                                            g_stack.getPowerDC(), 
                                                                                                                                                                                                            g_stack.getEstPowerAc(),
                                                                                                                                                                                                            g_stack.isNormal() ? "true" : "false");
}

void loop() {
#ifdef ENABLE_MQTT
  mqttLoop();
#endif
  
  ArduinoOTA.handle();
  server.handleClient();

  //if there are bytes availbe on serial here - it's unexpected
  //when we send a command to battery, we read whole response
  //if we get anything here anyways - we will log it
  int bytesAv = Serial.available();
  if(bytesAv > 0)
  {
    if(bytesAv > 63)
    {
      bytesAv = 63;
    }
    
    char buff[64+4] = "RCV:";
    if(Serial.readBytes(buff+4, bytesAv) > 0)
    {
      digitalWrite(LED_BUILTIN, LOW);
      delay(5);
      digitalWrite(LED_BUILTIN, HIGH);//high is off

      Log(buff);
    }
  }
}

#ifdef ENABLE_MQTT
#define ABS_DIFF(a, b) (a > b ? a-b : b-a)
void mqtt_publish_f(const char* topic, float newValue, float oldValue, float minDiff, bool force)
{
  char szTmp[16] = "";
  snprintf(szTmp, 15, "%.2f", newValue);
  if(force || ABS_DIFF(newValue, oldValue) > minDiff)
  {
    mqttClient.publish(topic, szTmp, false);
  }
}

void mqtt_publish_i(const char* topic, int newValue, int oldValue, int minDiff, bool force)
{
  char szTmp[16] = "";
  snprintf(szTmp, 15, "%d", newValue);
  if(force || ABS_DIFF(newValue, oldValue) > minDiff)
  {
    mqttClient.publish(topic, szTmp, false);
  }
}

void mqtt_publish_s(const char* topic, const char* newValue, const char* oldValue, bool force)
{
  if(force || strcmp(newValue, oldValue) != 0)
  {
    mqttClient.publish(topic, newValue, false);
  }
}

void pushBatteryDataToMqtt(const batteryStack& lastSentData, bool forceUpdate /* if true - we will send all data regardless if it's the same */)
{
  mqtt_publish_f(MQTT_TOPIC_ROOT "soc",          g_stack.soc,                lastSentData.soc,                0, forceUpdate);
  mqtt_publish_f(MQTT_TOPIC_ROOT "temp",         (float)g_stack.temp/1000.0, (float)lastSentData.temp/1000.0, 0.1, forceUpdate);
  mqtt_publish_i(MQTT_TOPIC_ROOT "currentDC",    g_stack.currentDC,          lastSentData.currentDC,          1, forceUpdate);
  mqtt_publish_i(MQTT_TOPIC_ROOT "estPowerAC",   g_stack.getEstPowerAc(),    lastSentData.getEstPowerAc(),   10, forceUpdate);
  mqtt_publish_i(MQTT_TOPIC_ROOT "battery_count",g_stack.batteryCount,       lastSentData.batteryCount,       0, forceUpdate);
  mqtt_publish_s(MQTT_TOPIC_ROOT "base_state",   g_stack.baseState,          lastSentData.baseState            , forceUpdate);
  mqtt_publish_i(MQTT_TOPIC_ROOT "is_normal",    g_stack.isNormal() ? 1:0,   lastSentData.isNormal() ? 1:0,   0, forceUpdate);
  mqtt_publish_i(MQTT_TOPIC_ROOT "getPowerDC",   g_stack.getPowerDC(),       lastSentData.getPowerDC(),       1, forceUpdate);
  mqtt_publish_i(MQTT_TOPIC_ROOT "powerIN",      g_stack.powerIN(),          lastSentData.powerIN(),          1, forceUpdate);
  mqtt_publish_i(MQTT_TOPIC_ROOT "powerOUT",     g_stack.powerOUT(),         lastSentData.powerOUT(),         1, forceUpdate);

  // publishing details
  for (int ix = 0; ix < g_stack.batteryCount; ix++) {
    char ixBuff[50];
    String ixBattStr = MQTT_TOPIC_ROOT + String(ix) + "/voltage";
    ixBattStr.toCharArray(ixBuff, 50);
    mqtt_publish_f(ixBuff, g_stack.batts[ix].voltage / 1000.0, lastSentData.batts[ix].voltage / 1000.0, 0, forceUpdate);
    ixBattStr = MQTT_TOPIC_ROOT + String(ix) + "/current";
    ixBattStr.toCharArray(ixBuff, 50);
    mqtt_publish_f(ixBuff, g_stack.batts[ix].current / 1000.0, lastSentData.batts[ix].current / 1000.0, 0, forceUpdate);
    ixBattStr = MQTT_TOPIC_ROOT + String(ix) + "/soc";
    ixBattStr.toCharArray(ixBuff, 50);
    mqtt_publish_i(ixBuff, g_stack.batts[ix].soc, lastSentData.batts[ix].soc, 0, forceUpdate);
    ixBattStr = MQTT_TOPIC_ROOT + String(ix) + "/charging";
    ixBattStr.toCharArray(ixBuff, 50);
    mqtt_publish_i(ixBuff, g_stack.batts[ix].isCharging()?1:0, lastSentData.batts[ix].isCharging()?1:0, 0, forceUpdate);
    ixBattStr = MQTT_TOPIC_ROOT + String(ix) + "/discharging";
    ixBattStr.toCharArray(ixBuff, 50);
    mqtt_publish_i(ixBuff, g_stack.batts[ix].isDischarging()?1:0, lastSentData.batts[ix].isDischarging()?1:0, 0, forceUpdate);
    ixBattStr = MQTT_TOPIC_ROOT + String(ix) + "/idle";
    ixBattStr.toCharArray(ixBuff, 50);
    mqtt_publish_i(ixBuff, g_stack.batts[ix].isIdle()?1:0, lastSentData.batts[ix].isIdle()?1:0, 0, forceUpdate);
    ixBattStr = MQTT_TOPIC_ROOT + String(ix) + "/state";
    ixBattStr.toCharArray(ixBuff, 50);
    mqtt_publish_s(ixBuff, g_stack.batts[ix].isIdle()?"Idle":g_stack.batts[ix].isCharging()?"Charging":g_stack.batts[ix].isDischarging()?"Discharging":"", lastSentData.batts[ix].isIdle()?"Idle":lastSentData.batts[ix].isCharging()?"Charging":lastSentData.batts[ix].isDischarging()?"Discharging":"", forceUpdate);
    ixBattStr = MQTT_TOPIC_ROOT + String(ix) + "/temp";
    ixBattStr.toCharArray(ixBuff, 50);
    mqtt_publish_f(ixBuff, (float)g_stack.batts[ix].tempr/1000.0, (float)lastSentData.batts[ix].tempr/1000.0, 0.1, forceUpdate);
  }
} 

void mqttLoop()
{
  //if we have problems with connecting to mqtt server, we will attempt to re-estabish connection each 1minute (not more than that)
  static unsigned long g_lastConnectionAttempt = 0;

  //first: let's make sure we are connected to mqtt
  const char* topicLastWill = MQTT_TOPIC_ROOT "availability";
  if (!mqttClient.connected() && (g_lastConnectionAttempt == 0 || os_getCurrentTimeSec() - g_lastConnectionAttempt > 60)) {
    if(mqttClient.connect(WIFI_HOSTNAME, MQTT_USER, MQTT_PASSWORD, topicLastWill, 1, true, "offline"))
    {
      Log("Connected to MQTT server: " MQTT_SERVER);
      mqttClient.publish(topicLastWill, "online", true);
    }
    else
    {
      Log("Failed to connect to MQTT server.");
    }

    g_lastConnectionAttempt = os_getCurrentTimeSec();
  }

  //next: read data from battery and send via MQTT (but only once per MQTT_PUSH_FREQ_SEC seconds)
  static unsigned long g_lastDataSent = 0;
  if(mqttClient.connected() && 
     os_getCurrentTimeSec() - g_lastDataSent > MQTT_PUSH_FREQ_SEC &&
     sendCommandAndReadSerialResponse("pwr") == true)
  {
    static batteryStack lastSentData; //this is the last state we sent to MQTT, used to prevent sending the same data over and over again
    static unsigned int callCnt = 0;
    
    parsePwrResponse(g_szRecvBuff);

    bool forceUpdate = (callCnt % 20 == 0); //push all the data every 20th call
    pushBatteryDataToMqtt(lastSentData, forceUpdate);
    
    callCnt++;
    g_lastDataSent = os_getCurrentTimeSec();
    memcpy(&lastSentData, &g_stack, sizeof(batteryStack));
  }
  
  mqttClient.loop();
}

#endif //ENABLE_MQTT

 

Read EVU smart meters with ESP32 and ESPhome and use them in Homeassistant

Loading

edit 7.11.24
In the meantime, I have also layouted an interface board with a USB type B socket for the 5V supply. (see layout below). Because as small and fine as the micro USB plugs are, I need something more robust.

new board version with USB type B socket for power supply

As I am asked more and more often for the production data, I am making the Gerber data of the circuit boards available for download:

ESP32_interface_2023-05-12

interface_usbB_2024-06-27


In the article entitled: “Reading energy supply company smart meters with ESP32 and sending data via MQTT” (link), I described how the energy supply companies’ smart meters can be read out via the customer interface. The measurement data is then available as topics via the mqtt broker and can be further processed in various home automation systems (HomeMatic, Homeassistant, etc.). All you need is an ESP32 board and a few small parts to establish the connection to the smart meter. As a small update, I have now embellished the structure (back then with pin headers on a breadboard) a little and made a circuit board.

Layout preview in designtool

The associated circuit diagram essentially corresponds to the sketch in the previous article. To make things a little more convenient with the new circuit board, the connection to the customer interface of the smart meter can be plugged in via an RJ socket. I have also implemented the power supply via a USB socket.

Once the ESP32 circuit board had been fitted and plugged in, the device was given a small housing and is now doing its job in the electrical distribution cabinet.

The hardware is therefore ready and functional. I have also considered changing something about the software. Until now, the ESP was running a program that decrypted the data from the smart meter and then sent it to the IP address of the broker via MQTT. However, as I am now also a user of the ESPHome integration in my HomeAssistant environment, I have flashed the ESP with an ESPHome base image. On GitHub there is the repository of Andre-Schuiki, where he publishes a version for ISKRA and SIEMENS Smartmeter for use with ESPHome. The installation instructions can be found under the following link: https://github.com/Andre-Schuiki/esphome_im350/tree/main/esp_home

The script for the ESPHome device looks like this:

 esphome:  
  name: kelagsmartmeter  
  friendly_name: KelagSmartmeter  
  libraries:  
  - "Crypto" # !IMPORTANT! we need this library for decryption!  
 esp32:  
  board: esp32dev  
  framework:  
   type: arduino  
 # Enable logging  
 logger:  
 # Enable Home Assistant API  
 api:  
  encryption:  
   key: "da kommt der key rein des neu angelegten ESPHome Gerätes rein"  
 ota:  
  password: "das automatisch generierte ota passwort"  
 wifi:  
  ssid: !secret wifi_ssid  
  password: !secret wifi_password  
  # Enable fallback hotspot (captive portal) in case wifi connection fails  
  ap:  
   ssid: "Kelagsmartmeter Fallback Hotspot"  
   password: "das automatisch generierte password"  
 captive_portal:  
 external_components:  
  - source:  
    type: local  
    path: custom_esphome  
 sensor:  
  - platform: siemens_im350  
   update_interval: 5s  
   trigger_pin: 26 # this pin goes to pin 2 of the customer interface and will be set to high before we try to read the data from the rx pin  
   rx_pin: 16 # this pin goes to pin 5 of the customer interface  
   tx_pin: 17 # not connected at the moment, i added it just in case we need it in the future..  
   decryption_key: "00AA01BB02CC03DD04EE05FF06AA07BB" # you get the key from your provider!  
   use_test_data: false # that was just for debugging, if you set it to true data are not read from serial and the test_data string is used  
   test_data: "7EA077CF022313BB45E6E700DB0849534B697460B6FA5F200005C8606F536D06C32A190761E80A97E895CECA358D0A0EFD7E9C47A005C0F65B810D37FB0DA2AD6AB95F7F372F2AB11560E2971B914A5F8BFF5E06D3AEFBCD95B244A373C5DBDA78592ED2C1731488D50C0EC295E9056B306F4394CDA7D0FC7E0000"  
   delay_before_reading_data: 1000 # this is needed because we have to wait for the interface to power up, you can try to lower this value but 1 sec was ok for me  
   max_wait_time_for_reading_data: 1100 # maximum time to read the 123 Bytes (just in case we get no data)  
   ntp_server: "pool.ntp.org" #if no ntp is specified pool.ntp.org is used  
   ntp_gmt_offset: 3600  
   ntp_daylight_offset: 3600  
   counter_reading_p_in:  
    name: reading_p_in  
    filters:  
     - lambda: return x / 1000;  
    unit_of_measurement: kWh  
    accuracy_decimals: 3  
    device_class: energy  
   counter_reading_p_out:  
    name: reading_p_out  
    filters:  
     - lambda: return x / 1000;  
    unit_of_measurement: kWh  
    accuracy_decimals: 3  
    device_class: energy  
   counter_reading_q_in:  
    name: reading_q_in  
    filters:  
     - lambda: return x / 1000;  
    unit_of_measurement: kvarh  
    device_class: energy  
   counter_reading_q_out:  
    name: reading_q_out  
    filters:  
     - lambda: return x / 1000;  
    unit_of_measurement: kvarh  
    device_class: energy  
   current_power_usage_in:  
    name: power_usage_in  
    filters:  
     - lambda: return x / 1000;  
    unit_of_measurement: kW  
    accuracy_decimals: 3  
    device_class: energy  
   current_power_usage_out:  
    name: power_usage_out  
    filters:  
     - lambda: return x / 1000;  
    unit_of_measurement: kW  
    accuracy_decimals: 3  
    device_class: energy  
  # Extra sensor to keep track of uptime  
  - platform: uptime  
   name: IM350_Uptime Sensor  
 switch:  
  - platform: restart  
   name: IM350_Restart  

 

“Tricky Traps” restoration

Loading

In this post, I’m going to take a look at the restoration – or rather repair – of a handheld game that I recently received for review. It has the name “Tricky Traps”. This means something like “tricky or tricky traps”

The game “Tricky Traps” by Tomy is a mechanical game of skill that was originally released in the 1970s. It consists of a maze-like playing field in which the player must navigate a small metal ball through a series of obstacles and traps. The aim of the game is to successfully maneuver the ball through the maze to the finish without it falling into one of the many traps. There are five balls available. The game is timed.

Tricky Traps

Game mechanics:

  • The player starts the game with a rotary knob, which sets a small electric motor in motion. This motor drives the “traps” and also the rotary knob itself via a gearbox. This is how the “timer” is realized. Once the rotary knob has completed about three quarters of a turn, the motor stops again and the game is over. This is solved by a contact spring, which is pressed down onto a mating contact by a small bar at the bottom of the rotary knob.
  • Once the game has started, the red button can be used to release a ball into the track. The white button at the bottom is the actual and only game button. It lifts the ball with a small cylinder so that it can move through the various parts of the playing field. You have to do this with the right timing.
  • There are numerous obstacles on the playing field, such as rotating disks, small ramps, narrow passages and a rotating magnet that can stop the ball or cause it to fall into a trap.

The colorful design is typical of the mechanical games of the 70s and 80s. It is made of plastic and the moving parts are usually brightly colored.

The technical problems that occur often in such old games are:

  • leaking batteries, which usually cause corrosion and destruction of the contacts
  • Brittle plastic, which mainly occurs with gearwheels that are mounted on brass shafts and therefore start to slip. This also means that housing gears often no longer hold together properly.
  • Electric motors whose brushes are worn so that they no longer turn
  • Resinous grease and oils that make moving parts sluggish
  • Wires and electrical connections that are corroded and broken

All of these points can be found very often during restoration and must be fixed. This can also be done more or less easily. After carefully opening and inspecting the appliance, I actually start by completely dismantling and cleaning the parts. Then I try to repair any broken plastic parts. Here I use various adhesives as far as possible. Sometimes it is also necessary to reproduce a part with a 3D printer. Of course, this assumes that enough of the original part is still available to reconstruct it accurately. The electrical components of these devices are the easiest to repair, as there are usually no electronics with any components with ICs that are no longer manufactured.

Revealing the parts after disassembly

Gearbox
Assembly after cleaning

 

“Ball” button to start the ball

Keysight oscilloscope dies in standby – power supply repair

Loading

An interesting problem has arisen with the measurement technology in the laboratories at my workplace. I use the term “measurement technology” to describe the equipment of a laboratory workstation for basic training. There are a total of nineteen laboratory workstations, each equipped with two laboratory power supplies, two desktop multimeters, a Keysight signal generator and a Keysight (Agilent) oscilloscope of the Infiniivision DSO-X 20xx series. All devices are network-compatible and are connected to the corresponding workstation computer via LAN. This means that the measuring devices can be accessed using different software (Agilent VEE, Matlab, LabVIEW etc.). The devices were purchased around three years ago and replace the almost twenty-year-old laboratory equipment.

However, it has now happened that the DSO-X2012A oscilloscope at one workstation no longer showed any signs of life. It occasionally happens that during laboratory exercises or when working freely in the laboratories, a student presses the emergency stop switch of the workstation and thus de-energizes it. But this was not the case. All the devices connected to the workstation worked, with the exception of the DSO. Voltage could also be measured at the end of the IEC plug. So the problem could only be with the oscilloscope itself. The rear panel is quickly unscrewed, a shield plate removed and the power supply unit is exposed. The first visual inspection immediately revealed the large filter capacitor with its upwardly curved cap. But first things first.

Power supply unit of the Infiniivision

The mains voltage was measured at the AC pins of the mains input, but no DC voltage was measured at any of the outputs of the power supply unit. Regardless of whether the power switch of the device was switched on or off. This suggests that the power supply unit is defective.

Input fusing

First, the power supply unit was removed and examined, starting with the AC input side. The print fuse in the area of the mains filter was the first defective component to be noticed. It is a slow-blow 6.3A/250V fuse. As a blown fuse always has a reason to switch off, the search continued. The mains rectifiers were OK, but the 100uF / 420V electrolytic capacitor, which is used to smooth the DC voltage on the primary side, had already suffered some thermal damage and was bloated.

original electrolytic capacitor 100uF /420V /105°C

Its capacity was also no longer within the nominal range. But even that was not the direct reason for the fuse blowing. This was quickly found. A mosfet used to control the transformer was low-resistance. More precisely, it had a short circuit between all the connections.

Mosfet STP12NM50

The following picture shows the installation positions of the components. These have been replaced. The mosfet was replaced with an original type and the power supply capacitor was replaced with a 100uF / 450V / 105°C type. Although it is about five millimeters higher, it fits easily into the power supply unit.

Installation position of the capacitor and the mosfet

Two SMD resistors on the back of the power supply board were defective in the area of the gate connection of the mosfet. These were an SMD resistor of size 0805 with 5.11 Ohm and an SMD resistor of size 1206 with 2.0 kOhm. The picture below also shows the installation position.

Mounting position of defective SMD resistors

After all the components mentioned had been replaced, a first functional test was carried out. However, this was sobering, as the power supply unit was still not working. The fuse remained intact and the DC voltage on the primary side was stable. But the gate of the mosfet was not activated – unfortunately. Because now came the time-consuming part of the repair. On the power supply board, installed upright, there is another board on which several controller ICs are installed. If you follow the gate line from the Mosfet, it ends at a pin on this control board. So this must be removed.

Controller board removed

To do this, the cooling plate had to be removed first. Then it became a bit tedious, because the controller board is not connected to the main board via a pin header or plug connection, but the contact pins are laid out and milled out. This means that the desoldering work has to be carried out very carefully so as not to damage the conductor tracks at the ends of the milled pins.

Mainboard without controller board

UC3842B

Once the removal was successfully completed, the controller board could be inspected. Lo and behold, the line routed from the gate of the mosfet ends at pin 6 of a small IC. This is a UC3842B VD1R2G. The housing of this IC was blown up. In addition to the controller IC, a SOT23 PNP transistor (PMBT 2907A) was also dead and had a low resistance on all pins.

Installation position of the defective components

After replacing the defective components, the power supply unit was reassembled and a function test was carried out. The oscilloscope started up again and the power supply unit did its job.

defekte Bauteile
Result after successful repair

It would now be interesting to find out why the power supply gave up the ghost after just three years. Especially as the oscilloscopes do not run continuously, but are only switched on during the relevant courses. We noticed the following: The oscilloscope is permanently connected to the power supply. However, the oscilloscope’s power switch does not switch off the AC supply, but only the controller control in the secondary area of the power supply unit. This means that the power supply unit operates in standby mode when it is switched off. And we have noticed that all oscilloscopes that are switched off have a power loss in standby that heats up the mosfets and especially the 100uF electrolytic capacitor. This would explain the bloated, dried-out electrolytic capacitor and the subsequent death of the power supply units. To verify this, the temperature of the components was measured on several devices that had not been switched on for days.

 
Thermal sensor on the electrolytic capacitor surface

The following could be determined here. Both the surface of the capacitor and the cooling plate of the mosfets measured temperatures of 56°C to almost 60°C when switched off. Should this be the case?

Temperature measurement on the electrolytic capacitor

 

Here are the required components:

  • resistor 5R11 0,1W 0,1% Farnell Nr.: 1872688
  • resistor 2k0 0.66W Farnell Nr.: 721-9844
  • PNP Transistor SOT23, SMD Stempel 2F Type PMBT2907A, 215 Farnell Nr.: 1626500
  • PWM Controller IC, UC3842B VD1R2G / 500kHz Farnell Nr.:2845218
  • capacitor 100uF / 105°C / 450V
  • fuse T6.3A 250V

Jun2019: Order numbers updated

 

 

 

Keysight DSO-X 2012A oscilloscope dies in standby – power supply replacement

Loading

In an old post from 2019 I reported about oscilloscopes from the manufacturer Keysight and their problem with a sudden failure. (see link). At that time, it was about oscilloscopes suddenly refusing to work – sometimes with a bang and subsequent smell of “power”. Or simply nothing happened at all after switching on. The reason was and is always the failure of the installed 12VDC power supply CCH0123F1-Z03A. The oscilloscope is designed in such a way that the power supply is still connected to the mains when the “main switch” of the oscilloscope is switched off and is operated in standby mode. The push button switch located on the front panel of the unit then switches the power supply into PowerON mode and 12V power on.

If the devices in the laboratories are permanently connected to live sockets, it is not surprising that the devices age faster than the good old boxes with the cathode ray tubes. The parts, which fall victim to the permanent power supply by thermal continuous load, I have, as well as also the repair expenditure in the contribution at that time represented. On the part of the distributing companies also a reordering or a replacement delivery of new power supplies is not intended or desired. If the devices fail within the warranty period, the replacement by the manufacturer is no problem. If the devices fail after a few years in the laboratory or workshop (it doesn’t matter if they are in use every day, or just stand around plugged in and switched off), then a normal repair service process is carried out by the manufacturer. There are then also the proper tariffs for the service of measurement technology to pay.

In the picture above: “new Meanwell Powersupply” below: “original Lineage Power”.

The power supplies are quite easy to repair, as described in the old report. However, the repair is also quite time-consuming. Of course, it is faster to install a new power supply. Unfortunately, the distributors of the Keysight oscilloscopes do not offer spare parts support and I could not find a direct supplier of the original Lineage power supply. But there is another alternative: In the forum of the EEV blog some users have found alternative power supplies that fit into the DSO-X oscilloscopes. A suitable model is the RPSG-160-12 from Meanwell. It is a 12V 160W power supply. The designation “G” in RPSG indicates that there is also a 5V standby supply. And it is exactly this function that the DSO-X needs. Because as described before, the front switch on the osci is not intended to disconnect the primary side of the mains supply, but only to switch a line on the DC low voltage connector to ground. This line controls the “PowerON pin” in the power supply.

Mechanically, the Meanwell almost fits on the mounting brackets of the DSO-X. “Almost” means that the distance between the mounting holes of the long side on the powersupply is about one millimeter further apart than the mounting bolts on the chassis of the oscilloscope. However, this can be quickly adjusted with a small round file or a 4mm drill bit. Now the Meanwell Powersupply can be attached to the oscilloscope chassis with the original Torx screws. The plug connection for the AC supply from the OSZI board to the power supply can be taken directly from the old power supply.

Pinout of the Meanwell connector strips

The 12V voltage supply at CN2 of the power supply is connected to pins 1,2,3,4 (+12V) and 5,6,7,8 (GND). The connection line to the oscillator must be adapted accordingly.

DC12V outputs on CN2

The picture below shows the pins of the connector strip on the oscilloscope labeled.

DC12V input to oscilloscope

I re-pinned the wires to fit and connected the connector to the powersupply as shown below.

The main power supply to the oscilloscope is now established. Only the “power-on line” (PowerOn) is missing. For this I disconnected the 7th pin (GND) and the 9th pin (Switch) from the old connector and soldered them directly to the standby board of the power supply. The wire at the lowest pin of the standby board is the signal “PowerOn” and the one above is GND. So the power supply can be powered up with the front power switch on the oscilloscope.

“Control lines” for the PowerOn of the power supply unit

 

General view of the wiring

After a short function test and checking of the voltage (can be corrected if necessary also at the trim potentiometer at the power supply) the rebuilding is finished and the assembly can take place again.

EVU Smartmeter read out with ESP32 and send data via MQTT

Loading

Little by little, I am bringing many of my smarthome components to a common standard. I have decided to bring all devices together via a NodeRed server. The HomeMatic system also communicates with NodeRed. Among other things, I also transfer the measured values of the EVU meter (I have a Siemens IM350 smart meter installed) to the HomeMatic CCU. This is done as mentioned in an earlier post, via the LED pulse interface (1000 pulses/kWh). For this purpose, a phototransistor is simply attached above the LED on the meter, which detects the flashing pulses of the LED and converts them into the instantaneous power in the meter sensor transmitter unit HM-ES-TX-WM and integrates them over time and then sends the data on to the CCU. This works quite well in itself. Only the update rate (in the minute range) is too long for me. Also, the phototransistor seems to react again and again to the stray light of the neighboring LED (which displays the reactive power in 1000 pulses/kvarh). This causes discrepancies between the count via the HomeMatic sensor and the values read directly from the meter.

This is definitely more accurate. If you look at the IM350 Smartmeter meter in detail, or read through the manual, you will quickly see that it has a so-called “customer interface”. This customer interface provides some measurement data via a galvanically isolated data line every second. This includes, among others, the momentary active power in both directions, as well as the meter readings of active and reactive power in the reference and feed-in direction. So perfect starting conditions to replace the HomeMatic meter sensor with my own design. After a little Internet research, I quickly realized that I am not the only one who deals with exactly this issue. The data of the customer interface tumbles out after request over a data request line with a speed of 115kbaud. However, they are encrypted, and not directly readable. To obtain the 16-byte decryption key, the utility must be consulted. The key is tied to the smart meter serial number and is unique to each smart meter. After some phone calls with my Carinthian energy provider, the key code was sent to me by mail. In the next step I tested with a USB-UART adapter on a PC, if data really come out of the meter when the interface is wired correctly. For this I crimped a RJ11 connector to a suitable 6pin cable and wired the open end of the cable according to the datasheet of the meter. Not much is needed for this. A 5V supply must activate the interface, likewise the Data Request line must be switched to 5V and already the data packets are available at the Data Out line. By the way, it also works with a 3V3 supply. With a terminal program on the PC (I usually use putty or hterm) you can visualize the encrypted data.

Now it was time to think about how to decode and process the data. For this, one finds two approaches with net:

* via a RaspberryPi, with a Python environment and a Python script. The scripts here take over the reception and decryption of the data and then make them available for further processing in different ways

* via an ESP32. The ESP is also able to decode a 128Bit AES encryption and still has plenty of resources to process the data and send it via WiFi. Furthermore, an ESP is available in sufficient quantities for little money. So I decided to use this solution. There is an open source project on GitHub from the user https://github.com/Andre-Schuiki/esphome_im350 in which he provides an ESP32 IM350 decoder as a basis for own projects. With his sources you get a decoder that reads the meter data every second and outputs it via the USB UART programming interface and also via Telnet over WiFi. I used this source as a basis.

My goal is to put the data obtained from the smart meter into MQTT messages and send them to my MQTT broker. From there it is then a simple matter to get them into NodeRed and the HomeMatic CCU and store them there. So I adapted the code. This involved setting the wifi connection to the router to a static IP. (are to be defined in settings.h). The readout readings, as well as the RSSI of the wifi connection, are now provided via MQTT Topics. (the IP address to the broker is also to be defined in settings.h). If you compile the code now and run it on the ESP, then it should log into the respective network. As long as the ESP is still connected to a PC, you can check what it is doing via the programming interface and a terminal. If you now connect the RJ11 plug to the customer interface of the meter, the triangle above the label “KU” should flash in the display of the meter every second. If this happens, the measured values should already be displayed in the terminal (provided that you have not forgotten to enter the KEY from the utility in secrets.h). If this also works, then a look at the MQTT broker (with e.g.: MQTT Explorer) makes sure that the messages arrive. Now the ESP can be removed from the PC.

Connection assignment
ESP32 in “free-flying” test setup

I chose a very simple solution and mounted the ESP on a breadboard. The 6pin cable to the smartmeter is soldered there. On the breadboard there is room for the pull-up resistors and a NPN transistor (BC547 etc.) for inverting the data pulses. I put the board in a small plastic box, which is now only connected with a cable to the customer interface and with a USB cable to a USB power supply.

The finished structure then (or currently) looks like this. The data ends up in the MQTT broker and NodeRed visualizes it and sends it to the HomeMatic CCU.

this is how the data arrives at the MQTT broker
and can be processed in NodeRed like this

if someone is interested in the customized scripts, I can send them to you. Regarding a publication on GitHub, I have to find out first which license conditions have to be fulfilled concerning the original repository. It will then be available here (public):

https://github.com/ingmarsretro/esphome_im350/tree/main/standalone_version_mqtt

Integrating the heat pump (NEURA) into the Smarthome

Loading

A smarthome is no longer a rarity today and is very widespread. There are countless systems on the market that make your own home “smart”. The digital voice assistants from Google, Amazon and co. in conjunction with smart light bulbs are among the systems that are easy and quick to install. But there are also complex smart home systems, in which actuators for every lamp and socket are installed in the house distributors. The windows and doors are equipped with signaling contacts and secure the home or report if once forgotten to close the windows after shock ventilation. It goes without saying that these systems also contribute to energy optimization when programmed sensibly. I also operate Smarthome components from various manufacturers.

For years, this has included the HomeMatic system, which communicates with its actuators and sensors both wired and via the Bidcos protocol. The HUE system from Phillips talks to its smart lamps and sockets via ZigBee. The gateways of these systems are connected to a LAN network and each system brings its own web server, through which it can then be controlled and set. An inverter of photovoltaic systems can provide its data via different interfaces (RS485, CAN, RS232). To bring all of them to a central display level, I decided to use the NodeRed system. The necessary NodeRed server runs on a Raspberry PI. (On the CCU3 with the Raspbian image is still enough space to run the NodeRed server – it is even available as a separate plugin for the CCU and is called “RedMatic”). With this configuration you can “slay” almost everything in the field of home automation. With ESP32 and Raspberry you can easily transfer status information via MQTT (Message Queueing Telemetry Transport). I use this for example with the small feed-in inverters of a balcony PV system, as well as with the PV inverters of an offgrid system. Here the data is received via different bus systems in the Raspberry or ESP32 and converted into the MQTT protocol. The MQTT broker collects the data from the individual devices and via NodeRed they can then be written to a database, visualized in the browser or on the smartphone and also easily processed in the HomeMatic system, as required.

Example of a smart home network

Thus, it is possible to network almost all systems with each other smartly and, importantly for me, to visualize them on ONE platform. One single system was missing until now. That is my old Neura heating heat pump. The company Neura has not existed for several years and the web server “webidalog” developed by “b.i.t.” has never been updated. So the heat pump has a web server on a small with Linux computer onboard and builds the web application with an ancient Java version. For the operation a Java Runtime must be installed on the PC, which runs only with some tricks on a current Windows computer (keyword: virtualization). For the operation via a smartphone an html – version with limited functionality is available. My plan now was to find an interface, with which I can read out the data of the heat pump at least once, in order to have flow- return temperatures of the floor heating, boiler temperature, etc. also available in my NodeRed system. But since there is almost no documentation for the system and reverse engineering is a bit critical if the system should continue to run, I came up with the following idea:

With a “headles browser” it should be possible to parse the html version of the Neura WebDialog website and find the relevant data and turn it into MQTT topics via variables. And here I have to give a special thanks to my colleague Mario Wehr, who built the software structure to parse the website. The software is written in PHP and finally runs on a Raspberry PI. All you need is a php8-cli runtime and a few modules. The way the software works is that every time the heat pump website is called, a login is executed, then the data is parsed and sent to MQTT broker. The continuous calling of the php script I then simply solved with a cronjob that is executed every minute.

 

>sudo crontab -e

and the job then looks like this:

* * * * * sudo php /home/neura2mqtt/neura2mqtt.php -c

(if you put the files in the /home/ directory…). I have published the project on github at: https://github.com/ingmarsretro/neura2mqtt.

Neura data on the NodeRed dashboard

 

 

 

 

A rebuild project for the Vectrex

Loading

It’s been some time again that I manage to find time and energy in the later evening hours to write here on the blog about one of my little projects. Over the past few years, I’ve gotten into the habit of listening to podcasts during car rides and at night. These primarily include podcasts on technical topics. Among them is a podcast called “Retrokompott” which is about home computers and technology from our youth. Their tagline is:

Retrokompott, eine Zeitreise in die Vergangenheit alter Homecomputer, Spielekonsolen und Games

[http://blog.retrokompott.de/]

In one of the contributions of Retrokompott one discussed for some episodes (172-177) about the Vectrex, the home – vector game machine of MBE. Among other things, homebrew projects, i.e. software developments of the users, were presented.  “Vectorblade” is a game title, which was developed by Malban [http://vide.malban.de/]. The project was created with the Vectrexcompiler (vide), also developed by Malban. The sources are publicly available on the website. In the “compote” article, people were so enthusiastic about Vectorblade that my interest was piqued. The game module was also available for purchase through Malban for a while. However, I have not found a source through which I can easily purchase the module. So I thought, I’ll just rebuild it for myself. The special thing about this gamerom is the size of the game. It has 192 kB. To address this memory, Malban used bank switching technology. He uses a flash memory from SST, the SST39SF020, in his design. The bank switching is controlled by a quad 2-input NAND Schmitt trigger (74AC132). Malban has published on git the layout. There he uses the memory in the DIL package and also the AC132. Detailed instructions can be found here.

Since I still have some boards left over from my old homebuilt Rom module project, I was able to quickly put together a test setup.  I didn’t have any flash memory available – but a sufficiently large EPROM. The video compiler and the source files are also published on Malban’s GIT. After a short study of his vide-compiler I managed to compile the project and create a ROM – file. With my “Far East Programmer” I could then “burn” the EPROM.  With a few wire bridges and an AC132, my old ROM board project then became the Vectorblade experimental setup.

Vectorblade test setup

With the exception that no settings can be saved, the test setup works and the game can be played :). The next step of the rebuild was to draw the PCB. Here I wanted to build in the Schmitt-Trigger device in SMD design and the SST still in DIL. I also realized this design and tested it successfully. But there is a little catch – none of my suppliers has the SST39SF020 flash memory in DIL design in stock. I have now some boards with DIL – layout but no chips… So once again to the PC and redraw the design on PLCC socket. Thought – done and ordered a set of boards from the Far East producer.

A suitable case can be created with the 3D printer itself. To be more precise, I found what I was looking for on Thingiverse and was able to choose from a variety of suitable designs.

The overlay is missing, but the game is fun even without it. Malban has managed to create a great game here.

MIDI DB50XG – an interface for the daughter board

Loading

Rummaging through a box of my old crafts I found the box below. It dates from the time when I was still working with Amgia, but also with PCs – I guess around 1996. I labeled the box “DB50XG MIDI – Wavetable Processor”.

Das Fundstück aus der Kiste

Inside is a circuit board from Yamaha, which is called the DB50XG. This board was designed as a daughter board for PC sound cards with “Waveblaster” expansion port. She expanded the sound cards with a polyphonic MIDI wavetable sampler. In this way, the General Midi Standard and the Yamaha XG Standard could be re-established. Today nobody thinks about it anymore. At that time, if you wanted to generate sounds with a PC from midi data, then either external hardware was required, or a sound card with an onboard midi synthesizer or wavetable chipset. The PC then took over the control, the sending and receiving of the midi data via a sequencer software. Today, the midi sounds are generated directly on the PC and the samples and sound models are integrated into the software. At that time, the performance of the PC hardware was not sufficient. If someone is wondering what I’m palavering about here – what is Midi and why do you need it? – then let me put it briefly here: Midi is the abbreviation for “Musical Instrument Digital Interface” – i.e. a digital interface – a data protocol for musical instruments. Roughly explained, it serves to network and control electronic musical instruments with each other. For example, a large number of sound-generating devices can be controlled via a single keyboard. I will not explain here how the Midi standard works, what the data packets look like and how it looks electrically. As always, there is plenty of information on the web.

Inside the box

Back to the self-made box. At that time I packed the DB50XG in the plastic box and from the “Waveblaster” port, a 26-pin socket strip, led the necessary cables to the outside to start up the Midi board. And that was pretty simple. The board requires a power supply of +/-12V and +5V. There is a Midi-IN and a Midi-OUT (through) pin, a reset pin and two analog audio out pins – one per channel. The table below shows the connector pin assignment:

pin number assignment
1 Digital ground
2 not connected
3 Digital ground
4 not connected
5 Digital ground
6 Supply +5V
7 Digital ground
8 not connected
9 Digital ground
10 Supply +5V
11 Digital ground
12 not connected
13 not connected
14 Supply +5V
15 Analoge ground
16 not connected
17 Analogue ground
18 Supply + 12V
19 Analogue ground
20 Audio out richt
21 Analogue ground
22 Supply -12V
23 Analogue ground
24 Audio out left
25 Analogue ground
26 reset

The whole structure was rather spartan back then. The power supply had to be established via one or more external power supplies. There was no galvanic signal isolation using optocouplers. So I had to rely on the proper setup of the Midi IO controller that I connected to the Amiga. Of course it couldn’t stay like this. And I can’t bring myself not to use the beautiful DB50XG board anymore or to throw it away in the electronic waste. The plan that emerged from this was to develop a new interface board – or to tinker, which should be as universally usable as possible.

DB50XG

It’s been a few years since this idea and I’ve always worked on it a little bit. I thought the interface board should fulfill the following points:

  • a simple power supply should supply the Yamaha board with energy. Ideally, there should be a USB port and, optionally, a connection for a universal power supply. All required voltages should be generated on the interface board from the 5VDC.
  • As in the past, the DB50XG should also be able to be plugged in as a “piggyback” circuit board
  • The midi-in signal should be able to be fed in via the 5-pin DIN socket and also via a pin header – of course nicely decoupled (this means that a microcontroller such as Arduino and co. can also be connected without any effort)
  • The sound, i.e. the audio signal, should be available for acceptance via a chinch socket and also as a 3.5mm jack socket and via a pin header per channel.
  • Word repetitions SHOULD be avoided, but I don’t care 🙂

This ultimately resulted in the following circuit diagram. The 5VDC supply of the USB source is routed directly to the 5V supply of the midi board. The +12V/-12V that are also required are generated by a DC/DC converter (TMR0522). This is supplied on the input side by the 5V mains. The optional “external” voltage input goes to a LM2596ADJ. This is a step-down voltage regulator that can work with input voltages up to 40V. The regulated output side is available in many areas. I have integrated the ADJ (Adjustable) type into the circuit here, as I have a few of them in the assortment box. The voltage source can be selected with a jumper on the board.

Based on this circuit diagram, I created a layout and initially produced it in my own etching bath. The result was the following circuit board, which served as a test setup. Technically, the board worked perfectly, but I didn’t like the arrangement of the components. I placed the step-down converter and coil on the back. The distance between the connection sockets was also too close together for me. And how you do it as a PCB layouter – you always do a second design. So also it is this time.

The test setup with a fitted Midiboard can be seen in the image below. The midi signal as a test source comes from the PC and is generated by a USB midi adapter from the Far East.

So sat down in front of the computer again and redrawn the layout. The following version came out. I then ordered this version from a circuit board manufacturer.

The finally manufactured printed circuit board then looks like this. Below she can be seen with the DB50XG board attached.