The final Beowulf cluster will have 20 of this nodes with 5×20 Raspberry PI/Asus Tinker.
Currently we have 20 Raspberry PI 3B running at 1400MHz.
One of our next tasks will be the evalution of the Asus Tinker devices as a replacment option for our Raspberry PI 3B.
One part of doing Apache-Tuning is to play around with the MPM configuration:
<ifmodule prefork.c> StartServers 4 MinSpareServers 3 MaxSpareServers 10 ServerLimit 256 MaxClients 256 MaxRequestsPerChild 10000 </ifmodule>
The particular configuration setting for Apache can be found in the file /etc/apache/httpd.conf
I’m using Webmin->Sysstats to get some basic data where I can start from.
First I wanted to know the average and total memory usage of the apache worker process(es).
As you can see after a few days you get a clew about how our worker memory consumption behaves.
Especially the per worker chart is import which gives you a hint for the MaxClients and ServerLimit settings.
The used bash script :
-getApachePerWorkerMem)
ps -ylC apache2 | awk '{x += $8;y += 1} END {print x/((y-1)*1024)}';;
-getApacheTotalMemPercent)
sys_mem=`free -m | awk 'NR==2{print $2 }'`
apache_total_mem=`ps -ylC apache2 | awk '{x += $8;y += 1} END {print x/1024;}'`
echo "scale=2;$apache_total_mem/($sys_mem/100)" | bc;;
Another important part is the relation of idle and busy workers.
Here we see that in this particular case we have an average ratio 10/2 (idle/busy) and our max is about (11/7) which gives a balanced amount of idle and busy workers.
The used bash script :
-getApacheBusyWorker) echo `wget -q -O - "$URL" --user="$USER" --password=$PASS |grep BusyWorker |cut -d" " -f2`;; -getApacheIdleWorker) echo `wget -q -O - "$URL" --user="$USER" --password=$PASS |grep IdleWorker |cut -d" " -f2`;;
have fun
Mario
For the last couple of years we have been using an Intel SS4400-E storage device for our internal backup stuff.
Suddenly the main RAID CIFS share stopped working. It seems for some reason nobody could mount this share anymore. All user got an access denied on mount attempts.
So what the hell was going on: The rather simple web interface wasn’t the burner, in fact the system log does not deserve the name Log 🙂
So mostly every RAID in this class is based on an embedded system driven by our friend Linux. This one is no exception.
So we first need the most important part,…. a working telnet or SSH connection. But a first “test” connection to the SS4400 device gave us “connection refused” as expected.
Luckily after some googling I found a hind that on this devices you can enable the SSH interface via a hidden CGI script.
http://hostname/ssh_controlF.cgi
Now you can login via “root” and your administration password.
By the way, this device comes with the Swiss Army Knife of Embedded Linux BUSYBOX which I also use for my embedded projects. I’m working with Busybox sense 2004 (version 0.94x), ….. great stuff :-).
But it seems, INTEL has some GPL violation with these devices. lol, ..On our bundled CDs we didn’t get any GPL sources from Busybox and Co? This is only mentioned in passing.
In our case a look into the system logs indicates that our RAID share had some file system problems. INTEL uses XFS as the file system on the storage device. Our system log was filled up with error messages of the file access layer.
In our case the solution was really simple. On device startup the file-system reports that there are errors, but then the corrupt partition is mounted and then ,… kabudl……
To fix this you first have to detach the corresponding RAID with “NASdetach”. Now you can unmount the partition and run the XFS repair binary “xfs_repair“.
After 10 – 15 min the repair should be finished.
Reboot the storage device and you should be able to access your share.
have fun
Mario
httpServer.cpp: In member function "std::string HttpJailServer::receive(int, e_t, size_t)": httpServer.cpp:145:36: error: "read"was not declared in this scope httpServer.cpp: In member function "void HttpJailServer::sendRaw(const char*,ze_t)": httpServer.cpp:216:49: error: "write" was not declared in this scope httpServer.cpp: In member function "void HttpJailServer::send(int, const stri, const string&)": httpServer.cpp:250:15: error: "close" was not declared in this scope
on building vpl_xmlrpc_jail-1.2 for Moodle VPL module, there seems to be a missing include for these methods.
Just add,
#include <unistd.h>
to the httpServer.cpp, these should fix it.
have fun
Mario
Vsock, VMXnet are currently broken. So I patched this modules and build a new installer package. It is running now with out problems a couple of weeks.
Download -> vmware-tools.ubuntu-10.10-2.6.35.tar.bz2
have fun
After upgrading from debian squeeze I run into the problem that every commit was failing with “Couldn’t perform atomic initialization“.
See Debian bug report
My apache logs are showing :
[error] [client ] Couldn’t perform atomic initialization [500, #200029]
[error] [client ] Couldn’t perform atomic initialization [500, #200029]
[error] [client ] SQLite compiled for 3.7.4, but running with 3.7.3 [500, #200030]
So it looks like that libapache2-svn is build against libsqlite3 3.7.4 which is currently in Sid. In Squezze we have libsqlite3 3.7.3
A quick fix would be to take the libsqlite3 package from Sid for now.
have fun
I forgot the OS is a Ubuntu Lucid LTS, just for info.
A few days ago colleagues told me that the have problems to read/execute files via normal HTTP access which they have uploaded before. A quick look via SSH turned out that the newly created files had all 0600 rights. Because of that the access user/group schema is XYuser/www-data the web server couldn’t access the files anymore.
Hmm, shit, this shouldn’t happen here. So as a brave Linux user, i added umask 022 to the apache environment variable. But,.. what the hell, the files are always created with 0600. ….Oook this will not be a 5min job.
After an hour digging around with google it turns out that this is bug (link) in the recent WebSVN module used by the Lucid distro.
To fix this i downloaded from Debian testing the apache2.2-bin deflated it and copied the corresponding DAV modules to my server. … FIXED 🙂
Remember that taking packages from Debian or another Ubuntu distro is only a temporary solution! 🙂
Have fun
For my VM Linux Severs I’m using following kernel module blacklist:
blacklist mii blacklist serio_raw blacklist pcspkr blacklist psmouse blacklist lp blacklist parport blacklist snd blacklist soundcore blacklist snd_via82xx blacklist gameport blacklist snd_ac97_codec blacklist parport_pc
Don’t forget to call update-initramfs -u after altering the blacklist.conf !
Do only blacklist the mii module if you have the vmxnet module up and running !!
have fun 🙂
I noticed that on our virtual environments (VMware ESX) based on GNU/Linux (mostly Debian or Ubuntu distros ) the vmxnet modul is not used or not correctly loaded. Because of lsmod shows also a loaded pcnet32 module which uses mii.
I added follwing script snippet to the /etc/init.d/open-vm-tools
log_progress_msg "vmhgfs"; modprobe vmhgfs log_progress_msg "vmsync"; modprobe vmsync log_progress_msg "vmxnet - stoping network"; /etc/init.d/networking stop log_end_msg 0 log_progress_msg "vmxnet - removing pcnet32"; rmmod pcnet32 log_end_msg 0 log_progress_msg "vmxnet - removing vmxnet"; rmmod vmxnet log_end_msg 0 log_progress_msg "vmxnet - loading vmxnet "; modprobe vmxnet log_end_msg 0 log_progress_msg "vmxnet - starting network";/etc/init.d/networking start log_end_msg 0
Check the dmesg out for:
[ 42.273810] vmxnet 0000:00:11.0: PCI INT A disabled [ 51.808321] VMware vmxnet virtual NIC driver [ 51.808422] vmxnet 0000:00:11.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18 [ 51.815601] Found vmxnet/PCI at 0x1424, irq 18. [ 51.816793] features: ipCsum zeroCopy partialHeaderCopy [ 51.820787] numRxBuffers = 100, numRxBuffers2 = 1 [ 62.762575] eth0: no IPv6 routers present
This addon to the startup script removes the pcnet32 and vmxnet32 modules and loads explicit vmxnet. 🙂
Take care if you get an symbol not found error while loading the vmghfs module. If so you have to uncomment the loading of this module.
Have fun.
For months I was searching for a system capable to track and store out internal documents. I tested several systems like alfresco, opendocman,…..
In the end I decided to give OpenKM a try.
OpenKM runs on a JBoss application server, so you are bound to Java. They use Jackrabbit for their hierarchical content storage. The frontend is realized with GWT (Google Web Tollkit).
For a full feature list click here.
I installed OpenKM on a recent Debian 6.0
First you need to install OpenOffice. Make sure you install …-math, …-writer, … -draw, …-calc packages.
OpenOffice hast to bee run in headless mode. I use following startup script:
#!/bin/sh
### BEGIN INIT INFO
# Provides: OpenOffice Headless
# Required-Start:
# Required-Stop:
# Default-Start: 2 3 5
# Default-Stop:
# Short-Description: OpenOffice Headless
### END INIT INFO
PIDFILE=/var/run/openoffice-server.pid
case "$1" in
'start')
if [ -f $PIDFILE ]; then
echo "OpenOffice headless server has already started."
sleep 5
exit
fi
echo "Starting OpenOffice headless server"
soffice "-accept=socket,host=localhost,port=8100;urp;StarOffice.ServiceManager" -nologo -headless -nofirststartwizard & > /dev/null 2>&1
touch $PIDFILE
;;
'stop')
if [ -f $PIDFILE ]; then
echo "Stopping OpenOffice headless server."
killall -9 soffice && killall -9 soffice.bin
rm -f $PIDFILE
exit
fi
echo "Openoffice headless server is not running."
exit
;;
*)
echo "Usage: $0 { start | stop }"
;;
esac
exit 0
To get the preview stuff working you need also pdf2swf. Check your /bin directory there should be a pdf2swf executable or if you prefer you could take the swftool package from ubuntu.
The last step to get the previewer feature running is to set the corresponding config. flags at the OpenKm.cfg file.
Mail notification is straight forward. Go to /opt/OpenKM-4.1_JBoss-4.2.3.GA/server/default/deploy/mail-service.xml . Edit the section to your needs.
If you face the problem that the links in your notification mails are pointing to localhost, you can alter the application.url flag in your OpenKM.cfg.
GoTo /opt/OpenKM-4.1_JBoss-4.2.3.GA/server/default/deploy/jboss-web.deployer/server.xml and edit
Change the port field to 80.
One last thing needs to be done :-). Security! Holy Moly… what… yep.. Jboss runs out of the box like an open barn door. So a good idea would be to secure the jmx-console and web-console stuff. Because of that the jmx-console and web-console are standard servlets you can protect them very easily via enabling the security-constraint.
1. Edit \server\default\deploy\jmx-console.war\WEB-INF\web.xml and uncomment the security-constraint section:
<!-- A security constraint that restricts access to the HTML JMX console to users with the role JBossAdmin. Edit the roles to what you want and uncomment the WEB-INF/jboss-web.xml/security-domain element to enable secured access to the HTML JMX console.--> <security-constraint> <web-resource-collection> <web-resource-name>HtmlAdaptor</web-resource-name> <description>An example security config that only allows users with the role JBossAdmin to access the HTML JMX console web application </description> <url-pattern>/*</url-pattern> <http-method>GET</http-method> <http-method>POST</http-method> </web-resource-collection> <auth-constraint> <role-name>JBossAdmin</role-name> </auth-constraint> </security-constraint> <login-config> <auth-method>BASIC</auth-method> <realm-name>JBoss JMX Console</realm-name> </login-config> <security-role> <role-name>JBossAdmin</role-name> </security-role> </web-app>
2. Edit \server\default\deploy\jmx-console.war\WEB-INF\jboss-web.xml. Uncomment the following section:
<jboss-web> <!-- Uncomment the security-domain to enable security. You will need to edit the htmladaptor login configuration to setup the login modules used to authentication users.--> <security-domain>java:/jaas/jmx-console</security-domain> </jboss-web>
3. Edit \server\default\conf\props\jmx-console-roles.properties file
4. Edit \server\default\conf\props\jmx-console-users.properties file
5. Edit \server\default\conf\login-config.xml edit following section
props/web-console-users.properties props/web-console-roles.properties
6. under \server\default\conf\props\ copy and rename the two jmx-console-xx to web-console-xx.
If you want to start your OpenKM automatically at boot time you can use this init script:
#!/bin/sh
# /etc/init.d/jbossokm: Start and stop JBoss Application Service
### BEGIN INIT INFO
# Provides: OpenKM
# Required-Start:
# Required-Stop:
# Default-Start: 2 3 5
# Default-Stop:
# Short-Description: OpenKM
### END INIT INFO
ECHO=/bin/echo
TEST=/usr/bin/test
JBOSS_START_SCRIPT=/opt/OpenKM-4.1_JBoss-4.2.3.GA/bin/run.sh
JBOSS_STOP_SCRIPT=/opt/OpenKM-4.1_JBoss-4.2.3.GA/bin/shutdown.sh
$TEST -x $JBOSS_START_SCRIPT || exit 0
$TEST -x $JBOSS_STOP_SCRIPT || exit 0
start() (
$ECHO "Starting JBoss OKM."
rm -rf /path/to/server/default/tmp
rm -rf /path/to/server/default/work
rm -rf /path/to/server/default/log
su -l -c "$JBOSS_START_SCRIPT -b 0.0.0.0 > /dev/null 2> /dev/null &"
$ECHO "Done."
)
stop () (
$ECHO "Stopping JBoss OKM. "
su -l -c "$JBOSS_STOP_SCRIPT -S > /dev/null &"
sleep 10
$ECHO "Done."
)
case "$1" in
start )
start
;;
stop )
stop
;;
restart )
stop
sleep 30
start
;;
* )
$ECHO "Usage: jbossokm {start|stop|restart}"
exit 1
esac
exit 0
If you get an exception at shutdown or your shutdown script does not kill your JBoss then you could check sysctl-setting under /etc/sysctl.d/bindv6only.conf.
Make sure that the net.ipv6.bindv6only=0. !
If you would like to show your appreciation consider to support the guys from OpenKM with a donation 🙂
See PayPal subscription link at the bottom of the page
A few days ago a colleague asked me if I could install a Newsletter System for him on one of our servers. He decided to use OpenEMM 6.1.
OpenEMM is a feature-rich enterprise software for e-mail marketing, newsletters and service mails (transaction mails and event or time triggered mails). OpenEMM offers sophisticated bounce management, link tracking, lots of realtime statistics, a CMS module and a scripting feature to implement individual tasks……. cool 🙂
Installation goes by hand, cause sadly there are no deps.
However – after a bit of googling I found a wiki which descriptives the installation process for the common linux distros.
As a brave installation instruction follower I went through the installation step by step.
In the end there was only one task left to start the application server resin and login in for the first time.
And the fun starts 🙂
Instead of a shiny login screen I got….
javax.servlet.ServletException: org.springframework.jdbc.UncategorizedSQLException: Hibernate operation: Cannot open connection; uncategorized SQLException for SQL [???]; SQL state [null]; error code [0]; Cannot create PoolableConnectionFactory (Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.); nested exception is org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.)
hmmm, so what is jdbc trying us to say?
So I checked the stack trace:
Caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333) at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195) at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366) at java.net.Socket.connect(Socket.java:529) at java.net.Socket.connect(Socket.java:478) at java.net.Socket.(Socket.java:375) at java.net.Socket.(Socket.java:218) at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:256) at com.mysql.jdbc.MysqlIO.(MysqlIO.java:293) ... 57 more
“Connection refused” what the hell…. ok, database credentials are ok, openemm and openemm_cms schema are created. The MySQL error log says nothing? I’m able to connect to the schemas via mysql client ……. oh mannn, it turned out that I’m not the only one with this problem, lol, but all forum posts I could find so far offered no solution for my problem.
Then after half a day more or less searching for a solution I found a post where someone is talking about to check the MySQL “bind-address” parameter. Checking my.cnf turns out that the address binding was set to the external address of the server, lol what…, who changed that???
Quickly changed the address-binding back to “localh0st” restarted OpenEMM and requested the logging screen. Unsurprisingly the HTML error stays the same but,…. the stack trace changed…yeahhhh.
Now the trace shows:
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost. at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2497) at com.mysql.jdbc.MysqlIO.readPacket(MysqlIO.java:599) ... 72 more
Humpf,…… firing on google. I interpreted this message that jdbc could successfully establish a connection to the database and than for some reason the connection was closed from the db side.
A few hours later checking a lot posts concerning this problem I found a small side node in a post that it make sense to check the host.allow entrys.
Added ….
mysqld : localhost : allow
and ….. it works yeahhhhhh 🙂
Just installed KDE 4.4 from experimental and surprise, surprise it is working out of the box.
I didn’t have to do anything else than to alter my Synaptics configuration to favor packages from the experimantal branch.