Problems with a stalling wicd 1.7.0 when scripts are set for a connection

Yesterday I updated to wicd -1.7.0 and experienced issues when connecting to my local wired network. I found out that the issue only comes up when using wicd-gtk. Wicd-curses manages to get a connection.

So I searched the web and found out that this problem only occurs when having (dis)connection scripts set. The issue is already filed at wicd’s launchpad upstream with a working patch by Jonathan (Comment #17). As I use wicd every day, the fix for this issue is quite important to me.
Gentoo currently only has the 1.7.0 (1.6.2 worked flawlessly) version in its tree and so I made an ebuild which includes Jonathan’s patch, fixes the issue and can be found in gentoo’s bugzilla.

If you are experiencing the same issues in gentoo, check out the new ebuild and the patch.

Happy emerging
Regards,
Phillip

Server Optimization I – MySQL

My last post about server optimization dates February because the last weeks/months have been quite busy. I promised to continue with MySQL optimisation which I will be doing now. As with the other posts, I will only write down significant new information and otherwise link to the information in the web so that you can use this guide as a condensed view on the topic.
First of all visit documentation at mysql.com which contains a whole chapter about optimization. If you only want to
optimise the MySQL server
this subchapter will
suit your needs.

The MySQL Query Cache

The query cache, although not a panacea, can bring big performance improvements and is, at least with mysql 5.0, disabled by default. However you have to know that it only optimises queries. It cannot
“look” into your application and group all queries of the same request or similar.
The MySQL Performance Blog has a nice tutorial about configuration and background of the query cache.

Adapting MySQL Cache Sizes

MySQL has several caches which need to be adapted to your personal needs. If you are using PHPMyAdmin, it can give you hints about the cache sizes which need to be optimised in your current setup. Just open server_status.php which is linked on the start page as Show
MySQL runtime
information. To change values edit your /etc/mysql/my.cnf and restart MySQL afterwards.
If you want to know how to view this information directly via SQL command, look at this optimisation guide underneath the subheading Getting information about current values or into the MySQL documentation. Both guides also lists other possible improvements.

Identifying slow queries

Optimising your web application or developing it with database performance in mind, normally should be the first step to do. Though it may often be the case that you cannot access the code of your application, have not the required skills, not enough time or that you do not want to change the standard application for easier updates. Problems in this area can often be avoided or at least minimized by using an object relational mapper like hibernate for Java as it optimises the queries on object level before executing them.
If you cannot use an ORM mapper for some reasons or if you want to know which queries use most resources you can activate the MySQL Slow Query Log. Pete Freitag directly states which entries are needed in the my.cnf file.

Optimising your application

After you identified slow queries or even before generating them, this blog post for PHP experts can help you finding database related performance hits in PHP with PHP.

As a general measure I strongly recommend using indexes for frequently used or searched attributes. Although from 2001 the following guide explains these topics and their background greatly.

Optimising the compilation

If you compile mysql by yourself, you can also get speed improvements by compiling MySQL with special
options. The MySQL
documentation
lists several possibilities.

Optimising your Linux kernel for MySQL

Apart from MySQL
itself you can also optimise your kernel parameters (sysctl.conf) for
MySQL. If you want to, refer to this guide.

Other resources

If this still is not enough you will find plenty of other resources in this forum post. Some of the links listed there were already mentioned before.

Finally…

Before investing in new hardware be sure to check the configuration of your database and,  when you can, the database queries in your application. Even if the latter is not possible, simple server side tuning can bring huge improvements especially when the query cache is not activated or it or other caches are too small so that MySQL hast to write more data to disk than necessary.

I hope this small guide/list of links gave you an overview over MySQL performance tuning. For me it will serve as an aid to memory and thus it will be expanded if future issues arise.

Regards,
Phillip

Automatische Vertragsverlängerung bei O2 und kostenlose Zusatzoptionen wie z.B. Frei-SMS

Heute kam meine neue Handyrechnung. Merkwürdigerweise war diese fast doppelt so hoch wie sonst. Da ich meine Telefonminuten und SMS über eine Monitoringanwendung konstant überwache und so Inklusive-SMS und Inklusiveminuten eigentlich nicht überschreiten kann, war ich doch verdutzt bis schockiert.

Ein Blick auf den Einzelverbindungsnachweis offenbarte mir dann, dass alle SMS kostenpflichtig abgerechnet worden waren.
Dazu muss gesagt werden, dass mein Vertrag einen Monat vorher ausgelaufen war und ich ihn nicht manuell verlängert hatte, da ich mit den Konditionen so zufrieden bin. Ich ging natürlich davon aus, dass alle meine Vertragsoptionen verlängert werden, wenn ich nicht explizit etwas anderes vereinbare. Etwas Abweichendes hatte mir die Frau bei der Vertragsverlängerungs (VVL)-Hotline bei meiner Verlängerung vor 2 Jahren auch nicht mitgeteilt.

Also Anruf bei O2. Dort teilt man mir mit, dass kostenlose Zusatzoptionen eben nicht mit verlängert werden und erzählt mir, dass das bei Abschluss ja auch immer gesagt wird. Nein! Bei mir wurde das am Telefon bei der VVL damals eben nicht mitgeteilt. Dessen bin ich mir hundertprozentig sicher, eben weil es damals eine ziemliche Anstrengung war und ich mich so noch an viele Details der zahlreichen Gespräche erinnern kann.
Der Mann bei O2 sagt mir nach Rücksprache mit seinem Chef dann, dass sie mir einen Anteil der Kosten als Gutschrift erlassen können und dass mehr als dieser Anteil nicht drin sei. Also keine komplette Erstattung. Wenn ich den Vertrag verlängere (24 Monate), was ich auch gleich tue, kann ich natürlich die
Frei-SMS wieder haben.

Damit gebe ich mich erst mal zufrieden und recherchiere noch eine Weile im Internet und erinnere mich zurück an die VVL damals, um einen Fehler meinerseits auszuschließen. Ich finde mehrere Threads in diversen Foren, die sich mit dem Thema beschäftigen. Offensichtlich sind hier mehrere Leute etwas verunsichert. In den Threads in den Foren kann ich nur leider keine eindeutige Linie feststellen und die Tatsache, dass meine Vertragsinformationen im O2 Kundenportal nur sagen, dass ich einen O2-Tarif mit 100 Inklusivminuten habe und mich f├╝r alle Details an die VVL-Hotline verweisen, hilft mir dabei auch nicht weiter.
Zusammen mit der Gegebenheit, dass ich am Telefon bei der VVL damals definitiv nicht auf ein Auslaufen der Frei-SMS hingewiesen worden bin, frage ich mich langsam, wie ich denn hätte wissen sollen, dass die Frei-SMS auslaufen. Ich frage mich – inzwischen etwas gereizter – auch, warum ich für die mangelnde Informationspolitik von O2 Geld bezahlen soll.

Also rufe ich O2 erneut an, wo ich im Kundencenter zufälligerweise auf den gleichen verständnisvollen Mitarbeiter wie beim Vorgespräch treffe. Nach dem nun deutlich energischeren Darlegen der Argumente meinerseits, versichert mir dieser, sich erneut beim Chef für mich stark zu machen. Und siehe da! Nach kurzer Zeit in der Warteschleife werden alle meine Kosten übernommen. Zusammen mit diesem Blogpost hat mich das dann auch nur ca. 2,5h gekostet. Ist ja ein Klacks ;-).

Was hat mich das gelehrt, bzw. worauf ist bei kostenlosen Zusatzoptionen bei O2 zu achten?

  • kostenlose Zusatzoptionen wie Frei-SMS verlängern sich nicht bei einer automatischen VVL, nur gebuchte kostenpflichtige Optionen wie z.B. Internet-Packs werden dabei automatisch verlängert
  • Eine automatische VVL verlängert den Vertrag gewöhnlich um 12 Monate eine eigene Verlängerung für gewöhnlich um 24 Monate
  • VVL sind in einem Zeitraum von 5 Monaten bis kurz vor dem Ablauf (prinzipiell ein Tag vorher, besser 3 Tage) des alten Vertrages möglich. Kostenlose Zusatzoptionen müssen dabei durch den Kunden neuverhandelt werden.
  • Verträge müssen bis spätestens 3 Monate vor Ablauf des alten Vertrages gekündigt werden

Meiner Ansicht nach ist die Informationspolitik bei Vertragsverlängerungen bezüglich dieser Attribute bei O2 mangelhaft:

  • Im Kundencenter von O2 können kostenlose Zusatzoptionen (zumindest VVL ohne Handy) des aktuellen Vertrages nicht eingesehen werden.
  • Die Tatsache, dass kostenlose Zusatzoptionen sich nicht verlängern, kann nur in den aktuellen Tarifen bei O2 eingesehen werden. Ist ein über mehrere Jahre verlängerter Vertrag dort nicht mehr zu finden, schaut der Kunde in die Röhre und muss sich auf die Aussagen und Aufklärungen am Telefon verlassen.

Deswegen also immer manuell und persönlich den Vertrag in einem Zeitraum von fünf Monaten vor bis zum Vertragsende verlängern lassen, egal ob man mit den Konditionen zufrieden ist, oder nicht. Die automatische Verlängerung birgt leider zu viele Risiken.

Also schön die Call Center nerven
Grüße
Phillip

PS Die Angaben von Kündigungs- und Verlängerungsfristen sind nicht allgemein gültig, aber bei O2 normalerweise so üblich. Zur Sicherheit lieber nochmal nachfragen.

 

 

Dual head presentation of pdfs created with latex beamer class

I am using Latex for assignments, papers and presentations quite a long time. However, until recently, I did not know a solution for dual head presentation of pdfs created with the latex beamer class.
This changed with the finding of Impressive. So if you want to create a pdf with notes on the second screen you first have to set up your beamer class like this:

[code language=”latex”]\documentclass{beamer}

\usepackage[utf8x]{inputenc}
\usepackage{default}
\usepackage{pgfpages} %This is needed for notes presentation!
\setbeameroption{show notes on second screen}

\begin{document}
\begin{frame}
\frametitle{Note Test}
This is the frame text
\note[item]{Note for a itemized note list}
\note{Note for a continuous note text}
\end{frame}
\end{document}
[/code]

 

After that install impressives requirements and impressive itself. Then the following command will start the presentation on a beamer and notebook both with a 1024×768 resolution:

[code language=”bash”] ./impressive.py -g 2048×768 “pdfname.pdf”[/code]

Normally notes will be rendered on the right screen. So you have to set up the beamer as “left of” your notebook. With xrandr this can be accomplished like this or similar depending on your available outputs:

[code language=”bash”] xrandr –output VGA1 –left-of LVDS1 –output LVDS1 –mode 1024×768[/code]

Be sure to use the same aspect ratio and resolution on notebook and beamer. Otherwise you will get cropped slides and/or black borders.

Update: Someone pointed me to this useful article which achieved similar results by only using XrandR’s scaling and positioning parameters.

Happy presenting!
Phillip

Dual head presentation of pdfs created with latex beamer class

I am using Latex for assignments, papers and presentations quite a long time. However, until recently, I did not know a solution for dual head presentation of pdfs created with the latex beamer class.
This changed with the finding of Impressive. To create a pdf with notes on the second screen you first have to set up your beamer class like this:

[code language=”latex”]\documentclass{beamer}

\usepackage[utf8x]{inputenc}
\usepackage{default}
\usepackage{pgfpages} %This is needed for notes presentation!
\setbeameroption{show notes on second screen}

\begin{document}
\begin{frame}
\frametitle{Note Test}
This is the frame text
\note[item]{Note for a itemized note list}
\note{Note for a continuous note text}
\end{frame}
\end{document}
[/code]

 

After that you have to install impressives requirements and impressive itself. Then you can start impressive on a beamer and notebook both with a 1024×768 resolution like this:

[code language=”bash”] ./impressive.py -g 2048×768 “pdfname.pdf”[/code]

Be sure to use the same aspect ratio and resolution on notebook and beamer. Otherwise you will get cropped slides and/or black borders.

Happy presenting!
Phillip

Gentoo major version KDE upgrade HOWTO

This HOWTO will show you how to upgrade from kde 4.3 to kde 4.4 (and possibly others) with as few problems as possible (It went through without errors for me). The whole process including merging will take 4 to 10 hours depending on your PC. Plan accordingly.

Before upgrading check if you use the latest packages. Additionally I will assume you have autounmask, eix and gentoolkit installed. Portage 2.2 might also come in handy because it can automatically resume builds of many packages (–keep-going) when only few of them fail.

[code lang=”bash”]
emerge –sync
emerge -nva gentoolkit autounmask eix
eix-update
[/code]

If you want to use your computer while updating KDE it is pretty neat to have an alternative login and windowmanager installed. I prefer gdm and enlightenment.

[code lang=”bash”] emerge -nva eix gdm enlightenment
[/code]

 

Preparing the upgrade

[code lang=”bash”]
#create a working directory
mkdir /root/kde-upgrade

#save the list of your kde 4.3 packages
eix -I –slot 4.3 -C kde-base –only-names > /root/kde-upgrade/kde-old
#save the list of packages installed by which are also avalaible for kde 4.4
eix -I –slot 4.4 -C kde-base –only-names > /root/kde-upgrade/kde-new
#save the list of installed qt-packages
eix –slot 4 -C x11-libs –only-names -I qt > /root/kde-upgrade/qt-packages
#KDE 4.4 needs QT-4.6 that is why we create a list of all qt-packages and keyword them for later use. Be sure to change ~amd64 to your architecture
eix –slot 4 -C x11-libs –only-names -I qt | while read line; do echo “$line ~amd64” >> /root/kde-upgrade/qt-keywords; done
[/code]

Copy the qt-keywords to your package.keywords directory

[code lang=”bash”]
cp /root/kde-upgrade/qt-keywords /etc/portage/package.keywords/
[/code]

Download and install kde-4.4 keywords

[code lang=”bash”]
cd /root/kde-upgrade
wget -O kde-4.4-keywords “http://git.overlays.gentoo.org/gitweb/?p=proj/kde.git;a=blob_plain;f=Documentation/package.keywords/kde-4.4.keywords;hb=HEAD”
cp -p /root/kde-upgrade/kde-4.4-keywords /etc/portage/package.keywords/
[/code]

 

Prepare your alternative Desktop environment

Now we are starting with the essential stuff. Log out from kde and shutdown kdm from one the Ctrl-ALT-Fx-Terminals and start your favorite alternative login manager.

[code lang=”bash”]
/etc/init.d/xdm stop
#Start other login manager
gdm
[/code]

After that you can login again on Ctrl-ALT-F7.


Upgrade QT and remove old KDE

Update your qt installation now because KDE 4.4 requires qt 4.6.

[code lang=”bash”]
emerge -va `cat /root/kde-upgrade/qt-packages`
[/code]

After that we are ready to remove the old KDE. Old KDE version’s packages would be removed by an upgrade anyway. When we remove them before our installation we will save us a lot of hassle concerning package blocks etc.

I heard unmerging the old kde version is completely unnessessary now, as newer portage versions will handle softblocks by themselves. So it is completely ok to skip the unmerging.

[code lang=”bash”]
emerge -C `cat /root/kde-upgrade/kde-old`
[/code]

 

(While emerging qt) Check for keyworded packages

While QT ist emerging switch to another terminal and check if your following KDE merge has all its dependencies in place.

[code lang=”bash”]
#try to
emerge -vau `cat /root/kde-upgrade/kde-new`
[/code]

If you get an error about missing keywords on dependant packages, execute:

[code lang=”bash”]
autounmask package_category/package_name-package_version
[/code]

on the corresponding package (the package mentioned in “dependency required by”). Personally I had to autounmask phonon-kde, kopete, kdebase-runtime-meta, kaddressbook and libiodbc. Libiodbc-3.52.6 failed on ~amd64 so I installed the keyworded version which built normally.

 

(After emerging QT) Rebuild Qt dependant packages

Rebuild all qt-dependendant packages.

[code lang=”bash”] emerge -1va sip PyQt4 qscintilla qscintilla-python
[/code]

Apart from these packages you will have to rebuild all installed kde-themes in portage. After that remerge Phonon. I had to do this and for you it will not hurt ;-).

[code lang=”bash”]
#emerge your new kde’s phonon version
emerge -1va phonon
[/code]

 

Emerge your new KDE

So sit back and relax. The biggest part of the work is done now. Emerging new KDE packages will take several hours depending on your CPU power. It is best to do this over night. If you peek at the following command you will see a –keep-going at the end. This tells Portage-2.2 to automatically resume the whole merge with packages not depending on the failed package when a package fails. If you use Portage-2.1 remove this option and beg that you will wake up with a fresh and shiny new version of KDE.

[code lang=”bash”]
#DO!
emerge -va `cat /root/kde-upgrade/kde-new` –keep-going
[/code]

 

Final work

After our work before kde should build without errors. At least it did for me… If there remain errors visit http://forums.gentoo.org in your spare desktop environment.

[code lang=”bash”]
#revdep-rebuild to rebuild packages like e.g. digikam or similar
revdep-rebuild
#update your config files
etc-update
#read announcements generated during upgrade
eread
[/code]

If all is done, you can log-out from your temporary desktop environment,
kill your other login manager and start kdm again. Then log-in to your new shiny kde version and enjoy.

If you have suggestions or improvements let me know.
Hope this helps,
Phillip

Server Optimization

The last weeks I investigated server side optimization (not application optimization) issues further because our Joomla based web portal scales quite poorly. The server is still sufficient for our current load but I want to have some more resources for usage peaks and future development. I thought it would be nice to share my thoughts and experiences as appropriate tutorials are scattered all over the web. I will describe the main steps in brief and will link to further documentation at the web if possible and needed.

So far so good. Currently our application uses the following server software:

  • Joomla 1.5 as web application
  • PHP 5.2 with ZendOptimizer 3.3 as our only must have PHP extension
  • Apache httpd 2.2 as web server
  • Varnish 2 as reverse caching proxy for Java and PHP apps.
  • MySQL 5.0 as database management system
  • Gentoo Linux as server OS

Our website makes intensive use of Joomla extensions. Some of them were developed by our own IT department. The start page is news oriented and contains several extensions and also many images. Apart from that we have quite a few registered users which use interactive parts of the site like forums or lotteries. Additionally some content of the site is only available to privileged users.

With this information you should have a rough overview of our web site and the application comprehending my optimization efforts. The next part of the series will contain information about bringing MySQL to speed.

Comments are as always welcome
Phillip

Drupal WYSIWYG editor not showing up after installation

Ah. Silly me. I experimented with the drupal caching options just before installing the WYSIWYG module and wondered why the editor was not showing up.

The solution was as obvious as simple: I had to disable or clear Drupal’s JavaScript and CSS cache after installing the WYSIWYG plugin.

Hopefully you read this before long searching. Additional clues can be found in this drupal forum post ( http://drupal.org/node/426398 ).


 

Changing deprecated vpopmail paths to virtual mail accounts in domain.tld/vpasswd

I recently stumbled upon a problem of a big evolutionary grown qmail/vpopmail installation when we moved the installation to a new server.

Virtual accounts created years ago were created with a different vpopmail installation located in a different filesystem path. For this reason accounts had different paths set up in the /var/vpopmail/domains/domain.tld/vpasswd.cdb file. Regrettably there is (at least afaik) no possibility to change these paths with vpopmail commands like vmoduser or similar.

To avoid always creating symbolic links from the old directories to the new account directories I found a way to modify/update these paths. Normally values in the plain text vpasswd (not the .cdb one mentioned before) file get overwritten every time you update/delete or create accounts with the values from the vpasswd.cdb file. However if only the vpasswd file and not its .cdb counterpart is present the vpasswd.cdb file will be regenerated with the values from the vpasswd file.

So if you want to change paths, do the following:


#copy old vpasswd to a new location
cp vpasswd vpasswd.new
# edit vpasswd.new to suit your needs
vim vpasswd.new
# shutdown your mailserver services for some seconds to prevent regeneration of the vpasswd.cdb file before you are ready (check if you may do this before!)
/etc/init.d/svscan stop
cp vpasswd.new vpasswd
#remove cdb file
rm vpasswd.cdb
# create a testuser for the appropriate domain, to regenerate cdb file
vadduser testuser@domain.tld
# delete testuser
vdeluser testuser@domain.tld
# start mailserver services again
/etc/init.d/svscan start

Normally a shutdown time of a few seconds should not be critical for a mailserver, because sending servers normally have queue lifetimes much longer than this and sending users can wait a few seconds anyway, eh ;-)?

You should however check this first!

If you put the lines above in a appropriately changed script downtime can be less than 5 seconds.

Hope this helps,
Regards,
Phillip

Notebook endlich zur├╝ck

Wegen eines unfallbedingten Schadens musste ich mein Notebook am 12.11.2009 zum Lenovo Support/Geodis Gmbh in Heppenheim einschicken. Da der Schaden unfallbedingt war, fiel er nicht unter die Garantie. Auf Nachfrage nannte mir der Lenovo Mitarbeiter eine ungefähre Bearbeitungsdauer von 4-7 Werktagen.
Leider kam das Notebook erst am 1.12.2009 wieder bei mir an, was einer Bearbeitungsdauer (bei Eingang am 13.11. und Ausgang am 30.11.) von 12 Werktagen entspricht. Also fast doppelt so viel wie angek├╝ndigt.
Bisher war mir der Support auch aus eigener Erfahrung nur als tadellos bekannt. Ich hoffe mal, dass sich Derartiges nicht wiederholt. Als Informatiker in seinem Studium auf sein Notebook verzichten zu müssen, macht einen nämlich nicht unbedingt produktiver :-).

Grüße
Phillip