I have discovered the issue with TortoiseSVN and a failing SSL handshake. After fiddling around for quite a long time, I found out, that the issue with the SSL handshake is not related to the SVN server but to the TortoiseSVN client version or in particular its linkage against different versions of Neon.
TortoiseSVN 1.6.11 links against Neon 0.29.4 whereas the working TortoiseSVN 1.6.10 links against Neon 0.29.3. The current development version of the 1.6. branch of TortoiseSVN also links against Neon 0.29.3 and works, too. Together with the fact that Neon announces “Fix GnuTLS handshakes failures with ‘TLS warning alert’ (Bryan Cain)” as a release note for Neon 0.29.5, I strongly suspect the linkage against the 0.29.4 version of Neon as the culprit for all the errors.
For all people using TortoiseSVN and experiencing these errors this means either to downgrade to TortoiseSVN 1.6.10 or to use the latest development version of the 1.6 branch until the issue is resolved.
Update: Bug is taken care of. See http://svn.haxx.se/tsvnusers/archive-2010-10/0340.shtml .