TLS Renegotiation Remediation is Going to be Unpleasant

I was contacted by my colleagues in network engineering late last week to investigate what could possibly be causing a production issue that had started earlier that same week. An in-house client application making soap calls over an ssl connection periodically for data provided by a third party financial services firm began failing and they suspected a change somewhere in the network being the cause since "nothing changed in the application". The very same app continued operating just fine going to the exact same web service in an in-house test environment. Access to the service at the financial services firm was over ssl/tls-only and restricted by client certificate.

Below is a sample of the log for this web services client application:

16-Jul-2010 14:54:28 org.apache.commons.httpclient.HttpMethodDirector authenticateHost
WARNING: Required credentials not available for BASIC @redacted.host.tld:443

16-Jul-2010 14:54:28 org.apache.commons.httpclient.HttpMethodDirector authenticateHost
WARNING: Preemptive authentication requested but no default credentials available

16-Jul-2010 14:54:28 org.apache.commons.httpclient.HttpMethodBase readResponseBody
INFO: Response content length is not known
javax.net.ssl.SSLException: HelloRequest followed by an unexpected  handshake message

Later, the systems administrator informed me that they were able to workaround the issue by setting a system property at initialization of the jvm:

 Dsun.security.ssl.allowUnsafeRenegotiation=true

Ultimately, what this turned out being was an update of the JVM on the production boxes from whatever version was there previously to version 1.6.0_20. As we all know, java updates for the most part are cumulative and update 19 added what Oracle calls an "interim fix" for the TLS renegotiation vulnerability. I won't go into detail about the TLS Renegotiation vulnerability here—there is an excellent write up about it on the Educated Guesswork blog that I'm linking to, if you want to know more.

In a nutshell, version 1.6.0_19 disables TLS Renegotation by default. I think in the case of an ssl or tls connection that requires authentication by client certificate, the initial connection is made to the server by the client, then the server responds with a 403 looking for a client certificate. The client then renegotiates the session presenting the client certificate. The change to the jvm behavior due to the TLS Renegotiation vulnerability breaks this. In the case of a java application using the apache httpclient, it fails with a javax.net.ssl.SSLException: HelloRequest followed by an unexpected  handshake message exception.

Unfortunately, many vendors are utilizing this type of workaround while waiting for the TLS protocol to be fixed. As in Oracle's case, vendors, like Microsoft, are providing "patches" that simply disable auto renegotiation by default and letting support staff discover on their own when applications are broken. So, in case anyone finds this doing a search on the web, and because many vendors are not really providing examples of the types of applications and connections that _could_ fail should someone implement one of these patches, below is my draft list detailing the types of TLS/SSL connections that could fail. I have not personally tested or verified this list as it is an attempt to brainstorm potential issues so if anyone has any experience with them and knows that a particular item is incorrect, let me know and I'll update this post.

• SSL or TLS connections to servers that require client certificates.
• In apache specifically, if you upgrade encryption ciphers for specific sections of your site using Location directives, you could see problems.For example, if you specify Medium grade encryption as acceptable for most of the site and upgrade the cipher to a High grade one for login pages or Order Completion pages, this forces a renegotiation of the connection, which could break.
• FTPS (also known as FTP-SSL or simply FTP over SSL) connections in Explicit Mode.
• Secured LDAP connections where a STARTTLS operation is invoked and the existing unencrypted connection is upgraded to a secured connection.
• IMAP, POP3, NNTP, and SMTP connections that step up to a secured connection. These also invoke a STARTTLS operation to renegotiate the existing connection using TLS.
• I don't know of anything supporting it yet but connections supporting RFC 2817 - Upgrading to TLS Within HTTP/1.1 could potentially be impacted as well.