Intermittent SSLHandshakeException problem. Please help, I'm stumped!

[Jeremy Schichor]

My organization recently switched from a home-brewed authentication system to CAS, and while it works reasonable well most of the time, I've been receiving reports that some customers are getting validation exceptions. These exceptions don't happen to everyone, and I have as yet been unable to reproduce them.

The error in the CAS server's log is "javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake" while the error on the client side is "rg.jasig.cas.client.validation.TicketValidationException: CAS Server could not validate ticket."

The error seems to happening most frequently when a user who has logged into our portal server clicks a link that takes them to our LMS server. Neither the portal nor LMS servers require SSL, but the CAS server does and has a cert from GoDaddy.

I am fairly new to CAS and am completely stumped by this issue--I've always had a great deal of trouble addressing issues that I can't personally reproduce. Can anyone help me?

Thanks!

[Marvin S. Addison]

Post by Jeremy Schichor
The error in the CAS server's log is
"javax.net.ssl.SSLHandshakeException: Remote host closed connection
during handshake" while the error on the client side is
"rg.jasig.cas.client.validation.TicketValidationException: CAS Server
could not validate ticket."

That's actually a fairly specific error and it's exactly what it sounds
like. The remote host (the CAS client application) sent a RST packet to
the CAS server on the TCP socket used for ticket validation and the CAS
server is responding in kind by tearing down its side of the connection.
It's apparently happening during the SSL handshake, which is why it's
presenting as a SSL handshake problem.

Unfortunately, determining the root cause of a problem like this is
fairly difficult.

Post by Jeremy Schichor
The error seems to happening most frequently when a user who has
logged into our portal server clicks a link that takes them to our
LMS server. Neither the portal nor LMS servers require SSL, but the
CAS server does and has a cert from GoDaddy.

I'm doubtful that the server CA (GoDaddy) has anything to do with this.
My hunch is that there's some networking problem between the client and
server, though we can't rule out a SSL or crypto issue at this point.
I'd like to know more about your system architecture between the LMS
server and CAS. What platform (Java, PHP) is the client and what OS and
Web server platform is it running on? What is the server setup of the
CAS server? Load balancers or reverse proxies between?

M

[Jeremy Schichor]

Thank you for your reply!

The clients (portal and LMS) are written in Java and hosted on Resin 3.0.18 by machines running Red Hat Enterprise Linux 5.5. Neither have reverse proxies, and both have fail-over load balancing on inbound traffic only.

The CAS server is hosted on Jetty 6.1.26 by a machine also running RHEL 5.5, with neither reverse proxies nor load balancing.

I hope this provides some clue as to what might be causing this issue. The customer service folks are breathing down my neck and I don't know what to tell them!

Thanks again.

[Marvin S. Addison]

Post by Jeremy Schichor
The clients (portal and LMS) are written in Java and hosted on Resin
3.0.18 by machines running Red Hat Enterprise Linux 5.5. Neither have
reverse proxies, and both have fail-over load balancing on inbound
traffic only.

I want to clarify your comment about "inbound traffic." So outbound
connections from apps running in Resin do _not_ traverse the load
balancer, is that correct? I'm trying to rule out involvement of
additional network devices in the HTTPS connection from CAS client to
server where ticket validation is performed.

Are you using the Jasig Java CAS client for CAS integration? If so, what
version?

Since this is RHEL, are you running IBM JRE? (I recall IBM is the
default JRE, or at least it was in the recent past.)

M

[Jeremy Schichor]

I got some clarification from the network guy--all traffic for the portal, LMS and CAS servers pass through the load balancer, but only the inbound traffic to the portal and LMS are load balanced. Outbound traffic from the portal and LMS, and all traffic to and from the CAS server, treat the load balancer as a switch.

All of the machines are using the Sun JRE, version 1.6.0_16. We are using version 3.2.1 of the Java CAS client, and version 3.4.11 of the Java CAS server.

BREAKING NEWS: After much monkeying around with our setup, it turns out that setting the SLB to switch-only mode for the physical LMS server not only seems to have eliminated this issue, but also increased the speed of page loads from the LMS dramatically. Apparently our load balancers are something like ten years old, and have evidently not aged gracefully. So we've eliminated the problem and then some, but are currently running without a fail-over for the LMS.

Thanks for your interest and help!