Forcing TLSv1.2 breaks IBM Connections Surveys and Textbox.io

I had to force TLSv1.2 across all of Connections to fix a problem with RTE as I detailed in Rich Content widget widget stops working due to mix matched SSL protocols but after testing I’ve found that this breaks Textbox.io in Chrome and Surveys.

The process is well documented in How to Force IBM Connections 5.5 CR1 to Use TLSv1.2 but after making the changes the following happens.

Textbox.io

In IE and FF Textbox.io works fine but in Chrome the spell check service fails.

In Fiddler trace is saw Spelling server error:  Could not load url “https://connections.acme.com/ephox-spelling/1/correction”: 500 Internal Server Error

In the SystemOut.log I saw

[6/29/16 10:00:38:507 BST] 00000200 SystemOut     O ironbark-akka.actor.default-dispatcher-17, RECV TLSv1 ALERT:  fatal, handshake_failure
[6/29/16 10:00:38:507 BST] 00000200 SystemOut     O ironbark-akka.actor.default-dispatcher-17, fatal: engine already closed.  Rethrowing javax.net.ssl.SSLException: Received fatal alert: handshake_failure

spray.can.Http$ConnectionException: Aborted
    at spray.can.client.HttpHostConnectionSlot.reportDisconnection(HttpHostConnectionSlot.scala:228) ~[spray-can_2.11-1.3.3.jar:na]
    at spray.can.client.HttpHostConnectionSlot$$anonfun$connected$1.applyOrElse(HttpHostConnectionSlot.scala:161) ~[spray-can_2.11-1.3.3.jar:na]
    at akka.actor.Actor$class.aroundReceive(Actor.scala:465) ~[akka-actor_2.11-2.3.9.jar:na]

IBM asked me to put on SSL trace, *=info:SSL=all. It seems that the client is sending TLSv1.0 which of course is not allowed now TLSv1.2 has been forced.

[7/11/16 9:31:17:286 BST] 00000115 SystemOut     O   ironbark-akka.actor.default-dispatcher-7, READ: TLSv1.2 Alert, length = 2
[7/11/16 9:31:17:286 BST] 00000115 SystemOut     O   ironbark-akka.actor.default-dispatcher-7, RECV TLSv1 ALERT:  fatal, handshake_failure
[7/11/16 9:31:17:286 BST] 00000115 SystemOut     O   ironbark-akka.actor.default-dispatcher-7, fatal: engine already closed.  Rethrowing javax.net.ssl.SSLException: Received fatal alert: handshake_failure

IBM have logged a ticket with Ephox as well as investigating it from there end.

Surveys

When in a community with previous surveys I can not see any of the historical surveys nor could I create new ones.

In the SystemOut.log I saw the following

[6/29/16 10:01:56:542 BST] 0000033b StandardExcep E com.ibm.form.nitro.platform.StandardExceptionMapper toResponse eaa8e54e-7c38-4edb-a5ca-bcbd6d7f6c64
                                 com.ibm.form.platform.service.framework.exception.ServicesPlatformException: com.ibm.connections.directory.services.exception.DSException: com.ibm.connections.directory.services.exception.DSOutOfServiceException: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

Caused by: com.ibm.connections.directory.services.exception.DSException: com.ibm.connections.directory.services.exception.DSOutOfServiceException: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

Again, it seems like it’s sending TLSv1.0.

There’s at least one other person I know of who’s logged a PMR for these problems. It’s fairly urgent due to a problem with the RTE application which is only fixed when TLS1.2 is enforced. I’m hoping that these problems can be resolved sharpish so I can resolve the RTE problem for a customer.

Rich Content widget widget stops working due to mix matched SSL protocols

For an IBM Connections 5.5 customer I found that periodically the Rich Content widget (RTE) stops working showing “The page could not be created due to an error” like the image below.

The following exceptions are seen in the SystemOut.log and access.log.

[6/21/16 10:10:34:889 CEST] 00000282 ConnectContro W org.springframework.social.connect.web.ConnectController oauth2Callback Exception while handling OAuth2 callback (I/O error on POST request for “https://connections.acme.com/oauth2/endpoint/connectionsProvider/token”:Server returned wrong cipher suite for session; nested exception is javax.net.ssl.SSLProtocolException: Server returned wrong cipher suite for session). Redirecting to IBMConnections connection status page.
[6/21/16 10:10:34:904 CEST] 00001811 ServletWrappe E com.ibm.ws.webcontainer.servlet.ServletWrapper service SRVE0014E: Uncaught service() exception root cause rteServlet: org.springframework.web.util.NestedServletException: Request processing failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name ‘scopedTarget.ibmConnections’ defined in com.ibm.lconn.rte.config.SocialConfig: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.social.connections.api.IBMConnections]: Factory method ‘ibmConnections’ threw exception; nested exception is java.lang.IndexOutOfBoundsException
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:982)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:575)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)

46.58.58.130 – – [21/Jun/2016:10:10:34 +0200] “GET /oauth2/endpoint/connectionsProvider/authorize?client_id=conn-rte&response_type=code&acls=true&callback_uri=https%253A%252F%252Fconnections.acme.com%252Fconnections%252Frte%252Fcommunity%252F8e3190fe-fb3e-4ebd-83d4-1f319e7a5522%252Fentry&_oauth_client_auto_authorize=true&scope=Connections HTTP/1.1” 302 –
46.58.104.37 – – [21/Jun/2016:10:10:34 +0200] “GET /dm/atom/people/feed?self=true&format=json&anonymousCheck=true&membershipTimestamp=1461057779000&dojo.preventCache=1466496635731 HTTP/1.1” 200 404
46.58.58.130 – – [21/Jun/2016:10:10:34 +0200] “GET /connections/rte/connect?code=iK97VSiDgGJzorfylyBpEVt3OJ800r HTTP/1.1” 202 104
46.58.104.37 – – [21/Jun/2016:10:10:34 +0200] “GET /connections/rte/community/8e3190fe-fb3e-4ebd-83d4-1f319e7a5522/entry?acls=true HTTP/1.1” 500 615

There was an interesting thread in the Socially Integrated blog which was along the same lines but not a match so I raised a PMR. The ever dependable Dave McCarthy and Kevin Holohan shed some light on what was happening.

“According to a chat I had with L3, this is a bug in the JVM. If a connection is established with TLS 1.0 to a specific host (like say host1.ibm.com) then it caches the TLS cipher that was exchanged with the host so that it doesn’t have to do negotiate again. Now if another app in JVM tries to establish connections with TLS 1.2 to the same host (host1.ibm.com) the cached info is reused. This causes the problem. Host1.ibm.com is requesting TLS 1.2 but cached connection in JVM is at TLS 1.0”

It seems that the RTE application is not honouring the protocol as detailed in the Quality of protection (QoP) settings in the ISC and is trying to connect using TLS 1.2. This has triggered Technote How to Force IBM Connections 5.5 CR1 to Use TLSv1.2.

Furthermore I was made aware of LO89164 which is an update to RTE so that it utilises the intraservice URLs for each of the connections components and is be able to direct the HTTP requests issued by RTE component to the appropriate internal endpoint URLs, as referenced in the LCC.xml.

LO89164 can only be applied to CR01 which the customer is not running yet. LO89164 will be included in CR02. This ifix is only available from IBM by logging a PMR, it’s not on Fix Central.

When I am given a maintenance window I need to do the following:

1. WAS FP8
2. CR01
3. Apply LO89164 (included in 5.5 CR2)
4. Change QoP settings

This should cover off this problem. CR01 also includes other identified bugs in RTE which I won’t go into here but it’s worth applying….