[WAS]Configuring TLSv1.2 for WebSphere Application Servers 8.5.5

https://developer.ibm.com/recipes/tutorials/configuring-tlsv1-2-for-websphere-application-servers-8-5-5/

Configuring TLSv1.2 for WebSphere Application Servers 8.5.5

IBM BPM 8.5.6 Configuration for TLSv1.2 - By Chris Osborne & Joseph St.Clair

jtstclai@us.ibm.com
Published on December 19, 2016 / Updated on December 19, 2016

Overview

Skill Level: Intermediate

Must be familiar with WebSphere Application Server and Linux Admin functions

Working with IBM BPM 8.5.6 in a secure environment comes with its challenges including ensuing that the application server is TLS compliant to which ever level your organization requires. This document is based on configuring the application server.

Ingredients

• User must be an administrator for WebSphere and the Operating System (Redhat Linux is used for this example)
• There is a working installation of WebSphere and is currently setup for SSL_TLS
• User has admin access to the WebSphere Console and Application(s) using certificate based authenication

Step-by-step

1. Getting the Environment Ready

   1. Log into the WebSphere Console via a browser – ex. https://<FQN-Server-Name>:9043/ibm/console
   2. Shutdown any actively running applications
      • Click on “Servers -> All Servers”
      • Select all the application servers, then click the “Stop” button to shut down the applications
   3. Remove security for the WebSphere Console
      1. Click on “Security -> Global security”
      2. In the main Global security window, unselect the “Enable administrative security” check box
      3. Click “Apply”

   

   1. Stopping the WebSphere Node agents and Deployment Server
      • Click on “System administration -> Node agents”
      • Select all the node agents and click on the “Stop” button
      • Click on the “Deployment Manager“
      • Click the “Stop” button
      • On the “Stopping Server” screen click the “OK” button
      • At this point the applications and WebSphere environment should be down

2. Backup the Environment

   1. Create a backup and verify security has been disabled
      • open a putty session to the deployment server and log into the box
      • Be sure to sudo into “wasadmin”
      • Go to the following directory: “data/IBM/WebSphere/AppServer/bin”
      • Run the following command: “./backupConfig.sh” (Remember our example is running Redhat Linux)
      • Once completed, go to the following directory: “data/IBM/WebSphere/AppServer/profiles/DmgrProfile/bin”
      • Run the following command: “./backupConfig.sh“
      • Once completed, goto the following directory: “data/IBM/WebSphere/AppServer/profiles/Node1Profile/bin”
      • Run the following command: “./backupConfig.sh“
      • If you have a second or thrid node, putty into those servers and repeat the steps for the node backups

3. Ensure Security is Off!

   1. Confirm security is turned off on all servers in the cell
      • Return to the Deployment Server via a putty session
      • Goto the following directory: “/data/IBM/WebSphere/AppServer/profiles/DmgrProfile/config/cells/PCCell1”
      • Run the following command: “vim security.xml”
      • Verify the security has been turned off:

   <?xml version=”1.0″ encoding=”UTF-8″?>

   <security:Security xmi:version=”2.0″ xmins:xmi=”http://www.omg.org/XMI” xmins:orb.securityprotocol=”http://www.ibm.com/websphere/appserver/schemas/5.0/orb.securityprotocol.xmi” xmins: security=”http://www.ibm.com/websphere/appserver/schemas/5.0/security.xmi” xmi:id=”Security_1″ useLocalSecurityServer=”true” useDomainQualifiedUserName=”false” enabled=”false” cacheTimeout=”600″

   • If the setting is set to “true“, change it to “false“, then save the file
   • Verify this on both nodes: “/data/IBM/WebSphere/AppServer/profiles/NodeXProfile/config/cells/PCCell1”
   • Once the security setting has been turned off on BOTH nodes, start the deployment server:”/data/IBM/WebSphere/AppServer/profiles/DmgrProfile/bin/startManager.sh”
   • Start each of the nodes: “/data/IBM/WebSphere/AppServer/profiles/NodeXProfile/bin/startNode.sh”

4. Configuring TLSv1.2

   In order to configure the application server to use the Transportation Layer Security (TLS) version 1.2 we will need to return to the IBM WebSphere Adminstration Console.

   1. Log back into the WebSphere console
   2. Click on: “SSL certificate and key management > SSL configurations > CellDefaultSSLsettings > Quality of protection (QOP) settings”
   3. Click on the pull down box for “Client authenication” and select “Required“
   4. Click on the pull down box for “Protocol” and select “TLSv1.2“
   5. Click on the “OK” then “Review” and “Save“
   6. Click on: “SSL certificate and key management > SSL configurations > NodeDefaultSSLSettings > Quality of protection (QOP) settings” (Node 1)
   7. Click on the pull down box for “Protocol” and select “TLSv1.2“
      Click on the “OK” then “Review” and “Save“
      Click on: “SSL certificate and key management > SSL configurations > NodeDefaultSSLSettings > Quality of protection (QOP) settings” (Node 2)
   8. Click the pull down box for “Client authenication” and select “Required“
   9. Click on the pull down box for “Protocol” and select “TLSv1.2“
   10. Click on the “OK” then “Review” and “Save“
   11. Click on: “SSL certificate and key management > SSL configurations > XDADefaultSSLSettings > Quality of protection (QOP) settings”
   12. Click on the pull down box for “Client authenication” and select “Required“
   13. Click on the pull down box for “Protocol” and select “TLSv1.2“
   14. Click on the “OK” then “Review” and “Save“

   

   1. Stopping the WebSphere Node agents and Deployment Server
      • Click on “System administration -> Node agents”
      • Select all the node agents and click on the “Stop” button
      • Click on the “Deployment Manager“
      • Click the “Stop” button
      • On the “Stopping Server” screen click the “OK” button
      • At this point the applications and WebSphere environment should be down

    

5. Change the SSL Properties Files on the Server

   We will need to return to the Application Server's properties files to update the SSL settings. You will need to (in this case Linux) open a putty session for the deployment server.

   1. Open a putty session into the WebSphere Deployment Server
   2. Go to the following directory: $WAS_HOME/profiles/DmgrProfile/properties/
   3. Edit the file: “ssl.client.props” file
   4. Change the line: “com.ibm.ssl.protocol=XXXX” to: “com.ibm.ssl.protocol=TLSv1.2“
   5. Save the file
   6. Go to the following directory” $WAS_HOME/profiles/NodeXProfile/properties/
   7. Edit the file: “ssl.client.props” file
   8. Change the line: “com.ibm.ssl.protocol=XXXX” to: “com.ibm.ssl.protocol=TLSv1.2“
   9. Save the file
   10. Log into your second WebSphere application Node
   11. Go to the following directory” $WAS_HOME/profiles/NodeXProfile/properties/
   12. Edit the file: “ssl.client.props” file
   13. Change the line: “com.ibm.ssl.protocol=XXXX” to: “com.ibm.ssl.protocol=TLSv1.2“
   14. Save the file

6. Recycle & Complete the Configuration

   After all these changes we need to ensure we recycle the environment to complete the configuration changes we have made. 

   1. Starting the environment:
      • Putty into the “Deployment Sever” and run the following command: “/data/IBM/WebSphere/AppServer/profiles/DmgrProfile/bin/startManager.sh“
      • Start each of the nodes by running the following command: “/data/IBM/WebSphere/AppServer/profiles/NodeXProfile/bin/startNode.sh“
      • Note: To start the second node you have to putty into each node
   2. Verify each of the WebSphere nodes are in Sync
      • Log into the WebSphere console
      • Click on “System administration > Nodes
      • Verify each of the nodes are in sync, if not, select all the nodes and click on the “Full Resynchronize” button
   3. Adding security back to the WebSphere Cell and Applications
      • Click on: “Security > Global security“
      • Click on the check box “Enable administrative security“
      • Click on the check box “Enable application security“
      • Click “OK“, “Review” and “Sync“
   4. Restart the environment
      • Click on “System administration -> Node agents“
      • Select all the node agents and click on the “Stop” button
      • Click on the “Deployment Manager“
      • Click the “Stop” button
      • On the “Stopping Server” screen click the “OK” button
   5. Starting the environment:
      • Putty into the “Deployment Sever” and run the following command: “/data/IBM/WebSphere/AppServer/profiles/DmgrProfile/bin/startManager.sh“
      • Start each of the nodes by running the following command: “/data/IBM/WebSphere/AppServer/profiles/NodeXProfile/bin/startNode.sh“
      • Note: To start the second node you have to putty into each node
   6. Verify the WebSphere Console and Application are using TLSv1.2
      • Open a browser and connect to the login page of the WebSphere console
      • Right click on the server certificate next to the WebSphere console address
      • Verify the server is running TLSv1.2
      • Log into the WebSphere console
      • Start the server applications, once the application(s) have started, open a browser and check the certificate to verify TLSv1.2 is working as expected