Problem Determination for WebSphere for z/OS

60 Problem Determination for WebSphere for z/OS

6.1 What is a hang?

A WebSphere Application Server hang is when an address space for the application server

on z/OS is not processing work and appears to be idle. A hang can be noticed when the

server no longer processes requests. The scope of this symptom is restricted to a hang in the

WebSphere for z/OS environment.

6.2 Symptom flow chart: Hang

Figure 6-1 shows the flow chart for this symptom. Each box has a number that refers to the

analysis in more detail.

Figure 6-1 Flow chart for symptom: Hang in the application server

6.3 Diagnosing a hang

Follow the these steps to analyze and solve the hang problem:

1. Determine whether it is an application server hang.

Typically, a user who is not getting a browser response reports a hang. This could be a

hang in the application, the HTTP server, or the WebSphere Application Server.

You can perform either of these checks to determine whether it is a WebSphere for z/OS

hang:

Identified

problem and

solution?

9

Yes

1

Determine if

application

server hang

No

No

2

Check and set

hang detection

variables

3

Analyze output

from hang

dectection

variables

Identified

hang?

4

5

Take corrective

action

10

Take corrective

action

11

Search IBM

support pages

12

Identified

problem and

solution?

Yes

Yes

No

13

Assemble

MustGather

documentation

Analyze dump

8

Capture dump of

hung ASID,

master and other

7

Issue diagnostic

commands

6

14

Contact IBM

support

Chapter 6. Hang 61

– Run a simple test by entering a display command for the WebSphere for z/OS server

that you suspect has a hang and wait for a response, for example:

MODIFY <server Name>,DISPLAY

If you do not get a response, it is very likely that the WebSphere for z/OS server is

hung.

– Check the latest time stamps in the WebSphere for z/OS server logs against the

current system time. How much time has passed between system time and last

recorded activity?

If you get a response from the server or the last recorded activity is close to the system

time, then the problem might be with the HTTP server or the application itself.

Check whether the request from the browser has arrived at the HTTP server by reviewing

the HTTP server access logs and error logs. For more information, see 19.5, “IBM HTTP

Server logs and trace” on page 232. If the HTTP request has arrived at the server and the

HTTP server is responding to requests, then it might be the application that is hung.

2. Check and set hang detection variables.

WebSphere for z/OS V6 has a thread hang detection option and it is enabled by default. To

adjust the hang detection policy values or to disable them, go to the Administrative

Console and select Servers → Application Servers → server_name. Under

Infrastructure, select Administration → Custom Properties. Then, select New. The

properties are:

– Name: com.ibm.websphere.threadmonitor.interval

– Name: com.ibm.websphere.threadmonitor.threshold

– Name: com.ibm.websphere.threadmonitor.false.alarm.threshold

For full explanation of the thread detection properties, search for the WSVR0605W

message at the WebSphere for z/OS V6 Information Center at:

http://publib.boulder.ibm.com/infocenter/wasinfo/v6r0/index.jsp

If the WebSphere for z/OS server address space is hung, you cannot set or adjust these

properties at this time. You must wait until the hang situation is cleared.

3. Analyze output from hang detection variables.

If you have the hang detection variables set, then you can see WSVR0605W messages in

your servant region job log output (Example 6-1).

Example 6-1 WSVR0605W message example

Trace: 2005/07/20 15:45:45.013 01 t=6C1AC8 c=UNK key=P2 (13007002)

ThreadId: 00000016

FunctionName: com.ibm.ws.runtime.component.ThreadMonitorImpl

SourceId: com.ibm.ws.runtime.component.ThreadMonitorImpl

Category: WARNING

ExtendedMessage: BBOO0221W: WSVR0605W: Thread "HAManager.thread.pool: 0"

(00000030) has been active for 642415 milliseconds and may be hung. There is/are

1 thread(s) in total in the server that may be hung.

Note the thread name and ID. They might help you determine the problem when you are

searching the IBM support pages or reporting the problem to IBM.

4. Have you identified the hang?

Using the hang detection properties and the output from the WSVR0605W message, have

you been able to identify the hung thread? If yes, then take corrective action as described

in step 5. If no, issue some diagnostic commands and prepare for a dump as described in

step 6.