Why do they get Hung?
The main reason the threads hang is because it can not finish the request in a timely manner. If your data needs continuously updating, some of queries loose efficiency because their execution paths over time need to be re-analyzed. Over the last couple years we have seen this in several of our queries.

Since we have a set number of available DRP threads (40 in our case), when they are all hanging, the application server is no longer able to serve requests.

This problem is an Enterprise problem. This means all ATG applications run into this problem, if the queries don’t return.

The solution to the problem.

Overview
The solution is to first monitor and identify DRP hung threads, send email regarding their hung status, and then kill them before they can do damage. This solution requires a monitor running inside the application server that will periodically take an inventory of the running DRP threads. When a thread age reaches a predetermined threshold, it will be terminated, and information about the terminated thread will be sent to a predetermined email for evaluation.

The following are the control inputs for the DRPThread Monitor stored in a property file. The property file is loaded upon startup.

This tells the monitor to auto-start.
threadMonitorAutoStart = true

This is how often it checks the DRP threads
threadMonitorWaitInterval = 5000

This is percent of DRP threads in use before checking threshold
threadMonitorThreshold= 80

This is the Age limit of threads. When the age limit is surpassed
thresholdAgeLimit = 300000

This is the max age of any thread. A thread over this age is killed
maxThreadAge= 1800000

List of email address sent notifications when a DRP thread is killed
emailNotificationList = hwilliamson@greattastingjava.com

Pseudo Code

1. Application server starts up
2. Starts DRP Monitor

a. If threadMonitorAutoStart = true, then monitor continues to start up, else it stops.
b. Loop until forced stop

i. Check DRP server
1. percent active thread > threadMonitorThreshold && age of thread > thresholdAgeLimit) || age of thread > maxThreadAge then
a. Email info about hung thread
b. Kill thread
ii. Sleep threadMonitorWaitInterval

Code Sample

while (!forcedStop) {

try {

RequestServerHandler handlers[] = drpServer.getRequestHandlers();

int activeCount = drpServer.getActiveHandlerCount();

double percentActive = (activeCount/handlers.length) *100.0;

for (int i = 0; i < handlers.length; i++) {

if (handlers[i] instanceof DrpServerConnection) {

DrpServerConnection connection = (DrpServerConnection) handlers[i];
if (connection.getHandlingRequest()) {

if ((percentActive> threadMonitorThreshold && connection.getCurrentRequestTime() > thresholdAgeLimit) ||connection.getCurrentRequestTime() > maxThreadAge)
{
logger.debug(”killing:” + connection);

String path = connection.getCurrentRequestPathInfo();
long time = connection.getCurrentRequestTime();

connection.killHandler();

Mail mail = new Mail();
mail.postMail(emailNotificationList,”DRP Thread Killed”, “Path:” +path +”\n”
+”Time:” +time+”\n”,”hwilliamson@greattastingjava.com”);
logger.debug(”emailing:” + emailNotificationList);

}
}
}
}
sleepDRPThreadMonitor();
} catch (Exception ex)
{
ex.printStackTrace();
}
}

I have written a multithreaded hash table framework. You can put multiple tasks to do work for you with a simple “put” with a key,Then when it completes you can do a “get” to return the data worked on. The “get” will block until that requested thread is finished working.

The best way to use this is to first “put” all the work that you need to get done. Then after you can start doing a “get” on what ever data is needed.

I have used this framework to reduce a 6 minute batch process down to 13 seconds.By slicing up the work to do the data gathering first (puts) , and then data assembly second(gets).

Of course The slowest slice is the fastest your program is going to run.
Good luck…

HWThreadedHash.java
HWThread.java
HWThreadedHashExample.java

threads,multithread,performance,hashtable,hash, Java, programming, software, examples

This is what I found:

Bean Managed Transaction in Message Beans:
• Single threaded if more than one data source was used.

Container Managed Transaction in Message Beans:
• Multi threaded even if more than one data source was used, but slower due to XA Transactions.

The solution I found:
Use Container Management Transactions, but use a stateless session bean with bean managed transactions to access your second data source.

Message Beans, Java, MQ, Transactions

So here are the instructions for configuring Optimizeit with ATG Dynamo 6.4

1. When Optimizeit first starts it will ask you for the path to the JVM you want to use. I set it to the Dynamo jvm C:\ATG\DAS6.3.0\DAS\os_specific_files\i486-unknown-win32\jre\bin\java.exe

2. In my C:\ATG\ATG6.4.0\home\localconfig I edited the PostEnvironment.bat file added/edited the following lines:

set OPTIT_HOME=C:\\Optimizeit\\OptimizeitEntSuite60
set CLASSPATH=%OPTIT_HOME%\lib\optit.jar;
%OPTIT_HOME%\\lib\\flexlm.jar;%CLASSPATH%
set PATH=%PATH%;%OPTIT_HOME%\\lib;
SET JAVAPARAMETERS=%JAVAPARAMETERS% -Xrunpri:directi=f
-Xnoclassgc -Djava.compiler=NONE -Xbootclasspath/a:
"%OPTIT_HOME%\\lib\\oibcp.jar"
set JAVA_ARGS=%JAVA_ARGS% %JAVAPARAMETERS%

3. Then I made the following chang to In my C:\ATG\ATG6.4.0\home\bin\startDynamo.bat

%JAVA_VM% %JAVA_ARGS% %SECURITY_ARGS%
%DYN_EXTRA_ARGS% intuitive.audit.GenericAudit
atg.applauncher.dynamo.DynamoServerLauncher
%DYNAMO_MODULES% %CONFIGPATH% -shellrestart %LOGARGUMENT%

Technorati Tags: Java, ATG, Dynamo, Dynamo 6.4, Optimizeit 6.0, Optimizeit

Technorati Tags: Java, ATG, Performance, Optimizeit , Dynamo