Each of the first three tests were done once with 200 clients talking to the server serially and then with all them talking to the server concurrently. Each number is an average of the time taken to get a locator and handle to the CASPort and the actual method invocation. All numbers are in milliseconds.
The stress test framework in GT3 Core was used for testing. This framework allows a block of code defined in an "init" method for initializing, code to be timed in "run" method and finally a "postRun" methd. Two of the numbers the stress framework that are of relevance here are
The server and client were run on the same machine. Database (PostGres) was used and was also on the same machine.
All tests were run with two different sets of data in the backend database. But in both cases the default bootstrap was done which populated the database with service/action implict to CAS service.
In the case of smaller data set, other than the data described above, a single trust anchor, usergroup, user and object was added. Two policies were added - one granting that user group super user permissions on the CAS server and another granting it access to the external object added. (only the latter is returned on an assertion generation request.)
In the other case, 100 user groups, 100 users, 100 resources and 1000 polices was added above the deafult data. Each user group was given ten polices across the resources.
This test measures the time taken to query the database to get the CAS nickname of the user making the call. There are no parameters to process in this call and is the most rudimentary query in the CAS service. Every single CAS invocation does this to get the name of the user to ascertain permissiosns.
The test was run with 200 client threads, serially and cocurrently.
Serial | Concurrent | ||||
Small | 253 (253) | 155(26832) | |||
Large | 253(253) | 214(40958) |
This test measures the time taken to query the database to get all applicable policy for the user making the call. The CAS server checks permissions for the operation and performs the actual retrieval of policy and is returned as an array of PolicyData objects.
The test was run with 200 client threads, serially and cocurrently. With smaller data set only one policy is returned, with larger dataset 10 amongst the 1000 policies are returned.
Serial | Concurrent | ||||
Small | 315(315) | 219(34425) | |||
Large | 479(479) | 286(40298) |
This test measures the time taken to query the database to get all applicable assertions for the user making the invocation. The CAS server checks permissions for the operation and performs the actual retrieval of assertion. Since the assertion is returned as xsd:any the time taken to convert them in SAMLAssertion object is also measure.
The test was run with 200 client threads, serially and cocurrently. With smaller data set assertion contains only one authorizatin policy , with larger dataset 10 polices amongst the 1000 policies are embedded in the assertion.
Serial | Concurrent | ||||
Small | 416(416) | 303(50032) | |||
Large | 568(568) | 400(52304) |
For credendial of size 70995, using 50 threads serially, the avarage timetaken was 6833ms.
These targets add some specific number of users to the database. One of the users needs to have the subect DN of the credential that will be used while running the tests. This is picked up from a variable "subject1DN" from a file, whose name is passes as a system property to each of this target. But default, this is set to "casTestProperties". To override it, set -Dcas.test.properties while invoking the target.
To get configuration parameters for database access, a file with database information is required as shown here. This is by default, "casDBProperties". To override it, set -Dcas.db.properties while invoking the target.
To set up small datasets in database, the following from CAS_HOME
psql -U tester -d casDatabase -f etc/delete.sql
ant bootstrap
ant populateDB -Dargs="small"
To set up large datasets, but credential size less than 50K, in database, the following from CAS_HOME
psql -U tester -d casDatabase -f etc/delete.sql
ant bootstrap
ant populateDB -Dargs="large"
To set up large datasets, but credential size greater than 50K, in database, the following from CAS_HOME
psql -U tester -d casDatabase -f etc/delete.sql
ant bootstrap
ant populateDB -Dargs="large largeCred"
<sysproperty key="org.globus.ogsa.client.timeout" value="200000"/>
The above is the solution if "Read timeout error" is seen on the client side. In all other cases, a default GT3 container with CAS service deployed should suffice.
ant basicQueryStress -Dstress.threads=300 -Dstress.processes=1 -Dstress.concurrent=yes
ant queryStress -Dstress.threads=300 -Dstress.processes=1 -Dstress.concurrent=no
ant credentialStress -Dstress.threads=200 -Dstress.processes=1 -Dstress.concurrent=yes
ant credentialStress -Dstress.threads=1 -Dstress.processes=1 -Dstress.concurrent=no
The whoami method on CAS port was overridden to just return a dummy string without any processing. The above described "Basic Query Test" was done to measure performance with 200 serial and concurrent client. On serial testing, time taken was 252ms (252) and on concurrent testing it was 187ms (34209). So it appears that irrespective of the application code, the infrastructure takes a considerably long time on concurrent calls.