Understanding the different test types

The behavior of the loader.io test types are an occasional source of confusion, so I whipped up some diagrams to help clarify how these behave.

Clients per test

This is what a clients per test run looks like (click the image to enlarge):

Example test run with "clients per test"

Each arrow represents a client making requests against the web server. The length of the arrow represents the response time for a client to access all of the urls in a test.

The test parameters are

  • clients (per test): 100
  • duration: 20 seconds

In this case, loader.io will split up those 100 clients across the test duration, so that 5 new clients start making requests at each second. Note that each client only makes one request per test url, then stops.

Clients per second

This is what a clients per second run looks like:

Example of "clients per second" test

The test parameters are

  • clients (per second): 5
  • duration: 20 seconds

In this case, loader.io will start 5 new clients at each second.

A couple of things to note:

This test looks very similar to “clients per test”, because the two types are equivalent. You can either specify the total number of clients you want in the test (and loader.io will distribute the clients at regular intervals), or you can directly specify the number of clients you want to start at each second.

Even though this is a 5 client per second test, sometimes there are more than 5 clients active at a time. (For example, at t=9, there are 8 clients active). This is because clients that started at an earlier time had not completed yet.

Maintain client load

This is what a maintain client load run looks like:

Example "maintain client load" run

The test parameters are

  • from: 0 clients
  • to: 50 clients
  • Duration: 20 seconds

In this case, once a client completes its requests against the web server, it starts making requests again, and will continue to cycle through the test urls until the test has completed.

Another way to think about these test types is:

For clients per test and clients per second tests, the total number of requests in the test is fixed by the test parameters, but the number of clients active at any one time will vary based on how quickly the web server can respond to requests.

For maintain client load tests, the number of clients active at any one time is fixed by the test parameters, but the total number of requests in the test will vary based on how quickly the web server can respond to requests.