Skip to content

Testing

Testing notes: load and stress tests, the C10K problem and code coverage.

3 min read

References

Testing

Automated Testing Pyramid

  • System level -> Regression/System/Stress/Performance testing
  • Controller level -> Integration testing
  • Service level -> Unit testing

Load and Stress Test

A load test, by definition, measures the performance of a system under an expected load. In contrast, a stress test overloads a system in order to find the breaking point.

A load test is performed in order to ensure that a website or web application is capable of handling specific numbers of users at once. Load Testing is often used in the process of capacity planning, to ensure that a system is able to handle growth to specified levels of simultaneous traffic.

A stress test is used to specifically push a system beyond its intended capacity to identify components that begin to slow down, identify bottlenecks in the system, and bring to light possible points of failure.

  • jmeter - Apache JMeter to load test functional behavior and measure performance. One JMeter instance runs for max 800 users.
  • loader.io - Loader.io is a FREE load testing service that allows you to stress test your web-apps & apis with thousands of concurrent connections.
  • ab - Apache HTTP server benchmarking tool
  • BlazeMeter - Continuous Testing platform. Distributed JMeter for stress test. Uses JMeter scripts. Since it is distributed runs multiple JMeter instance.

Example ab testing for 100 requests with 10 concurrency (clients): We make 100 requests to [url] with 10 concurrency (we can think of it as 10 users making requests at the same time) as follows.

Terminal window
$ ab -k -n 100 -c 10 [url]

Example ab testing for 10 clients 1000 times added header:

Terminal window
$ ab -k -n 1000 -c 10 -H “Authorization: Bearer <…token…>” http://api_url/

Usually what you get per service at least 4000 transactions per second. 4000-7000 per second per service.

What it means in practice in real world is that application service scalability is never the issue, the problem is always the database.

If the database is not scaled, the more application servers you will start upfront everyone will meet the backend and this usually probably causes problems.

If there is huge number of failed request issue, it is due to dynamic pages. It happens because the Content-Length can vary between the requests. When using ab with such pages you need to use the -l option.

-l Accept variable document length (use this for dynamic pages)

C10K Problem

The C10K problem

example test can be done with

Terminal window
$ ab -n 10000 -c 16 -l https://domain.com:443/index.html

This runs 10.000 HTTP requests with a concurrency of 16.

Example test results:

Concurrency Level: 16
Time taken for tests: 403.432 seconds
Complete requests: 10000
Failed requests: 0
Total transferred: 131810000 bytes
HTML transferred: 129120000 bytes
Requests per second: 24.79 [#/sec] (mean)
Time per request: 645.492 [ms] (mean)
Time per request: 40.343 [ms] (mean, across all concurrent requests)
Transfer rate: 319.06 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 436 487 66.4 477 1854
Processing: 137 157 65.0 149 2022
Waiting: 136 150 34.4 146 1168
Total: 576 644 106.7 627 2822
Percentage of the requests served within a certain time (ms)
50% 627
66% 641
75% 652
80% 656
90% 669
95% 716
98% 841
99% 1012
100% 2822 (longest request)

Code Coverage

  • JaCoCo - Java Code Coverage Library
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version></version>
<executions>
<execution>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>report</id>
<phase>prepare-package</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version></version>
<executions>
<execution>
<id>default-prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>default-report</id>
<phase>prepare-package</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
<execution>
<id>default-check</id>
<goals>
<goal>check</goal>
</goals>
<configuration>
<rules>
<rule>
<!--requires a line coverage minimum of 50% for every class except test classes-->
<element>CLASS</element>
<excludes>
<exclude>*Test</exclude>
</excludes>
<limits>
<limit>
<counter>LINE</counter>
<value>COVEREDRATIO</value>
<minimum>50%</minimum>
</limit>
</limits>
</rule>
</rules>
</configuration>
</execution>
</executions>
</plugin>

Related articles

1 min read

Sonar

Static code analysis with SonarQube and SonarCloud from Maven builds.

  • Observability & Testing
  • Developer Tools
1 min read

APM

Application performance monitoring for Java servers, including setting up the Glowroot agent on GlassFish.

  • Observability & Testing
1 min read

ELK Stack

Running the ELK stack in Docker, creating the default Kibana index pattern and shipping Java logs with Logback.

  • Observability & Testing

Type to search the whole site.