Servers Performance Monitoring in JMeter

As a performance engineer, it is our responsibility to gather the performance stats from all the layers. The statistics from the tool alone will not help us in identifying the performance bottlenecks. As the technology getting complex every day, it is critical to monitor and gather the metrics from across the tiers. In this blog article, we are going to focus on Servers Performance Monitoring in JMeter.

When the test is on, you should not go out for a coffee or browse the internet. As a responsible performance engineer, you should monitor the execution in the tool or in APM solutions or the logs.

Servers Performance Monitoring

JMeter stats reveals the end-to-end performance stats in runtime. But it doesn't reveal how your backend is performing. When the load is being injected, your servers will undergo a lot of processing and sometimes it will even break. By monitoring the servers' performance, you can identify when the server is getting throttled and you can get started with your detective work.

How to enable Servers Performance Monitoring in JMeter?

There are two setups you need to take care to monitor the servers performance in JMeter.

1. Server Agent setup
2. PerfMon Metrics Collector setup

Server Agent setup

As I mentioned, by default JMeter doesn't collect the servers' health metrics (except Tomcat). But you can leverage open source library called SIGAR which should run on the each server where the application under test is hitting.

Download the PerfMon Server Agent from GitHub. Extract it and keep it in each server which you would like to gather the metrics. No need to have a root or administrator access.

Go to the extractor folder and run startAgent.bat if you are in Windows or run startAgent.sh in macOS/Linux operating system.

This will start the agent in the default port 4444 as shown below.

To change the port you can use this command .\startAgent.bat --tcp-port 3450.

PerfMon Metrics Collector setup

Open the JMeter test plan which you will be executing. Open the Plugins Manager to install the PerfMon Metrics Collector listener. Click on Available Plugins and search for PerfMon to install the listener.

Once the plugin is installed, restart the JMeter and add the jp@gc - PerfMon Metrics Collector by right clicking on the Thread Group > Add > Listener > jp@gc - PerfMon Metrics Collector. This will add a listener to your test plan as shown below.

Configuring jp@gc - PerfMon Metrics Collector

Click on Add Row button which will add the Host, Port, and Metric to collect with the default values.

You can add multiple metrics to collect by clicking on Add Row button.

Testing the Server Agent

After adding the listener, validate your test plan by right clicking on the Thread Group > Validate. This will send one iteration of your requests in the test plan.

Head back to the command prompt (or terminal) where the Server Agent is up and running where you will see the Yep, we received the 'test' command output as shown below. This means the server agent is ready to measure the metrics.

Start the test

Once the validation is successful, you can start your test in CLI mode. Just for demonstration purpose, I am executing the test in GUI mode. In this example, I have configured to monitor the CPU and Memory metrics.

After few seconds of the test started, listener will plot the graphs which it got from the server agent.

This is my Memory screenshot from the Resource Monitor in Windows which matches the JMeter screenshot.

Configuring Metrics

To configure the metrics in the listener follow the below steps as shown in the screenshot. If the quality of the image is not good, right click on the image and open it.

Apart from typical metrics such as CPU, Memory, Swap etc, there are two metrics: EXEC and TAIL will be helpful based on the use case.

EXEC helps to execute command and collect the data and display it in the chart. This command helps to get the process count in Powershell.

C\:\Windows\system32\WindowsPowerShell\v1.0\powershell.exe:(Get-Process).Count

Here is the runtime chart for the above command.

TAIL helps to parse the file and extract the metrics and display it in runtime. Suppose, if you want to parse through some logs file which has one column, you can do that using TAIL.

Troubleshooting

If you are trying to start the server agent, you might face issues in the command prompt or terminal with the below error.

INFO    2021-01-12 19:39:53.797 [kg.apc.p] (): *** Logging available processes ***
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000010014ed4, pid=10284, tid=10180
#
# JRE version: OpenJDK Runtime Environment (15.0+36) (build 15+36-1562)
# Java VM: OpenJDK 64-Bit Server VM (15+36-1562, mixed mode, tiered, compressed oops, g1 gc, windows-amd64)
# Problematic frame:
# C  [sigar-amd64-winnt.dll+0x14ed4]
#
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
# C:\Tools\ServerAgent-2.2.3\hs_err_pid10284.log
#
# If you would like to submit a bug report, please visit:
#   https://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.

To solve this issue, please check this GitHub Issue. Download the sigar-amd64-winnt.dll and put it under lib folder. It is advisable to take a backup.

If you are not able to start, the reason might be a firewall.

Following commands could be useful for the server agent.

.\startAgent.bat --sysinfo

.\startAgent.bat --interval <seconds>

.\startAgent.bat --udp-port 0 --tcp-port 5555

Final Words

By leveraging the Server Agent and PerfMon Metrics Collector, you can monitor the performance of the hosts from the JMeter which will help you to find the bottlenecks and validate the health metrics.

Please let me know if you face any issues while implementing this solution.