Run massively scalable performance tests on web, mobile, and APIs

Request a Demo
May. 28th, 2018

RTMP Performance Testing with JMeter: Learn How

RTMP is one of the most commonly used protocols for real-time video streaming. In this article, we will look at ways to load test services that use this protocol, with Apache JMeter™. We will cover a few different options.

 

About RTMP

 

Real Time Messaging Protocol is a proprietary TCP-based protocol developed for real time streaming audio, video and data. The protocol can split video and data into fragments while simultaneously maintaining the ability to transmit bigger chunks of information, to ensure smooth delivery of video and audio streams. The size of the fragments used can be negotiated dynamically between the client and server, and even disabled completely if desired. The default fragment sizes are 128 bytes for video and most other data types, and 64 bytes for audio data.

 

RTMP Servers

 

Unfortunately, RTMP has known problems with licensing, for example, its official specification does not allow users to write a full-fledged RTMP server, if you follow it thoroughly. However, there are several implementations:

  • Red5 - one of the first RTMP-servers, written in Java
  • Wowza - proprietary RTMP server, written in Java. In addition to RTMP, it also supports RTSP, Apple HLS and Smoothstreaming (Silverlight)
  • Erlyvideo - open source RTMP server with separately distributed proprietary modules, written in Erlang
  • NGINX-based Media Streaming Server      

 

Because of the unclear specification, every developer had to add something to her/his server implementation, and developing a universal client is not such a simple task (if it is real).

 

For the demo, we will use Red5 Pro server. It has a trial access (30 days) and to start working with it you only need 2 clicks (really, and it’s adorable), also it has another free version with reduced functionality.

 

You can download it from here and run .bat file for Windows or .sh for Linux users.

 

Note for Windows users: DO NOT USE ‘Run as Administrator’ option, just double click, otherwise you will get an error when trying to stream a video.

 

Now you can go to http://localhost:5080/live/broadcast.jsp and start broadcasting. After that, you can watch it using RTMP and 1935 port. That is all!

 

Creating Your JMeter Script

 

At the moment there are no JMeter supported plugins that allow you to read the stream or write to it. Two plugins were found: FlazrJmeter, which is based on the Flazr library and RtmpPluginForJmeter, which is based on the Red5 library, but the last commit for the first one was 2 years ago and for the second one it was 7 years ago. We will discuss FlazrJmeter below, but the RtmpPluginForJmeter is too old, so we will write our own sampler to work with RTMP using Red5 library.

 

Option 1: JAVA Request Sampler + Red5-client library

 

We will not use the JSR223 sampler now due to the amount of code. Instead, we will use the Java Request Sampler. Besides, the JSR223 Sampler may work slower than the JAVA Sampler in some cases, because it compiles code during the test.

 

We can both stream the video and watch it. A client who streams the video is called a publisher and the watcher is called a subscriber. The client is the element that interacts with the server. 

 

To quickly learn how to implement Java class for the Java Request Sampler, read here.  We got the following class for the subscriber: click here.

 

rtmp load testing

 

The parameters:

  • url is the RTMP server address
  • port is the RTMP port, the default is 1935
  • app is the name of the application that broadcasts the video stream (for Red5 Pro, it is ‘live’ by default)
  • streamName is the name of the stream
  • timeout is the duration of the subscription in seconds, the time that the client will be subscribed to the stream. The stream must exist for the whole time of the subscription.

 

For the publisher, the class is similar, but we will add a new parameter for the set up path to the video that will be streamed.

 

jmeter testing rtmp

 

Now let us proceed to the implementation of the subscriber and the publisher.

 

For the subscriber implementation, tests from the red5-client repository and the source code of the RTMP Bee tool have been observed.

 

The subscriber has the following flow:

  1. The client establishes connection using the client.connect() method.
  2. The ConnectCallback processes the answers of the server.
  3. If the handshake is successful, client.createStream() initiates StreamConnection.
  4. If step 3 is successful, client.play() starts playing video stream until the timer expires.

 

For the publisher implementation, YouTube RTMP Client implementation and this topic has been observed.

 

The first three steps of the publisher’s flow are similar to the steps of the subscriber thread, but at the fourth step we execute client.publish() to start broadcasting with the ‘recording’ parameter instead of live, because we stream the flv file. Also we need the flv file that will be streamed.

 

After downloading the code and potential corrections making, you need to package it and put in to /lib/ext. Besides, all third-party libraries from the .pom file should be put into the folder. Thereafter you can choose these classes in the Java Sampler as shown on the pictures above.

 

Option 2: JAVA Sampler + Flazr library

 

Another option for performance testing RTMP is with the same sampler as before, but with another library. We can use part of the FlazrJmeter plugin. In addition, we need the PullRequestTest.java and PushRequestTest.java files from the repository, which are classes that are ready for subscription/publishing, and the Java Request Sampler.

 

You need to compile them and then put them and the required libraries in the lib/ext folder as in the example above.

 

jmeter load testing video streaming

 

The parameters for PullRequestTest are the following:

  • IP, port, app, stream - similar to the ones in the Red5 paragraph
  • Version - the version of the tested server, you can set 000000
  • Threads - number of threads, you can regulate the number of threads using this field, but it will be more correct to set the value to 1 and set up the thread number using a thread group
  • Load - the number of clients. In our case it is 1

 

configuring RTMP load testing

 

The parameters for PushRequestTest are the following:

  • IP, port, app, stream similar to the ones in Red5 paragraph
  • Video - the path to the video that will be streamed

 

Option 3: Other libraries

 

Flazr and Red5 are the most commonly used open-source libraries, but there are others too:

  • JUV RTMP library is a commercial library
  • srs-librtmp library supports only audio, but it is a lightweight library

 

Option 4: Process OS Sampler

 

Another way to load test RTMP is to use ready-made console applications:

 

You can use these tools from the console without JMeter, if they suit you, but if you test a use case that includes RTMP streaming, you cannot start them from the console, because it will not be connected to other actions from the use case. Therefore, the script for the scenario will be incorrect.

 

To add this call to the test plan, you can use the OS Process Sampler, which can be used to call 3rd party tools from JMeter. You can see work of the Sampler and the description of the interface here.

 

For example, the screenshot below shows the sampler with the command to subscribe from the Flazr tool.

 

rtmp performance testing guide

 

The name of the executed file is in the Command field, the path to the file is in the Working directory and the Command parameters are options for the executed file. The Response Tab will show the entire log of the app, so you can use assertions and extractors, if necessary.

 

Read more about how to load test live streaming with JMeter from these blog posts:

 

Our free JMeter academy provides an advanced course where you can learn even more ways to create complex performance testing scenarios. Sign up here.

 

Running Your Performance Tests with BlazeMeter

 

After creating your RTMP testing script in any of the four ways we’ve shown here, you can then run it in JMeter. But, if you need to scale your test, run it in BlazeMeter. All you need to do is upload your file, configure the load and the global locations your test is running from, and run.

 

You will immediately get insightful results you can share with your managers and teammates. By the way, you can also collaborate on your tests with them! The results can be viewed in real-time, or you can store the data and look at trends over time.

 

rtmp load testing with blazemeter

 

To learn more, request a demo, or put your URL in the box below and your test will start in minutes.

     
arrow Please enter a valid URL

Interested in writing for our Blog? Send us a pitch!