Load Testing Asterisk Applications - HOWTO

I've been working on Load Testing some of our asterisk applications and thought I'd document the process here so I can find it when I need to do it again.

There is a bit of setup to be done but when you get the whole picture is it's actually very easy. To do it you need at least 2 asterisk boxes - the one you are load testing (lets call it 'stressbox' for this example) and a another one to drive load to it which we'll call 'driver'.

We need to do the following steps:

  • Create an sip account on stressbox for driver to connect to
  • Register the account from the driver machine
  • Create a dial plan which simulates a call
  • Create a call file for the call
  • Start you system monitoring
  • Create lots of call files

So here is the step by step:

Create an sip account on stressbox for driver to connect to

In the sip.conf file on stressbox (the one we are testing) add the following:

[loadtest]
type=friend
username=loadtest
secret=1111
context=test
host=dynamic
Do a 'sip reload' in the asterisk console to register this change.

Register the account from the driver machine

Then to register the account from the driver machine add the following to sip.conf on driver:
register => loadtest:1111@xxx.xxx.xxx.xxx
Where xxx.xxx.xxx.xxx is the ip address of the stressbox server.

Your driver machine should now be able to make calls to the stressbox machine and you should see a line that looks like this if you were watching the console:

-- Registered SIP 'loadtest' at yyy.yyy.yyy.yyy port 9350 expires 120

Create a dial plan which simulates a call

Then on the driver machine we need to create a dial plan which simulates the call we want to do and place it in extensions.conf. Here's an example dial plan:

[dialplan-loadtest1]
exten => s,1,Answer
;wait for welcome to play
exten => s,n,Wait(10)
;enter username
exten => s,n,SendDTMF(1234)
exten => s,n,Wait(4)
;enter password
exten => s,n,SendDTMF(0000)
;listen to menu
exten => s,n,Wait(10)
;select menu option 1
exten => s,n,SendDTMF(1)
exten => s,n,Wait(3)
;select menu option 2
exten => s,n,SendDTMF(2)
exten => s,n,Wait(4)

;Play some audio
exten => s,n,Playback(sampleaudio1)
;Hangup
exten => s,n,Hangup
This example does a login to a phone system and then selects some options and plays some audio and hangs up. The dial plan can be as simple or as complex as you like.

Create a call file for the call

Last config step is to create a call file which will tell asterisk on driver to call asterisk on stressbox. Create a file test.call with the following:
Channel: SIP/6@xxx.xxx.xxx.xxx
MaxRetries: 1
RetryTime: 5
WaitTime: 10
Context: dialplan-loadtest1
Extension: s
Priority: 1

A bit of explanation of these lines: Channel: SIP/6@xxx.xxx.xxx.xxx - the 6 is the extension to dial on the stressbox server - change as necessary. The ip adress should be the stressbox one. Context: dialplan-loadtest1 - this is the dial plan that will be run for this call.

Start you system monitoring

If you are going to do some load testing then you probably want to monitor the load on the machine - it is recommended to record the load of all machines involved - i.e. stressbox and driver to ensure that driver is not the bottleneck in the load test.

For this I use a vmstat on linux via a little script I wrote which makes it easy to collect stats from multiple machines

It would be used as follows:

./multivmstat stressbox,driver 5
This will connect via ssh to each box and run vmstat on them every five seconds. This can be graphed later - I'll post the scripts for this shortly.

Create lots of call files

You are not ready to test you system. To do this you need to create copies the test.call file and move them into /var/spool/asterisk/outgoing on the driver machine.

I use a little script to do this:

#!/bin/sh
#usage: makecalls callfile count delaybetween

callfile=$1
count=$2
delaybetween=$3

callnumber=0
asteriskcalldir=/var/spool/asterisk/outgoing/
asteriskuser=asterisk

echo "Using $callfile for $count iterations with a $delaybetween second delay"

while [ $callnumber -lt $count ];
do
   callnumber=`expr $callnumber + 1`
   echo "Call Number: $callnumber"
   cp $callfile $callfile.tmp
   chown $asteriskuser:$asteriskuser $callfile.tmp
   mv $callfile.tmp $asteriskcalldir
   sleep $delaybetween
done

This is used as follows:

sudo ./makecalls test.call 5 2

This will make 5 calls with a 2 second delay between the start of each call.

Note: It needs to be run as sudo so it can change the ownership correctly.

Credits: Thanks to Stuart Elvish for his help with getting this working so nicely.

Happy load testing. Cheers, Mark

Comments
Hi Mark,

That looks really interesting, nice to see its possible to load test the system.

How are the tests going, does the Asterisk system strain under the pressure of lots of calls? I always was afraid that the playback of all the audio to many calls wouldn't be too good for performance.

Is there a way of somehow 'listening' on the loadtest machine? I'm thinking of the situation where a call is in a queue and may need to wait an arbitrary length of time to get dealt with and then may have to enter more dtmfs to continue.
# Posted By Eddie Long | 1/6/09 10:24 PM
Hi Eddie,

Performance tests are going well - and it's always amazing what you find when a system is used concurrently versus testing it as a single developer.

Re listening in to it - I typically dial in from a softphone at the same time to check the audio quality good and not getting broken up - which is what happens when there is too much load. So I'm not listening into the actual calls, but can hear how the system is performing overall.

Re knowing that the dialplans are completing successfully - at the moment the dial plans have a know length so I check the CDR records and can see the calls that have dropped out or gone wrong because the times don't match up.

Cheers,
Mark
# Posted By Mark Lynch | 1/6/09 10:31 PM
BlogCFC was created by Raymond Camden. This blog is running version 5.1.004.