Epic Battle: Concurrent Map Reduce vs Concurrent Suitelet

Epic Battle: Concurrent Map Reduce vs Concurrent Suitelet

Comments 168 comments
Spoiler: 50 concurrent Suitelet win. (Tested in Sandbox)

I am a big fan of Map Reduce scripts, spreading the load among 5 queues makes a huge difference when processing your data. Unfortunately to take advantage of the concurrent queues feature your account must have a SuiteCloud plus license.

So I figured it would be cool to simulate this feature with Suitelets. Suitelets support up to 50 concurrent connections and they are available without the need of SuiteCloud plus.

The results were very surprising.

Test scenario:

  • Task: Create and delete a custom record with no fields.
  • Account: Account with SuiteCloud Plus license in Sandbox.
  • Map Reduce: Run task 10k times by returning an array of 10k elements on the getInputData step and let the reduce step handle the main logic.
  • Suitelet SS2.0: Run task once per call. Return “OK” as response. Available without login (External).
  • Use Apache JMeter to call the Suitelet 10k times.
  • Additionally call the Suitelet using Ajax with browser (Chrome) 5k times (to avoid browser memory issues).

Here are the results:

testresults

 

Map Reduce w/buffer size of 1 : 196 seconds ~ 51 per second
Map Reduce w/buffer size of 64: 518 seconds ~ 19 per second
Suitelet w/JMeter w/50 at a time: 55 seconds ~ 181 per second
Suitelet w/JMeter w/100 at a time: 30 seconds ~ 333 per second
Suitelet w/JMeter w/200 at a time: 24 seconds ~ 416 per second
Suitelet w/JMeter w/250 at a time: 24 seconds ~ 416 per second
Suitelet w/JQuery: 1000/44 secs ~ 22 per second

As you can see, I ran multiple test variations. After I saw that the Suitelet was able to handle 50 requests at a time I went crazy and increased the number of concurrent requests. I got up to 250 requests being handled successfully. Once I reached 300 requests at a time I started getting errors and not all the records were successfully created.

As you can see, after 200 requests you pretty much get the same speed. This could be caused by my system, JMeter, my internet connection or by some throttling happening on the Netsuite side, I don’t know.

Conclusion:

This approach can be pretty handy if you ever need to process large quantities of records and you need even more speed than what Map Reduce gives you. You would need to develop your Suitelet in a way that even if something goes wrong you have a way to identify the errors and retry the operations. Additionally, you would need to get a tool that bypasses the browser restrictions such as Apache JMeter and configure it if you need to pass any data to the Suitelet.

Anyways, the difference is huge. Basically you could process 1 million records in 40 mins in the Suitelet vs 5:30 hrs in a Map Reduce… and this is in Sandbox, go figure how faster this would be in Production.