Aftrer my previous post about high-performance RoR deployment methods, I’ve got lots of messages/emails/IM conversations about some errors in previous benchmarks and there were lots of suggestions about extending of tested software set and modifying testing methodology. So, that is why I decided to perform deep and wide performance testing for all deployment schemes I can find. In this article you can find description of testing methodology and, of course, benchmarks results. If you want to know details of specific software setup, take a look at articles in category “Ruby On Rails” to find all articles from “Looking for optimal Solution” series (I’ll post all of them in next few days).
Truly speaking, when I’ve started all these testing stuff, my main aim was to find most optimal solution for our project, but later, thanks to Dmytro Shteflyuk, I decided to perform testing with optimized software settings, to find maximum theoretical speed, that I can expect from RoR on my servers. So, you can interpret results of testing from two different points of view: as simple performance comparison of different solutions, or as highest performance marks that specific solutions can provide you (of course, you can get more by creating some more complicated schemes like “nginx/lighttpd for static content + rails for dymanic”, but this benchmark was done only for dynamic content without any caching).
First of all I want to describe hardware/software plaftorm where all performance benchmarks were done. Our development server has following configuration:
All tests were performed on simple RoR appliaction with simple single-action controller:
class TestController < ApplicationControllerdef hw@hello = "Hello, world!"@time = Time.now()endend
and simple view:
<h1>Test#hw</h1><p>Hello: <%= @hello %></p><p>Time: <%= @time %></p>
Rails Framwork was started with default settings in production mode. Tests were performed by ab (Apache Benchmark) utility with following params:
$ ab -c 100 -n 10000 http://SERVERIP:PORT/test/hw
where PORT is specific port number that has was chosen for every test.
I decided to not use any DB-related code because I want to get top performance marks for Ruby On Rails engine, not for some mysql/postgres/oracle/etc software. Simple code, simple tests, simple and understandable results.
Notice: Before each tests all files from tmp/sessions were deleted because with lots of test requests to server I’ve got very poor results for some software because of file-system layer lags. So, if you’ll decide to check my tests on your own hardware/software, clean sessions dir after every test run.
While tests were running I’ve monitored server with top/iostat/vmstat to understand test results better… and that is why all fastcgi/proxy/lsapi tests were done with 4 backend processes - server has 4 CPUs and 4 processes performed better than 2/5/8/10.
So, let me describe tested configurations shortly before I will show you benchmark results (links will be pointed to detailed description of tests later).
Now, when you know about all of my tests, I can show you information about tests results. First of all, if you want to see results in table view, here is screenshot from my Excel spreadsheet:
If you like to analyze results in graphical presentation, you can take a look at the following diagram, that shows QPS (queries per second) results for all tests:
Speaking about these results, I can say, that all of them were predictable - TCP is really slower that unix sockets, that were used in fastcgi tests. So all tests with tcp-based backend commmunications have less QPS, than unix socket based. But you should understand, that tcp-based frontend-backend communication provides you with greay scalability mechanism and you can move backend instances between servers without any problems. So, if you need absolutely best performance on one server, I would recommend you to use nginx with fastcgi backend processes. But if you need better scalability, you can use nginx with mongrel-servers as backend processes.
