Performance Analysis Of 4 Xoops Article Modules

Categories:


CONCLUSIONS

One comforting aspect of drawing conclusions from a performance analysis compared to something like a feature set analysis is that the outcome is an objective one. It is completely free of personal preferences, likes and dislikes, or any other such subjective influences. The proof is right there in the numbers for all to see, and barring any mathematical errors everyone should be able to draw very similar conclusions. Perhaps the only difference being that people will have different views on what are acceptable performance levels. In the next few paragraphs I'll briefly discuss the conclusions I have drawn from this data, ending with an overall summary. If I have done my job right, then you should be drawing very similar conclusions of your own.

AMS

Of the modules analysed in this article, AMS is clearly the most efficient module of the lot overall as far as MySQL queries go, and is the only module where the number of queries does not increase through the tests as the content grows. This is of particular importance in cases where the Xoops database is hosted on a separate server to your Xoops website as network lag time is minimised due to the webserver having to establish far fewer network connections to the database.

However, even though AMS is able to pull all the required data from the database in far fewer queries, it still has to process this information just like the other modules. This is where the influence of of the scripts are structured programically have a huge influence over page execution times. Here again AMS proves to be right up there among the best in the field, and has either the 1st or second best overall averages in all the test apart from one (where it comes third).

Articles 0.17

The Articles module is also very efficient in regards to minimising MySQL queries, and is actually at the head of the class in all but one category were it lets itself down quite badly. Unfortunately this one category is the index view which is the most important view to have right as it receives the highest amount of traffic. If it wasn't for this one flaw the Articles module would've actually taken the crown from AMS as the most efficient module on MySQL queries.

The page execution times of the Articles module is also very badly effected by the index view. In fact, the executions times of the index view are so bad that this issue alone rules out the Articles module from being a serious contender for holding anything more than a few hundred articles. The fact that the Articles module has the 3rd lowest MySQL query count in the index view, yet has by far the most expensive page execution times would suggest that this delay might be a result of the data being processed inefficiently.

News 1.2.1

in regards to the MySQL query count, the News module doesn't set any benchmarks in any of the views apart from the admin index view where it actually doesn't produce any queries at all. In the topic view the News module query count climbs at a steady rate, and by the end of the tests is producing nearly 10 times the amount of queries that the Articles and AMS modules are.

On the other hand, page execution times is where it all really counts at the end of the day, and it is also where the News module really shines. The news module has the lowest overall page execution time averages on more than one occasion, and where it doesn't come first, it is only very very marginally behind the best. However, the high query count produced in the topic view may adversely effect performance in scenarios where the database and webserver are hosted on different servers.

WF-Section 2.07 Beta

Put simply, WF-Section is an absolute query monster. It has the highest query count in all but one view by quite a margin. The index view query count climbs at rate that is triple the next worst module through the tests, and ends up with a very unworkable query count of 1111 by test 10. This is not the worst of it though, the index admin view really really needs some work as it climbs to well over 20,000 MySQL queries by test 10. These two factors alone rule out WF-Section as a serious contender for hosting any sort large article repositories.

The rate at which the page execution times climb with the index and the admin index views in WF-Section is a good reflection of it's inefficient use of MySQL queries. One interesting point though is that even though WF-Section has be far the most queries in the index view, the page execution times are a lot lower than the Articles module, and also drop at a much faster rate as the power of the server is increased. This indicates that WF-Section is quite efficient in the way it processes the data it pulls from the database. This is not enough to save it in regards to hosting large article repositories though.

Overall Conclusions & Closing Statement

Based on the figures in the tests, and purely on a performance & scalability basis, the best module out of the four analysed is the News 1.2.1 module. However the margin between it and the AMS module is so tight that these figures could almost swing either way depending on which way the wind was blowing on the day. It is so close that there will be absolutely no noticeable performance difference between the two when used them on any typical x86 hardware used over the past 5 years, and also regardless of wether you have 10, or 10,000 articles. One thing that might go in favour of AMS is in situations where the webserver and database are on physically different machines. The overhead of establishing the extra MySQL queries over the network would more than likely swing the performance & scalability crown in the favour of AMS when hosting larger article repositories.

The Articles and WF-Section modules on the other hand are not suited for large article repositories at all. The page execution times for the index view of both modules renders them unfit for hosting article repositories anything much larger than 400~500 articles. This issue is even further emphasised in situations where you have a high traffic site and/or are using shared hosting services. The server resource usage generated by these two modules would be enough to be a serious concern in both these situations. WF-Section has a double whammy here as the admin index view is several times worse than the index view. Even though this view will receive far less traffic than the index view, it is still an issue of concern for those looking for a high performance & scalable module.

Finally, in closing I'd like to say that all these modules have their own merits, and you should not choose any module based on one set of criteria alone. Be this performance & scalability, or design or feature set. Balance in you selection process is the key, and clearly defining what areas are of most importance to your site before you start you evaluation process is going to help a great deal.

Comments

Anonymous's picture

Re: Performance Analysis Of 4 Xoops Article Modules

I think in fairness to you article I would like to show some screenshots of the most recent
version.

WF-Section Screenshots

As you can see from the shots, we have not only brought down the amount of query used for each section (Please note that not all the queries are used by WF-Section) but this amount will not change with the amount of articles with the database.

You may also note the amount of time it takes to render all the pages shown, you will see that this will take a load of the server compared to the beta version that you are using in the tests.

Scott

Brashquido's picture

Re: Performance Analysis Of 4 Xoops Article Modules

Nice one Scott, looking forward to seeing the new version and updating the article with the new results :-) .

Anonymous's picture

excellent

good work

Anonymous's picture

Excellent Article

It'd be great to see these tests repeated on a new install of xoops 2.3.1 (latest release) and with the latest versions of the modules.