GATHERING THE DATA & TEST WORKFLOW
Gathering the data and inserting it into a spreadsheet was done manually which was a VERY time consuming task. This consisted of enabling the MySQL debug block in the preferences area of the Xoops admin panel. Then each time I ran the test I copied the entire execution time and MySQL query count and pasted it into the relevant spot in an Excel spreadsheet I had prepared. As part of the test preparation I established that with no blocks or banners displayed that the MySQL query count overhead imposed by the Xoops system was 6. I placed a formula in my spreadsheet that would then subtract 6 queries from the actual entered value from the MySQL debug block so as to obtain the actual module query count.
This is how each test workflow went for each server (one server at a time);
Run test script to insert desired amount of topics and articles per topic, relevant to current test.
Reboot computer to ensure clean environment.
Open browser on computer where I was running the tests from.
Open the index page of each module on server currently being tested. Copied data to spreadsheet. Repeated entire step 3 times.
Open the topic (number 1 always selected) page of each module on server currently being tested. Copied data to spreadsheet. Repeated entire step 3 times.
Open the article (number 1 always selected) page of each module on server currently being tested. Copied data to spreadsheet. Repeated entire step 3 times.
Open the admin index page of each module on server currently being tested. Copied data to spreadsheet. Repeated entire step 3 times.
Progress to next test, repeating entire process each time.
CONSIDERATIONS & LIMITATIONS
As always, no testing process is ever 100% perfect, and this one is no exception. There are always some elements you cannot control, and always some things that don't always represent a real world environment. Below is a list of considerations and limitations of this test that I can think of. If you can think of any others, please let me know and I'll add them.
All tests were run using one session representing only one user on the server. In the case of real world environments the execution times can be quite a bit worse than what is displayed in this analysis. In cases where there are multiple users it is not as simple as multiplying X users by the execution time shown in this analysis as other factors to do with server config will come into play.
The MySQL server database for Xoops was hosted on the same physical machine as the Xoops website in all cases. In real world environments where they are hosted separately, execution time will be negatively effected. As the MySQL query count rises, so will the execution time due to the lag times introduced from have to establish and transmit data over a network connection.
As all modules were tested with the default installation settings, there will be additional performance gains possible through tweaking the module settings. Likewise, enabling options that aren't enabled by default are likely to have an adverse effect on site performance.
The script execution time was increased to 1 hour for each of these servers to prevent script timeouts. The default setting is 30 seconds, but any script that is used on a regular basis that is taking more than 5 seconds is really unusable in a production environment.
All modules where set to run with no caching enabled to ensure accurate MySQL query count. With module caching enabled all of the tested modules would experience notable performance increases.
Even though the exact same procedure was followed for each view, or each module, of each server, there are still some unexpected spikes and dips in the page execution times. These were minimised by taking an average of three trials. You can see these discrepancies when comparing the same graphs from different servers.