Proven high-quality software development
June 25, 2010
In June 2010, a research team at SoftServe conducted an evaluation of 4 Web CMS’s: Drupal, WordPress, Joomla! and Radiant.
The evaluation was performed against average web projects (20-30 pages for RIA and Sharing Portal-like content-driven web sites) such as: Technical Feasibility, Schedule Feasibility and Business Impact. Based on multiple overlapping business requirements from the clients, project communication and CMS analytics, we built a feature list, which allowed us to apply trim filtering to reduce the final list to four Web CMS’s. Technology was an important requirement and we choice not to include Java and.NET, because of technology trending in the mentioned time period.
The team selected the above-mentioned four Web CMS’s, from multiple lists reported by various analytical open source CMS related resources including: Forrester, Gartner, CMS Wire, CMS Report and CMS Watch. Additionally, we analyzed them in context with our service offerings, tailored to our clients’ needs in Q3/Q4 in 2009 and Q1 in 2010.
There are additional keys to open source Web CMS selection criteria: CMS Learnability, Maturity and Usability. Use the following suggestions in your approach to the open-source Web CMS selection:
You can’t avoid the challenge of a CMS selection because of the great number of open source products available in the market, which is still growing. If you prefer a commercial CMS, then the vendor is usually responsible for it; however when you select to use an open source product, the responsibility lies with you or your project team. We propose the following tips on how to strengthen and solidify the decision-making process.
The impact of those aspects to your business (Budget, Time-to-Market, Quality and Manageability & Support) is represented in the next paragraph.
The Business Impact was considered in the terms of the relations between Learnability, Maturity and Usability on the Budget, Time-to-Market, Quality, and Manageability & Support.
| Impact | Learnability (1-weak, 4-strong) |
Maturity (1-weak, 4-strong) |
Usability (1-weak, 4-strong) |
| Budget | 2 | 4 | 3 |
| Time-to-Market | 2 | 4 | 3 |
| Quality | 4 | 4 | 4 |
| Manageability & Support | 4 | 4 | 4 |
Learnability, Maturity and Usability evaluation scores of all Web CMS’s (Drupal, WordPress, Joomla!, Radiant) are compiled into the table with detailed description provided below.
In addition to the technical and schedule feasibility study, we did an evaluation on Learnability, Usability, and Maturity of the 4 Web CMS’s. The table below represents the final scores; a detailed meaning of every aspect is available later in the article.
| Web CMS | Learnability (1-poor, 4-excellent) |
Maturity (1-poor, 4-excellent) |
Usability (1-poor, 4-excellent) |
| Radiant | 3 | 2 | 2 |
| Joomla! | 4 | 3 | 3 |
| Drupal | 3 | 4 | 3 |
| WordPress | 4 | 4 | 3 |
Learnability is the capability of a CMS to enable the user to learn how to use it. Learnability may also be considered as an aspect of usability, and is a major concern in the design of complex software applications.
Within Maturity we merged Feature Feasibility, Extensibility, Security, Quality, Market Share, and Community Support.
We included Usability criteria as an important attribute if you select a new CMS to obtain a product which is to be quickly and easily adopted, or in situations with large number of general business staff. Some Web CMS systems are clearly more usable than others, even when they offer equivalent functionality.
Note: The evaluation approach included an assessment of Learnability, Maturity and Usability of each Web CMS during implementation of Technical Feasibility and Schedule Feasibility Studies. The scores of the qualitative evaluation were compiled from subjective feedback from developers, obtained via a survey and from project leaders responsible for the feasibility study (see results in table below).
| Technical Feasibility | Schedule Feasibility | |||
| Web CMS | Feature Coverage* (0-absent, 4-rich) |
Extensibility (1-bad, 5-excellent) |
Learning Curve | Development Velocity |
| Radiant | 2 | 3 | Medium | Slow |
| Joomla! | 3 | 3 | Fast | Medium |
| Drupal | 3 | 4 | Medium | Medium |
| WordPress | 3 | 4 | Fast | Fast |
* scoring rules:
0 – no such feature "out of the box"
1 – available as an extension of languages and library, requires more manual development efforts
2, 3 – fair feature, express extension or as a good add-on and easy to customize
4 – 100% coverage requirements
