How?

Instead of relying of a small sample of reviewers who might not have your best interests at heart, at hosting scorecard.com we rely everyone who has ever expressed an opinion of the hosting company in question.

The internet is full of people with opinions.  Every day tens of thousands of people with no financial or other incentive will publish their opinion. They do it on blogs, they do it on forums they do it on mailing lists and newsgroups.  Sometimes they do it in response to a question; sometimes they do it because they are very happy or unhappy with their current host.  Regardless of where or why these are real people with real opinions.  These are the people that you want to ask, all of them, not just a few.

That’s what we do at hosting scorecard.  We look at everyone opinion not just a few.

How does it work?

Have you ever been to the website sucks-rocks.com or one of its variations?  The idea is that you query a search engine for the phrases "SomeCompany sucks" and "SomeCompany rocks" and use the number of results for each as an indicator of the online reputation of SomeCompany.

We have taken that idea and expanded on it, we look at 18 different keywords and how they are used in association with different hosting companies.  We also break the searches down into 1 week intervals so we can see the trend from week to week.

The big question has to be “Does it work?”

Let the data speak for itself.

Below is one of our hosting graphs.  This graph is a summary of 8900 different opinions that have been expressed over a 6 month period. This graph is specific to one hosting company, the searches used to create it contained the full domain name of that host. Each blue cross is the score for a particular week and the thick blue line is the trend for the last 6 months. The thin blue lines represent how consistent the weekly scores and are drawn 1 standard deviation on ether side of the main trend line. The red arrow is the final score for this host, in this case about 94. The vertical bars at the bottom represent volume.

One thing you would notice is the weekly scores are fairly consistent, most of them occur close to the dark blue trend line.  
graph

Now let’s look at another graph.  This graph is a summary of 5200 opinions of a different hosting company that have been expressed over the same 6 month period.  The individual scores are again in a fairly narrow range round the trend line, but the overall score is much different, its about 66.

graph

There is only one difference in the way data was collected for both these graphs.  The searches used to create the data for the graph each contains the domain name of a different hosting company.  The data was collected at the same time by the same methods.  Because the only difference in the data collection is the host name, then any changes in the data is because of the host name and not some other influence.   

Consistency is also important; the fact that there is not a wide swing in the weekly results would indicate that the results represent the real reputation of the host and not just a random fluctuation within the data.

We can therefore conclude that the first hosting company has a much better online reputation than the second one does.

Conditions for inclusion in the list.

In order to be objective it is also necessary to disclose the criterion used to determined which hosts are included on the list, its easy to exclude hosts to create a list that is just as biased as one that ranks hosts by affiliate payout.

When we started this project the first task was to create a large list of hosts.  We ended up with 10,000+ hosts.  We collected preliminary data on all of them.  We pared the list down to about 1200 based solely on volume.  An amazing number of those 10,000 had virtually no online presence and could easily be eliminated.  For remaining 1200 we collect data weekly.  The final list that comprises this site is extracted from those 1200 based on the following rules:

1. Must be in English
The primary language of the site must be in English. An English speaking reader is probably not interested in German or Spanish hosts if they cannot read the language.  We are currently limiting our list to US, Canadian and UK based hosts.

2. Must be a Host.
The hosts included on this list are primarily web hosts, not sites that happen to do hosting in addition to something else.  For example godaddy has a very large hosting business but they are domain registrars first and therefore are excluded from the list. See Note 1

3. Volume.
The system for ranking hosts only works when there is a reasonable number of opinions per week.  If there are not enough hits per week then the results are essentially random and cannot be trusted. Therefore we include any sites that averages 50 or more hits that render an opinion and at least 75 total references a week.

4. Score.
Ultimately the purpose is to determine which host to chose, nobody want to pick a host that is below average, therefore we filter out all hosts below our minimum score. That score is currently 75.

5. Consistency.
Consistency is important.  Some hosts can meet the above 2 conditions and yet have wildly changing score from week to week.  The average may be fairly consistent but weekly scores are all over the map.  If the weekly scores change too much then the overall score cannot be trusted. This condition is related to the volume condition since it is mostly low volume sites that are affected.

These 5 rules are somewhat arbitrary, however they are consistent. Every host that passes those conditions will be included on the list and no hosts that fail any of the tests are included.  From time to time we might tweak the threshold values but we will always apply the rules consistently.

Note 1.
The reason for the exclusion these kinds of sites are that when we do a search we are unable to clearly distinguish between the primary business and the hosting portion of the business.  We would much rather

Note 2.
Score is calculated as follows:
100 X ( (Good References)  / (Good References + Bad References) ) - Standard Deviation.

Note 3.
We measure consistency by calculation the standard deviation of our data, and excluding any host that has a SD of greater than 12.  This measure only eliminates a few host from the list but these are the hosts that tend to have wildly varying data from week to week.
 MENU  Linux Hosting  Windows Hosting  Either Hosting  Other Hosting  Other Info