Server Issues which can affect Search Engine Rankings by Rob
Sullivan
I was recently helping someone troubleshoot some problems at SearchEngineWatch
forums and it struck me that I should write an article about this and other
issues. Because sometimes when you run into a brick wall and can not figure out
why your site is not getting listed or even crawled in the search engines it is
not always a site issue.
In fact, sometimes it is a server issue which you may or may not have control
over. In that case, it does not matter what you do to your site, you just
aren't going to get it listed. This article covers some things to look at and
how you can go about fixing them.
Let me start by giving you an idea of what happened with the person on the
forum. His site had been up for some time, in fact he recently put up 3 sites.
Two of which were getting crawled and indexed and one wasn't showing up in the
engines.
We started by looking at the usual suspects - such as improperly coded
robots.txt, poor navigation, banned domain and so on, yet none of these were an
issue. The site was very search engine friendly, easily navigable and didn't
use a robots.txt file.
So the next thing I checked was the WHOIS information and on one site I found
improper information and on another WHOIS lookup I found no information at all.
This lead me to believe that perhaps there was a hosting/server problem at the
root of the issue.
So I contacted the site owner and got him to verify with the web host that the
DNS information was properly configured. It turns out that it wasn't. Since the
site wasn't properly resolving via DNS it wasn't getting crawled by the search
engines.
This is just one of the many server issues which can affect the positioning and
ranking of your site in the search engines. While DNS resolution and IP
configuration are generally the two most common culprits there are several
other server issues which can harm your search engine ranking and even cause
indexing problems.
DNS issues
Let's take a closer look at what happened with this site: What I found is that
I could connect to the domain name through my browser. Further, the owner had
enough foresight to register his site as the only site on the IP, so I was also
able to connect to the site via IP in the browser.
I could also ping both the site and IP and receive proper results, indicating
that at least at some level there was DNS resolution happening.
However, when I went to perform the WHOIS lookup at the different site I wasn't
seen the results I should have, which means the DNS for the site hadn't been
properly configured.
I like to use whois.webhosting.info to perform these lookups as you can also
perform IP searches to see if the site resolves properly to the IP. Further,
you can also see how many and what types of sites may also be hosted on a
domain. This can give an indication of other issues which could cause a site to
not be indexed.
For example, had I performed a WHOIS lookup and found a bunch of porn or casino
sites, I could have assumed (probably quite rightly) that Google had greylisted
or blacklisted that IP address because of the sites hosted on it. Then, any
legitimate site would also suffer the flag, causing indexing problems by the
engine.
But that wasn't the case here. As I said, it was a single site on a unique IP
address.
I gave the information about the DNS to the site owner, indicating my concerns
that the site wasn't properly resolving and I found out a few days later that
one of the hosts name servers was indeed acting up. I don't know what "acting
up" means, but it could indicate that the resolver wasn't always resolving the
IP to the site for some visitors, or it was improperly resolving it. In either
case, should a search engine come across such inconsistency it will, over time,
quit coming to the site.
More issues which could affect rank
There are other issues, not DNS related, but server related which could affect
your indexing and ranking possibilities. These include improperly configured
servers, or overloaded servers among other things.
Improperly configured servers
As you read above, there was an improperly configured (or failing) name server
causing problems with the site, but there are other server issues which could
affect the site as well.
Once the site is resolved via a name server, the request is then forwarded to
the webserver hosting the site. If the webserver isn't properly configured
however to accept the request, then it will deny the request to the site, even
if the site exists on the server.
While this is a rare occurrence, it does happen. If the webmaster mistypes the
domain name for example, or attaches the site to the wrong IP attached to a
different server, this could cause issues.
Another example of an improperly configured server is one which denies requests
to domains without the "www." Your webserver should be configured to accept all
requests for a domain whether they have the www or not.
In fact, the ideal situation is for the server to automatically issue a "301"
server site redirect to any request made for a non-www domain which then
forwards users to the www domain. This is because sometimes search engines will
see the www and non-www domains as 2 separate sites. Therefore link popularity
and rankings could be split among the domains. In a worst case scenario, one or
both of these domains could be banned by the search engines for duplicate
content.
In addition, if you are fortunate enough to have your own IP address, you
should check with your web host to ensure that the IP address resolves to the
site. This way if a user (or search engine) tries to connect to the site via IP
they will be served the site properly, and not receive a "page not found"
error.
Alias issues
As mentioned above, it is best to have an non-www requests 301 redirected to
the www domain. However, for ease of use (and because its quicker and easier to
configure) most web hosts will allow all requests to a site to return that
site, so that any request to the www, or non-www domain will return the same
site. As I eluded to earlier, this could get the site penalized or banned. At
the very least, the site won't properly propagate link popularity, which will
ultimately affect its ability to rank competitively in the search engines.
With some servers, the only way to combat this is to set up a unique site for
each alias and have the non-www alias 301 to the www domain. Sometimes that
requires the appearance of 2 sites per domain on the web server. While most web
hosts won't agree to this, perhaps they can help you come up with a solution
that will work for you.
Overloaded servers
Sometimes, when you perform a WHOIS lookup you will find hundreds and hundreds
of sites hosted on a single IP.
While in many cases this isn't an issue, you can still run into problems in the
search engines.
This is because, to save time, many engines will cache the IP of a domain and
simply request the domain by IP rather than a full DNS request each time it
wishes to index the site.
That means, every other site hosted on that IP has the potential to negatively
impact your search engine rankings.
This is because every site which is associated with that IP will by default be
associated with your site, because of the common IP. If one of the sites has
done something to spam the engine in the past, there is a chance that your site
could suffer the consequences.
Go back to the example I gave earlier - a legitimate site sharing an IP with
porn or gambling sites. I can almost guarantee you that the legitimate site
will not rank competitively.
Consider it guilt by association. In this case the association is the sharing
of an IP address.
While search engines are getting much smarter at identifying these types of
things, some issues still arise. Therefore it's safer to take precautions now.
If you do find such a site on your shared IP, you could always request that
your site be moved to a new IP so that it doesn't suffer the consequences of a
penalized IP.
Another problem with overloaded hosts is that sometimes there is a site (or a
few sites) which use so much of a servers Subscriptions that other sites suffer. If
this is the case, then your site may not respond in a timely manner which means
search engine crawlers could stop coming to the site and it will fall from the
index.
Again, the solution here is to move to a different webserver or change hosting
providers altogether.
As you may see, there could be many issues outside of the site itself which can
affect search engine rankings.
The best way you can identify these issues is to become a web detective and
perform a little searching of your own using WHOIS servers and PING commands to
test out your site to make sure it resolves properly to the web host and IP it
is supposed to.
Many times when you've ensured that there isn't an issue on the site itself,
then its likely a hosting issue.
And in some cases, you may be able to get these resolved quickly while others
take a little longer. If you find its taking too long to get hosting issues
solved, might I suggest shopping around for a new host? One that will treat you
like a customer and not a number or pay check? That way you can ensure that
your site receives the attention it deserves, and problems are dealt with
quickly
About the Author
Rob Sullivan - SEO Specialist and Internet Marketing Consultant. Any
reproduction of this article needs to have an html link pointing to
http://www.textlinkbrokers.com
|
