Back in March, I did an experiment on whether Google indexes content inserted into your page with JavaScript. Weeks later, the results are conclusive... no. The words inserted into the test page via JavaScript were never indexed. Lots of pages talking about the odd words I used come up in searches for them, but the page I created doesn't come up in the Google results.

I recently saw a site advertising on Craigslist for SEO help. Their pages were almost 100% JavaScript and that meant that while Google indexed the page and saw their meta tags, almost all of their actual content didn't count. I did a "site:domain.com" search for them, found a page with what looked to be a unique phrase, then searched for the phrase. Nada, zilch, zero. Just more proof of my point.

"But, Web 2.0 is so cool," you say. "I can't remove all the neat JavaScripty geegaws and doodads from my pages. They'll be boring and flat." Well, you don't have to remove them. But you do need to do a little extra planning when creating your pages.

Basically, you need to ask yourself one question: "How would I display this page to someone who didn't have a JavaScript capable browser?" It doesn't necessarily have to be pretty or even that well organized, but it needs to have all the content the JavaScript would have inserted, just as as plain HTML instead. Why? Because Google's spider can read that HTML and the content in that HTML has rich keywordy goodness that can drive trafffic. It's going to strengthen your positioning for your chosen keywords and maybe even get you visitors via search terms you didn't predict.

So, when you're creating your page templates, go ahead and create the "gee whizbang" Web 2.0 version, but also create a "dull and ugly" Web 1.0 version. Then generate the final page that's delivered to the visitor using some PHP or some server-side scripting language that can fill in either template.

The nice thing about Google, Yahoo, and MSN is that they identify themselves in the user agent string they send to your server. They don't try to disguise themselves like some more shady spiders do. All you have to do is have a piece of PHP at the beginning of your page that uses the following logic: "If the user-agent string is for a known search engine spider, use the Web 1.0 template, otherwise use the Web 2.0 template."

Between query strings and server-side scripting, just about any content you can deliver via your cool Web 2.0 application can be delivered via its Web 1.0 counterpart. If your single web 2.0 URL has two states, represent it with two Web 1.0 URLs (i.e. grumpy.php?state=foo and grumpy.php?state=bar) using query strings and make sure that when it's in 2.0 mode, it can spot those query strings and adjust its content accordingly.

If you don't want to go to that length, go with something even simpler. Ignore the user-agent string and just plug that content in between <noscript></noscript> tags on every page for every visitor. It's more bandwidth intensive and increases the time it takes your pages to load, but visitors with JavaScript enabled won't see it, and those without JavaScript won't be left in the dark, staring at a blank or mostly blank page.

The one caution on the <noscript> method is that I've seen people using it for black-hat SEO. They take their huge batch of spamdexing keywords, and instead of trying to hide them by shrinking them down really small and/or making them the same color as their page background, they put them between <noscript> tags. That way they're not visible to anyone with a JavaScript enabled browser unless they view the page source, but the search engine spiders see them. So be careful in how you use that method.

Maybe at some future point, the Web spiders will scan your javascript for keywords and content, or maybe even execute it and then index the interactive page in its various states. But for now, they don't, and if you want to get all that lovely free traffic the search engines send, you need to accomodate them by offering them versions of your pages they can read.

If you talk to a lot of SEO people, they'll tell you that the Google Page Rank is pretty much meaningless and is not something you should pay attention to. Still, people pay attention to page rank.

For some, page rank is a form of validation. The higher your page rank, the better a job you're doing. For others, it can actually be a component of their income stream, whether they're "selling links" or using it to qualify for some of the richer offers via services like PayPerPost (which I tried and ended up removing from this site... more on that another day).

But the reason the experts say to ignore page rank is that it quickly goes out of date. While Google's internal page rank is regularly updates (and isn't an integer, but a float), the page rank you see via the Google tool bar and various page rank checking sites is updated on average about 4 times a year.

The page rank you're seeing for this site is over 80 days old, last updated January 25th. The publicly visible page rank is the page rank Google displays through the Google tool bar. And Google changes those numbers in big "exports" that happen over a few days every few months. In the last couple of years, Google has had gaps between exports as long as 4 months and as short as 2 weeks.

So, with the last export occurring on January 25th, my page rank doesn't reflect all the items that influence page rank that have accrued since that date. Now, it's entirely possible it will remain right where it is, but it could also move.

And while I don't put that much value on it economically, especially since I abandoned my brief flirtation with PayPerPost, page rank is sort of validating, so I'd like to see where a couple of my newer sites sit just to tell me how well I've been doing. And I should know soon.

A Page Rank Update Is Imminent

While Google has gone as long as 122 days between exports, they average about 90 days and generally don't go more than 111. So with the last export happening 81 days ago, another one can most likely be expected within the next 30 days. As the last four have occurred on Thurdsdays, it could be as soon as sometime this week.

So keep your eyes on your page rank (via whatever methods you choose). If it's going to change, you'll see that change within the next few weeks (or sooner).

As part of my "Search Engine Optimization" for the clip art library on my art site, I've been exploring using a sitemap to help ensure that all my pages got indexed.

My main stumbling block is that I have a lot of pages generated from different databases, plus a number of static pages. I thought it would take a long time.

Imagine my relief when I read deeper on Google's page and found this quote. "This program does not replace our normal methods of crawling the web. Google still searches and indexes your sites the same way it has done in the past whether or not you use this program."

If you've got a lot of pages that are four or five clicks away from your home page, Google's less likely to find those pages and get around to crawling them. Their logic is that if they're that buried, they must not be that important. So those pages will be found more slowly and crawled less often.

What a sitemap does is sort of let those pages say "hey, here I am and I'm more important than it seems" to the Google spider. So for your pages that are one or two links away from the home page or get direct links from other sites, getting them in the sitemap may be of middling value, because the spider will find them easily and regularly anyway. But for the pages that live deeper in your directory structure, a sitemap may be a great tool for getting them a little daylight.

For me, that was a lot of relief. The scattered static pages, despite being harder to round up and put in a sitemap are generally easier for Google to find. The pages that are harder for Google to find are generally easier to generate a sitemap file for. So, if I want to generate a quick and dirty sitemap with the more buried content, I can do it without harming the SEO on the content I left out.

Nice.

I've been reading different articles about what elements of a page Google indexes with an eye toward whether they index content that's added to the page via the JavaScript document.write() method. Not getting a conclusive answer, I decided to do my own test.

Why was I interested? Well, with all the "Web 2.0" technologies that rely on JavaScript (in the form of AJAX) to populate a page with content, it's important to know how it's treated to determine if the content is searchable. If it's not searchable, then it's not having an impact on search-driven traffic.

The test page had three pairs of nonsense words that, at the time of its creation, generated no hits in a Google search. Two were placed in the page via straight HTML. Two were placed in the page via a JavaScript that was part of the document. Two were placed in the page via a JavaScript on a different server that was sourced from within the page (<script type="text/javascript" language="javascript" src="URL to script on other server">).

The page was linked from a sitewide footer to ensure that Google found it, and was posted and linked on the evening of March 7th. Google alerts were set up for one word from each pair so Google would notify me by e-mail when it spotted a page containing those words.

An alert came in in the late evening of March 10th for "zonkdogfology", one of the words in the first pair (part of the straight HTML). By the time I got online in the early afternoon of March 11th, it was part of the Google index and a search for it turned up the page as the sole result.

I then searched for each of the six words at Google.

• The two HTML words both generated a search result that included the page.
• The two words inserted by a JavaScript in the page generated no search results.
• The two words inserted by a remotely sourced JavaScript generated no search results.

Now, it's too early to say conclusively that Google will never index the JavaScript-generated content, barring a change in their search/indexing algorithms. I'll continue to monitor the situation over the next two weeks to give Google time for any secondary processing and distribution to all their datacenters. It is worth noting though, that at least in the immediate term, content in your pages that is made part of the page via JavaScript document.write statements will not be searchable in Google.

GOING FORWARD: Over the next two weeks, I'll be watching to see two things. First, does the indexing change so this page shows up in searches for the four JavaScripted words? And second, how long does it take for MSN and Yahoo to pick up the page and how do they treat it?

Stay tuned.

Addendum: People have been asking why you'd want to index dynamic JavaScripted content... Look at the dozens of comments on this article. They're all going to be indexed by Google because the inclusion is server-side. That's got some value. Comments in general don't just enhance the user experience, but add indexable content to your page and can organically increase your keyword density.

If you're using an AJAX powered comment module, particularly one that's remotely hosted like JS-Kit, then it's important to know what you're getting and what you're losing. Yes, you may be adding functionality to your page easily and enhancing the user experience, but if you don't get the comments indexed, you lose all that juicy keywordy goodness.

Given, I didn't do a heavily AJAXed test with nodes and other constructs. I decided to do the most simple construct... document.write(). I may do other tests in the future. But this was a good place to start. See, in both instances of the JavaScript inserted words, they were included in the scripts as discrete strings. If Google merely indexed the page and made the script text part of the searchable index, the two words from the script that's hardcoded into the page would become searchable. If it read the remote script and indexed it in the same manner, we might see those last two words showing up either in the test page or get the remote script as a hit.