FreeSitemapGenerator.com Sitemap news feed

PDF files in your sitemap.xml

Wed, 06 Oct 2010 00:00:00 +0200

Freesitemapgenerator.com user Bruce suggested to us to include PDF files in our generated sitemaps. We did some research and found that Google says all valid URLs you want to be spidered should be listed in your sitemap.xml. So we went to work and added new filters to our crawler: it now will add PDF files it finds on your site automatically to your sitemap.xml. You do not have to change any settings in the control panel to activate this feature.

Sitemap vs No sitemap - the ultimate test - PART 2

Sat, 02 Oct 2010 00:00:00 +0200

Last month we posted an article on a generated sitemap for a very large site (125,000 pages per subdomain), Read more. Now a month (almost) has passed, and we are ready to publish the first results. See the image with the number of indexed pages per subdomain.
In words: the two top lines are the two subdomains for which we created and submitted a sitemap. Indexing was fast, within a week a total of 35,000 pages were in Google's index. For the English subdomain, which is 'www' (3rd line) it took a week longer to get indexed and it seems to take more time to get the pages crawled. It now has passed the 5000 line, while the two subdomains are declining. The other lines at the bottom are the subdomains we did not submit a sitemap for. With just one site tested, for a very short period, we do not want to give you any hard conclusions, but our opinion still is: a sitemap helps getting your site indexed.
The fancy graph was generated by Microsoft Excel :-)

Sitemap vs No sitemap - the ultimate test - PART 1

Sun, 05 Sep 2010 00:00:00 +0200

We have received a couple of (10+) e-mails over the last few months from (potential) users of freesitemapgenerator.com with more or less the same question: Should I create a sitemap for my site? Google says you should - so you should, see our links page but do sitemaps actually work? A year ago we posted a message on this page with some nice graphics of howmany pages were indexed by google before and after submitting a sitemap but now we are ready for a real double blind test!

For a customer we created a program to generate a sitemap for their database driven travel site. Each subdomain contains more or less the same information but in another language and has 125,000 unique urls. All subdomains are linked only from the English homepage and will not be promoted on their own. Our educated guess is that all subdomains will have about the same number of pages indexed by Yahoo!, Google and Bing. For two of the subdomains we did not submit the sitemap and did not upload it to the webserver. They just don't exist and all the spiders should try to figure out themselves all the possible urls. Now we wait. In about a month we will report the number of indexed pages for these subdomains. We don't take bets.

(and yes, our client knows about this test - we will add the last two sitemaps when this test is concluded)

We got an award!

Wed, 25 Aug 2010 00:00:00 +0200

We have been recommended and awarded as a top ranked Site Map Generator by leading cheap web hosting guide WHS! We have received e-mails from users with compliments - thank you - but we have never received an award and this one came with a cool badge too.

Thank you guys and girls at WHS. Check it out.

Freesitemapgenerator does not understand IDN (yet)

Thu, 08 Oct 2009 00:00:00 +0200

We have added an IDN to Punycode converter. If you want to add a site to freesitemapgenerator.com with an international domain name like 'löl.org' you can use this page to convert it to punycode ('xn--ll-fka.org'). We will add this convertor to the add-url-form as soon as possible so you can add International Domain Names without having to convert them manually.

XML2HTML package - release 1.0.1

Thu, 17 Sep 2009 00:00:00 +0200

Thanks to a bug report from Carl, we could solve an issue with the xml2html script.

Issue:
If you used a sitemap (created by freesitemapgenerator.com or any other sitemap.xml) that has no <priority> tags, an empty sitemap would be displayed due to the $MINPRIORITY setting (0.5) at the top of the script.

Solution:
We modified the script so that if no priority tags are used, all urls in the sitemap will be displayed on the sitemap.php page.

Should I upgrade to 1.0.1?
Only if you are using sitemaps without the priority tag. To upgrade, remove sitemap.php and sitemap.cache and follow the instructions on http://www.freesitemapgenerator.com/xml2html.html.

Google crawl stats

Sun, 06 Sep 2009 00:00:00 +0200

Google has a nice feature for webmasters: you can see howmany pages Googlebot has read from your site. We got this screenshot from one of our users who started using freesitemapgenerator.com recently and saw some unusual activity on his PR2 website:

(click on the image for a readable version)
All we can say is: wow!

Verification issue with redirects solved

Thu, 03 Sep 2009 00:00:00 +0200

If you had problems getting your site verified by our spider using the second option (link on page) we have solved one reported bug: * If your homepage is a redirect to another page, which contains the link your site could not be verified - now it will

Bing Sitemaps

Thu, 03 Sep 2009 00:00:00 +0200

What about Bing.com? All this time this site has been talking about "Google Sitemaps" but the generated sitemaps are compatible with bing.com-sitemaps too. A new checkbox for all your sites: auto-ping-bing to send bing sitemaps to bing as soon as your sitemap is uploaded to your site. Only works if you are using the auto-upload package ofcourse.

RSS feed for freesitemapgenerator.com

Thu, 03 Sep 2009 00:00:00 +0200

Title says it all! Welcome in the 21th century ... How long will it take us to get a twitter account?

Sitemap for blogs

Sun, 30 Aug 2009 00:00:00 +0200

Users with Blogs, flash or other websites that they cannot control completely reported issues with getting their site validated by our spider. We need to be sure the URL(s) you enter are yours so our spiders will only sites that want to be spidered. The easiest way to do this, we thought, was to let you put an empty text file on your website we can fetch. But if you are using blog software, shop software, a fancy cms, website admin tool it is pretty hard to get a text file up on your site. That's why we have added a second option to get your site verified: just put a link back to us on your website, we will find it. For more details, hit the "Make Sitemap" button in the control panel again and you will see both options explained.

Google SOAP API deprecated

Fri, 28 Aug 2009 00:00:00 +0200

We used to use the Google SOAP API using your API-key for the 'Indexed Pages' feature in the control pannel. Google now has deprecated the SOAP API. Thanks to all for the e-mails about this, and to our developers for quickly rewriting the code to use the AJAX API from google. We do not need your API-key anymore, we will use one (www.freesitemapgenerator.com-key) for all requests. We are not sure if the results from the AJAX API are the same as the SOAP API, but we do not have a choice but to migrate to this new API. For more information: see the sitemap links page.

Free sitemap generator Forum removed

Tue, 25 Aug 2009 00:00:00 +0200

Our forum has been removed. Very sad because some of you gave some nice tips and helped out others, but we were flooded by p*n*s enlargement sites and n*ke shoes... Maybe the forum, or something like it, will return in the future but for now: forum closed due to spam...

New hardware for freesitemapgenerator.com

Thu, 20 Aug 2009 00:00:00 +0200

We have installed 2 new servers, one for our premium members and one for our free users. You will notice that the queue is almost gone and your sitemaps will be generated much faster!

New hardware for freesitemapgenerator.com

Thu, 10 May 2007 00:00:00 +0200

More Hardware for FreeSitemapGenerator.com! This week we have installed two new servers for freesitemapgenerator.com and after some serious testing, these servers are now up and running! Sitemaps will be generated faster.

Blog and Sitemaps

Thu, 21 Sep 2006 00:00:00 +0200

Do you have a site at blogger? And you use freesitemapgenerator.com to generate Google™ sitemaps for you? Please read this post on the forum and help some fellow users!

Sitemap spider update

Fri, 15 Sep 2006 00:00:00 +0200

Spider update: thanks to a bug-report we got from Luciano (many thanks) we fixed a bug in the Free Sitemap Generator spider. The <BASE HREF="..."> tag was not parsed correctly. If your site is using this tag and your sitemap was useless, try generating a sitemap again: it is better now.

Googlebot bandwidth usage

Sun, 10 Sep 2006 00:00:00 +0200

We host a couple of sites that got Googlebot's attention for some reason. Yesterday it fetched 22,388 pages and today 42,933. That's a good thing I guess: to be found, you must be spidered first. But this is taken a lot of bandwidth. I browsed around and found a page (on google.com) saying I should e-mail google about this. Well, see the next post on this page: why not compress HTML output to reduce bandwidth usage? If you have the right modules loaded in your Apache webserver, just a single line in the .htaccess file will do:

AddOutputFilterByType DEFLATE text/html text/plain

This reduced bandwidth use (by Googlebot and others) with 60%!

Spider update (compression)

Sat, 02 Sep 2006 00:00:00 +0200

The spider/crawler has been upgraded. It now supports gzip and deflate compressed content. This can save bandwidth use on both sides. Check this forum post for more information.

Introducing auto queue

Mon, 28 Aug 2006 00:00:00 +0200

We have introduced a brand new feature: Auto Queue. This option allows you to select the sites you want to have fresh sitemaps generated every 14 ~ 21 days automatically. In combination with the Auto Upload package, this new sitemap will be uploaded to your site immediately. Because of the extra resources this will require, the Auto Queue feature is only available for premium users. Log in for more information.

Update: live data package 0.96

Sat, 08 Jan 2005 00:00:00 +0100

Live Data package update: new version is 0.96. If you are running version 0.94 or 0.95 you don't HAVE to upgrade now, but the new version is faster and more accurate. If you are running version 0.93 or earlier, you must upgrade if you want to continue using Live Data. Just replace the code you have now with new version as it can be found in the control panel. Remember this service is still in Beta Testing. We hope to release a 1.0 version soon!

Nothing to report

Fri, 09 Dec 2005 00:00:00 +0100

Four days without news. That's not good - we used to post news every day ... But at this time we really haven't any. At this time we have implemented everything we could and have no new ideas. If you have any, please post them at our forum.

Introducing: live data package

Mon, 05 Dec 2005 00:00:00 +0100

To speed up the generating process and have better <priority> tags, we have added what we have named Live Data package. Install it now on your site, just some lines of HTML and JavaScript to copy and paste.

One url issue

Fri, 02 Dec 2005 00:00:00 +0100

For some sites the generated sitemap only contains one URL. There can be many reasons for this so we created a new button on the control panel: Test. It will fetch your homepage and shows you the links we can see on it. This may help you to solve linking problems. In general: make sure you have links from your homepage to the most import sections of your site.

Auto ping google

Wed, 30 Nov 2005 00:00:00 +0100

After uploading a sitemap we can now Ping Google™ for you to let them know your sitemap has changed. Click on the Edit button to set this per site. Sites that will get auto-pinged will be marked with this icon .