Feature: XML Sitemap for every blog to boost Google indexing speed

in #engrave6 years ago (edited)

While preparing next big release of Engrave platform, we are continuously developing and introducing necessary features and fixes for our users. One of those features were XML Sitemaps to boost our users' blogs positions in search results and allow bots to crawl and index articles much faster.

And as always, you can find our source code on our Github.

What are XML Sitemaps

A sitemap is a special document created for a specific website. It contains every URL that should become indexed by search engines. It uses special syntax, called XML, to serve pieces of information that could be interpreted by bots and web crawlers. It's like a "Table of content" for a website or a blog.

Why you should use a Sitemap

Web crawlers want to index every page of every website but sometimes it's hard to find some of the URLs. For example, if you have a new blog without external linking to your articles.

Having sitemap with URL to every article is crucial for better Search Results visibility. Steemit links are getting indexed quite fast because of "old" domain (it exists for over 2,5 years) but new blogs - don't. So you need to boost it manually, using appropriate mechanism recommended by Google itself.

How to use Sitemap

Engrave will automatically create Sitemap for your blog and will update it whenever you make changes (like adding or removing an article). Sitemaps are available at /sitemap.xml URL, which means that for our official blog, the sitemap URL will look like that:

https://blog.engrave.website/sitemap.xml

Next, you need to add it to Google Search Console. Create an account, if you don't have one and click "Add property". Select "URL prefix" and fill the form with your blog URL. Use https:// prefix and type entire address.

Then, you will be asked to verify ownership for a website you're trying to add. It's quite simple - just click HTML Tag, copy only the "content" part as it is highlighted on this screenshot below.

Open a new tab, log in into Engrave Dashboard, navigate to Statistics and paste selected text into GOOGLE WEBMASTER TOOLS TAG VERIFICATION CODE CONTENT field and save it.

Go back to Google Search Console and click Verify. If it quickly failed - wait few seconds and try again - Engrave need a few seconds to refresh blog settings.

You should see confirmation for successful ownership verification.

Navigate to "Sitemaps", enter sitemap.xml into the appropriate field and click the "Submit" button. Your sitemap will be automatically added and fetched by Google Bot.

Refresh page to get a new status. Your sitemap has been added to Search Console and from this on, Google bot will periodically fetch it to get the newest articles to crawl automatically. Your pages should get indexed much faster than previously and it may lead even to better search results positions (but it's not guaranteed).

Thanks for using Engrave!


Originally posted on ENGRAVE Blog. Steem blog powered by ENGRAVE.

Sort:  

Thank you for your contribution. You can actually write a script to submit the sitemap periodically to Google, Bing and other search engines. I wrote a PHP script to do this long time ago and it still works!

For bigger sites, single sitemap.xml may become too big to be accepted by most search engines. e.g. Google denies sitemap with more than 5000 URLs, in which case you need to categories your URLs and store in separate sub-sitemap. Also, the sitemap can be zipped as well.

Your contribution has been evaluated according to Utopian policies and guidelines, as well as a predefined set of questions pertaining to the category.

To view those questions and the relevant answers related to your post, click here.


Need help? Chat with us on Discord.

[utopian-moderator]

Thank you for the details! I am aware of sitemaps limitations and I'm going to introduce a simple solution for that in the near future. Sitemaps could be automatically split into "chunks" (or like you said - sub-sitemaps). But right now there is no such a big blog on Engrave for which it could be a problem :)

I believe that using Google Search Console is much better than submitting sitemaps automatically because of useful information (like search positions, CTRs, etc) this service provides. I will, of course, consider pushing those sitemaps automatically to the services mentioned by you but for now - let bloggers do that manually :)

Thank you for your review, @justyy! Keep up the good work!

Hi Team @engrave,

I recently bought the domain name ChrisRice.blog and I just set it up on @engrave a few minutes ago.

  • It said that I can open my blog but once I did, it took me to someone elses @engrave blog.

Can you help me get it up and running?

I am going to transfer my blog ChrisRice.xyz on Blogger to ChrisRice.blog on @engrave. Then if Steem trends like I think it will I will forward the Blogger URL to ChrisRice.blog and use the domain ChrisRice.xyz as a splash page to display my social media accounts and upcoming projects.

~ @chrisrice

I was just turned onto your site today, very nice job. This is what I've been waiting for since the bootstrap phase!

Visit @engrave profile in a day or two :)

Really great to see this get added.

I have planned to add this for weeks and your comment actually gave me a kick to make it happen :)

Hi @engrave!

Your post was upvoted by @steem-ua, new Steem dApp, using UserAuthority for algorithmic post curation!
Your post is eligible for our upvote, thanks to our collaboration with @utopian-io!
Feel free to join our @steem-ua Discord server

This post has been included in the latest edition of SoS Daily News - a digest of all the latest news on the Steem blockchain.

Hey, @engrave!

Thanks for contributing on Utopian.
We’re already looking forward to your next contribution!

Get higher incentives and support Utopian.io!
Simply set @utopian.pay as a 5% (or higher) payout beneficiary on your contribution post (via SteemPlus or Steeditor).

Want to chat? Join us on Discord https://discord.gg/h52nFrV.

Vote for Utopian Witness!

set it up, it's working great! :)

I'm glad to hear that :)

Good day.

There are several questions:
When will you be able to create your own templates or when the template designer will appear?

There is a lack of a function where you can insert your own link or create a menu from your own links.

Insert a piece of your code into a template.

What will happen to the site if you suddenly stop your development and turn off your hosting, is it possible to place your code on your hosting?

I am sorry, translation via Google translator.

Напишу и на русском языке, вдруг вы понимаете).

Есть несколько вопросов:
Когда будет возможность делать собственные шаблоны или когда появится конструктор шаблонов?

Очень не хватает функции где можно вставить собственную ссылку или создавать меню из собственных ссылок.

Вставлять в шаблон кусок своего кода.

Что будет с сайтом если вы вдруг прекратите свою разработку и отключите свои хостинги, можно ли разместить ваш код на своем хостинге?

Hi! We hope it will be possible in a month or so. We're working on new dashboard and API which will allow us to introduce that kind of features much quicker.

Engrave is open source and everyone with appropriate knowledge, experience and funds are able to set up a project on its own. And, of course, all articles are stored on blockchain so you will never lose your content.

Hello.
Could you make it so that I could install the engrave platform on my hosting as easy as WordPress?

Just if you abandon the development of my domain will stop working, all the articles will end up on the steem blockchain, but all the indexed pages with my domain that will be registered to your hosting will lose their weight. I want to avoid this outcome.

The self-hosted version is in our roadmap but it is not a priority.

Where can I see your roadmap?

Hello.
Tell me what I need to change in the settings of the domain name?Снимок экрана 2019-04-15 в 13.39.44.png

Added CNAME record, but still doesn't work
Снимок экрана 2019-04-15 в 14.02.35.png

I can not add a site, this window appears.
Снимок экрана 2019-04-15 в 14.03.52.png

What am I doing wrong?

The question is removed, it all worked!

DNS propagation takes time so after any change, you need to wait a few minutes. Worst case, it can take even up to 48h.

Congratulations @engrave! You received a personal award!

Happy Birthday! - You are on the Steem blockchain for 1 year!

You can view your badges on your Steem Board and compare to others on the Steem Ranking

Vote for @Steemitboard as a witness to get one more award and increased upvotes!