a new Hexo plugin [hexo-stop-tag-plugins] -- write pure markdown with Hexo

This post will introduce the development of a Hexo plugin hexo-stop-tag-plugins, which helps hexo users to disable built-in tag plugins / nunjucks syntax, and write with pure markdown syntax.

To be more precise, this post's contribution include:

a new hexo plugin and NPM package hexo-stop-tag-plugins
one defect fix to hexo project, which is included in hexo 3.9.0 release

Repository

hexo plugin: https://github.com/think-in-universe/hexo-stop-tag-plugins
submit plugin to hexo site: https://github.com/hexojs/site
defect fixing of hexo: https://github.com/hexojs/hexo

Commit History

You might want to check the below commit history and PR to understand the contribution.

All the commits of https://github.com/think-in-universe/hexo-stop-tag-plugins, and shipped 0.1.7 version to NPM hexo-stop-tag-plugins, with success test results and code coverage.
Add the plugin to hexo site with Pull Request: https://github.com/hexojs/site/pull/966, and now hexo-stop-tag-plugins is available on the hexo plugins page https://hexo.io/plugins/
Fixed the defect about disableNunjucks in the Pull Request: https://github.com/hexojs/hexo/pull/3573, and released in hexo 3.9.0 yesterday (check release note for details)

a new Hexo plugin: hexo-stop-tag-plugins

_{Image Source: https://github.com/hexojs}

Why did I create hexo-stop-tag-plugins?

(1) Issue when building blog from steem with steemblog

In my last post, I introduced steemblog, which is a blog service and tool built by @robertyan that synchronize your steem blogs into GitHub pages or other static site hosts, e.g. https://steemblog.github.io/@utopian-io/ ,
https://steemblog.github.io/@robertyan/

During the creation of steemblog, we met issues such as Failed to build when markdown body contains special chars when we're synchronizing some Steem user's posts in markdown into steemblog.

For example, the {%s} string in the below code block from the post https://steemd.com/@oflyhigh/mfc-access , will break the conversion from post's source into html, with no html result returned.

sConnect.Format(TEXT("ODBC;DRIVER={%s};DSN='';DBQ=%s;"), sDriver, sDBInfo);

This issue happens because hexo enables tag plugins by default, which relies on the nunjucks syntax, and there's no settings to disable that by hexo users.

To make steemblog function well, we need to fix this issue.

(2) A common painful issue in hexo community

What's more, this is a quite common issue complaint by a lot of hexo users. You can checkout relevant links and examples from my PR and issues, and examples are:

In order to overcome this pain for the hexo users who want to get more control over the tag plugins syntax, we implemented this hexo plugin hexo-stop-tag-plugins for those who suffered from the common issue.

How to Use

To use this plugin, you can check out the description in the NPM package: hexo-stop-tag-plugins

It's as easy as install it with NPM.

$ npm install hexo-stop-tag-plugins --save

That's it. Then your post will be rendered by hexo as pure markdown, and you can use packages like marked as the renderer.

Technology Stack

JavaScript (ES6) / NodeJS

Implementation

Below we'll introduce how the plugin is implemented.

(1) set disableNunjucks at renderer level

The implementation of the plugin is not complex if you're familiar with hexo's source code. In project https://github.com/think-in-universe/hexo-stop-tag-plugins, we implemented the overrider.js to set the disableNunjucks property of renderer to true. Then hexo post renderer should skip the renderering with nunjucks.

We come up with this idea by reading thorough the renderer process of hexo, and look at relevant issues and pull requests in hexo repostiory.

override.js

/*
  The module is created to disable the tag plugins (https://hexo.io/docs/tag-plugins)
  and nunjucks syntax, which causes troubles in parsing markdown often
*/

'use strict';

const extensions = ['md', 'markdown', 'mkd', 'mkdn', 'mdwn', 'mdtxt', 'mdtext'];

module.exports = function(hexo) {

  let stop_tag_plugins = true;

  if (hexo.config.stop_tag_plugins === undefined || hexo.config.stop_tag_plugins === true) {
    stop_tag_plugins = true;
  } else {
    stop_tag_plugins = false;
  }
  for (let ext of extensions) {
    let renderer = hexo.render.renderer.get(ext);
    if (renderer) {
      renderer.disableNunjucks = stop_tag_plugins;
      hexo.extend.renderer.register(ext, 'html', renderer);
    }
  }

};

index.js

/* global hexo */

'use strict';

var overrider = require('./lib/overrider');

overrider(hexo);

We also added sufficient test cases to test this plugin, which you can checkout in test folder.

However, what makes us frustration is that the code blocks in markdown are not renderered correctly after we set the disableNunjucks property, and only placeholder of the code block are displayed.

This should be an issue of hexo, and that leads us to fix the defect in hexo with PR #3573

(2) fix the problem with the disableNunjucks property in hexo project

To understand the issue, you can read more details in the PR: https://github.com/hexojs/hexo/pull/3573

The property disableNunjucks was introduced in PR #2593 by one contributor. However, it's not added correctly and leads to problems.

To fix that, we modified lib/hexo/post.js in hexojs/hexo repository, which returns content after the placeholders (like a code block) are replaced.

lib/hexo/post.js

    // Render with markdown or other renderer
    return ctx.render.render({
      text: data.content,
      path: source,
      engine: data.engine,
      toString: true,
      onRenderEnd(content) {
        // Replace cache data with real contents
        data.content = cacheObj.loadContent(content);

        // Return content after replace the placeholders
        if (disableNunjucks) return data.content;

        // Render with Nunjucks
        return tag.render(data.content, data);
      }
}, options);

The PR #2593 that introduced the
disableNunjucks property didn't add tests. So, to assure the quality, added unit tests for the disableNunjucks property:

test/scripts/hexo/post.js


  // test for PR [#3573](https://github.com/hexojs/hexo/pull/3573)
  it('render() - (disableNunjucks === true)', () => {
    const renderer = hexo.render.renderer.get('markdown');
    renderer.disableNunjucks = true;

    return post.render(null, {
      content: fixture.content,
      engine: 'markdown'
    }).then(data => {
      data.content.trim().should.eql(fixture.expected_disable_nunjucks);
    }).then(data => {
      renderer.disableNunjucks = false;
    });
  });

  // test for PR [#3573](https://github.com/hexojs/hexo/pull/3573)
  it('render() - (disableNunjucks === false)', () => {
    const renderer = hexo.render.renderer.get('markdown');
    renderer.disableNunjucks = false;

    return post.render(null, {
      content: fixture.content,
      engine: 'markdown'
    }).then(data => {
      data.content.trim().should.eql(fixture.expected);
    }).then(data => {
      renderer.disableNunjucks = false;
    });
  });

test/fixtures/post_render.js

exports.expected_disable_nunjucks = [
  '<h1 id="Title"><a href="#Title" class="headerlink" title="Title"></a>Title</h1>',
  util.highlight(code, {lang: 'python'}),
  '\n\n<p>some content</p>\n',
  '<h2 id="Another-title"><a href="#Another-title" class="headerlink" title="Another title"></a>Another title</h2>',
  '<p>{% blockquote %}<br>',
  'quote content<br>',
  '{% endblockquote %}</p>\n',
  '<p>{% quote Hello World %}<br>',
  'quote content<br>',
  '{% endquote %}</p>'
].join('');

After this defect is fixed in hexo and released in hexo 3.9.0 version, we could say the hexo-stop-tags-plugin should work well for hexo users.

(3) publish the plugin to hexo/plugins and NPM

To make the plugin available to its users. We published it to NPM and hexo/plugins page.

The submission to hexo/plugins page is merged in PR: https://github.com/hexojs/site/pull/966

source/_data/plugins.yml

- name: hexo-stop-tag-plugins
  description: Disable tag plugins for all markdown posts
  link: https://github.com/think-in-universe/hexo-stop-tag-plugins
  tags:
    - markdown
    - tag_plugins
    - renderer

It takes a few days to have the plugin be reviewed and merged.

Retrospective

The hexo repository reviewers are not that active, and may need close follow-up from contributor to make PR to be reviewed and merged.
May need more thoughts and actions on how to push this plugin to the users who met the same issues.

Roadmap

The disableNunjucks property is set at renderer level and applied to all the posts. User may need another setting to make it only applied to some of the posts, which is more flexible.
hexo-stop-tag-plugins is quite useful if we want to bring more hexo blog writers onto Steem. It makes it easier for bi-directional integration between hexo and steem: hexo users can use the plugins to write posts that can be published onto Steem; Steem users can use the plugins to render and organize their steem posts with hexo.
As a successor of this plugin, we may create another plugin that could make it easier for hexo users to share their posts onto Steem. And as a result, may introduce more developers and writers to Steem community.
My long-term vision is to make better integration between Steem and GitHub features (such as GitHub pages), to make developers enjoy their work and life in blockchain era.

_{picture made by me which combines GitHub and Steem logos}

How to Contribute

For anyone who wants to contribute to this project, feel free to fork https://github.com/think-in-universe/hexo-stop-tag-plugins, and submit Pull Requests.

Or please directly contact me (@robertyan) on Steem if you have any questions to discuss.

Github Account

my personal account: https://github.com/think-in-universe (can be verified with the links in the profile)

To reviewers: this post is set to "development" category because it contains a new project (a hexo plugin), and defect fixing to "hexo" project contributed by myself. If you have any concern, feel free to reach out to me directly.

Sort:

Trending

[-]

amosbastian (72) 5 years ago

Never heard of Hexo before, but seems like a pretty cool project. Your plugin project seems pretty cool as well. Funny to see so many tests compared to the actual code you added, but it's all of a high quality as far as I can tell. Curious to see what other plugins you come up with in the future!

Your contribution has been evaluated according to Utopian policies and guidelines, as well as a predefined set of questions pertaining to the category.

To view those questions and the relevant answers related to your post, click here.

Need help? Chat with us on Discord.

[utopian-moderator]

$7.65

16 votes

robertyan (63) 5 years ago (edited)

Thanks @amosbastian.

Thanks for the kind comments.

hexo should be quite popular with ~27,000 stars and ~50,000 "used by" on GitHub, and it's among the top static site generation tools and blog framework for years (competitors like Jekyll).

Yes. the change itself doesn't need thousands of lines of code, and mainly requires understanding about the framework. The fix to the defect in hexo project is also kind of one-line change. :)

In engineering world, the tests are so important to ensure the quality. I know a lot of utopian submission don't have good tests, but that's what utopian could help to produce / encourage high-quality open source projects. :P

$0.00

utopian-io (71) 5 years ago

Thank you for your review, @amosbastian! Keep up the good work!

team-cn (66) 5 years ago

Thanks being awesome NBC holder! Your 40.70 NBC earned you 6% team-cn upvotes！

steemitboard (66) 5 years ago

Congratulations @robertyan! You have completed the following achievement on the Steem blockchain and have been rewarded with new badge(s) :

You published more than 40 posts. Your next target is to reach 50 posts.

_{You can view your badges on your Steem Board and compare to others on the Steem Ranking}
_{If you no longer want to receive notifications, reply to this comment with the word STOP}

To support your work, I also upvoted your post!

Vote for @Steemitboard as a witness to get one more award and increased upvotes!

steem-ua (64) 5 years ago

Hi @robertyan!

Your post was upvoted by @steem-ua, new Steem dApp, using UserAuthority for algorithmic post curation!
Your post is eligible for our upvote, thanks to our collaboration with @utopian-io!
Feel free to join our @steem-ua Discord server

Hey, @robertyan!

Thanks for contributing on Utopian.
We’re already looking forward to your next contribution!

Get higher incentives and support Utopian.io!
Simply set @utopian.pay as a 5% (or higher) payout beneficiary on your contribution post (via SteemPlus or Steeditor).

Want to chat? Join us on Discord https://discord.gg/h52nFrV.

Vote for Utopian Witness!

cnbuddy (-4)(1) 5 years ago

Reveal Comment