Hello, Steemians!
Over the past month, I've been building a new search engine that indexes the steem blockchain. It's currently live at asksteem.com. The goal of AskSteem is to provide a reliable, powerful, and fast search engine that is optimized for steem. In this post, I'd like to cover some of the features that are available.
Query Syntax
There are many different ways that you can query the AskSteem index. I've created a video demonstrating each of them, but you may also read their descriptions and examples below.
Keyword/Phrase Search
Like many other search engines, you can search for general phrases and terms. AskSteem will try its best to find the document that is most relevant to your query based on our ranking algorithm.
Example Queries:
Tip: Click the example to go to that query on AskSteem
How to buy bitcoin
What is steem
Markdown tutorial
Exact Search
Putting a query into quotes requests that AskSteem only returns documents that have exactly that phrase in that order.
Example Queries:
"How to buy bitcoin"
"What is steem"
"Markdown tutorial"
Tag Search
AskSteem allows you to filter posts by tag.
Example Queries:
tags:life
tags:steemit
Author Search
You can filter posts by the author too.
Example Queries:
author:thekyle
author:abit
author:steemit
Creation Date Search
AskSteem provides a highly flexible and powerful date search tool for posts. You can search by exact date or by date range. Dates must be in the form of YYYY-MM-DD.
Example Queries:
Search for all posts posted on June 2, 2017
created:2017-06-02
Search for all posts posted between May 1, 2017, and May 31, 2017
created:[2017-05-01 TO 2017-05-31]
Search by Number of Votes/Comments
Similar to dates AskSteem has another set of robust tools that allow searches based on the number of upvotes or comments a post receives.
Example Queries:
Posts with 150 votes:
net_votes:150
Posts with between 100 and 150 votes
net_votes:[100 TO 150]
Posts with 50 comments:
children:50
Posts with between 40 and 50 comments:
children:[40 TO 50]
Posts with more than 50 comments:
children:>50
or less than 50:
children:<50
this also works with votes, less than or equal to 10 votes:
net_votes:<=10
Searches with Boosts
You can prioritize certain parts of your query with boosts. These are indicated by placing a ^n at the end of a term, where n is the power you want to boost that part of the query to.
Example Queries:
Give the term bitcoin a boost of two:
I really want posts to have the term bitcoin^2 in them.
Give the term mine a boost of two, and term steem a boost of three:
How to mine^2 steem^3
Inclusive/Exclusive Search
You can indicate whether you want documents to contain certain terms by placing a + or a - in front of the term.
Example Queries:
Find documents about mining but not bitcoin:
cryptocurrency +mining -bitcoin
Wildcard Search
You can use the wildcard expressions of ? for a single character, or * to match any number of characters.
Example Queries:
How to mine any cryptocurrency:
How to mine *
Boolean Search
AskSteem supports any combination of the previously mentioned search types in a single powerful query. This uses boolean values of AND, OR, and NOT, along with parenthesis to separate statements.
Example Queries:
Posts tagged with asksteem by @thekyle:
tags:asksteem AND author:thekyle
Posts with between 50 and 100 comments that have more than 500 upvotes and that are tagged with 'bitcoin' or have the term bitcoin in the document:
(bitcoin OR tags:bitcoin) AND (net_votes:>500 AND children:(>50 AND <100))
Posts created on June 2, 2017, with 100 or more upvotes but less than 10 comments:
created:2017-06-02 AND net_votes:>=100 AND children:<10
Developers
Because AskSteem integrates directly into the steem blockchain it can read metadata directly from posts and use that data when performing queries and displaying results. We encourage developers to add AskSteem compatible metadata to their posts so that we can show links to your application in our search results. The full documentation can be found at asksteem.com/developers, however, in this post I will summarize the most important tags.
| Tag | Description | Example |
|---|---|---|
| domain | The domain name or web address that your application is hosted on. | example.com |
| locator | The path to reach the post on the domain relative to the root. | /CATEGORY/@AUTHOR/PERMLINK |
| protocol | Either 'http' or 'https' if not provided then http will be used by default | https |
If none of the above metadata is provided then AskSteem will link to steemit.com for all posts by default, however, it is assumed that the platform creating the content will have the best interface for viewing it, so we would rather link there.
The domain and locator tags are required for custom linking to work, however, the protocol tag is optional and will default to http.
- The domain tag should be the domain name that your web-based steem application is hosted on and is subdomain sensitive (so if your hosting on www subdomain then put that).
- The locator should be the permalink to that particular post in your applications URL structure, also notice the leading forward slash, this is required.
The final URL that we point to will be generated by concatenating the domain and locator together with the protocol at the beginning which will be http unless otherwise specified.
Additionally, if you are building an application on the steem blockchain and need a search API please email us at [email protected], we are able to query custom metadata and make various other customizations to the ranking algorithm to support your use case.
Funding
The harsh reality is that search engines are expensive to run and that adding new features and improving performance are difficult if the basic funding needs of the project are not covered. AskSteem currently costs me about $100/month to run, and that number will, of course, continue to increase as steem grows and the index size increases.
Ideally, my goal is the have those costs covered through upvotes from the steem community and to use any extra money for adding new features and scaling the search infrastructure to meet demand. If this works then AskSteem will be the first search engine in the world to use a cryptocurrency based revenue model, instead of selling advertising.
Thank you for your time, and happy searching!
Happy to see this. I resteemed.
Thanks, I look forward to making it even better!
yay
How do your 'Resteem' a post?
Highly recommended tags for such post #steemdev and #steem-project
You almost missed out on 300$ of reward by not having this tag.
Well thanks for letting me know, I'll make sure to include those in future posts.
you can edit
Done!
This is something many were waiting for! Tried and it works well! It has to be developed, congrats for the project!
Yes thank you! I can now search then click back after reading an article and it will take me back to my search results. Small thing but needed! 😎
Glad you like it!
Looks great. We have the metadata, so searching by that should make life easier. Thanks
Hi @thekyle.
I'd like to tell you how happy I am for the lesson, not just on asksteem.com, but also on how to do a search and ferret out what might otherwise be obscure - but important - results. I've been doing a little research now and then on a couple of open source investigations, and just learning more about how searches operate will help me to research for my own topics. It will also help me better assist the activists and freedom-fighters who are busy helping us reclaim our democracy.
It's a pleasure to meet you. I'm @joanroberts, and I've been here for a whole week. Forgive me for not upvoting - my (literal) 2 cents is going to go to someone else, who is new, but who, like yourself, adds to the community. Trust me. When your vote is only worth 2 cents, when you see someone upvote you and you've got 4 cents, while it doesn't make you rich, it does make you smile.
Hi Joan thanks for your comment. I'm glad you're enjoying AskSteem and I hope you'll continue to use it for your investigations. Have a 23 cent upvote from me. :)
@thekyle, you just brought tears to my eyes. Thank you.
Great tool, thanks for building this. Is there a way to sort the search results? For examply by number of votes or comments? Or does anybody know another tool that does this?
Not currently through the Web UI, but that functionality is available in our developer API so it may come in the future.
Ah great, you have an API. Thanks for answering, I'll definitely look into that.
@thekyle please repost this for the benefit of those who haven't had a chance to see it. I wanted to resteem but that option is gone as the post is too old. Old posts can still be a valuable resource for us on Steemit.
I'm not going to repost because I don't want to put duplicates in my feed, however, feel free to resteem this newer post about user search functionality if you'd like to spread the word.
Great post! resteem! I hope one day it will have half of google traffic :)
Good work!!! Followed, up-voted and re-steemed the post for more visibility! I hope they can add it here on Steemit and replace the searcher we have now, it is really bad.
Yeah, I would definitely love to see some steem clients (including steemit.com) start using AskSteem as their search backend.
That would be perfect! :) Nice work man!
This is good. Got a question though: Is there a way to check who resteemed a post?
No, and sadly its not likely to be added in the future because resteems are not part of the blockchain and are kept in steemit.com's private database I have no way to index them.
Too bad :/ Thanks for the quick answer.
Great effort @thekyle
Thanks!
You're welcome, @thekyle
This is really cool! I added you to my Tools Collection
Thanks! Looks like a cool list so I'll have to check out some of the other stuff on there.
I like everything but the name. Expect slight competition from SteemCrawler and SteemaVista, but then Steemhoo and Steeng, will come in too. Hopefully before all that you will be bought out by Stoogle.
Haha well, thanks. Glad you like the concept, and maybe the name is one of those things that grows on you over time. :)
cool stuff ...resteemed
Thank you for providing a Steem search other than Google!
Yacy might make your indexing efforts more distributed and cost effective.
Yacy is a great Distributed Search engine. You can host a node that only indexes Steem in collaboration with anyone else who wants to index steem and share results. Yacy runs on Desktops, laptops and servers. Linux, Windows and Mac.
http://yacy.net/en/index.html
There is also a built in interface for your site that works out of the box. Together we are strong. Take a look if you're interested.
I have a basic demo that indexed some of steem and reddit on my Desktop here. http://camb.hopto.org:8090/
(this demo won't be up much longer)
I will be moving it to a proper server soon and begin indexing only distributed networks.
Good luck!
Very interesting, I'll be looking into this. :)
Yacy has been an undervalued project for years. It works great. If you have any questions getting started, feel free to ask me and I will answer as best as I can.
Hope my upvote helps a bit! Really cool tool!
Well they say every little bit helps. :)
Thanks, it looks and works good!
This is an awesome tool! Thank you for taking the time to create this for the benefit of all Steemians. ReSteem, followed.
Glad you like it! :)
About time, lol GREAT job man ! This is just ... well, Great !!! Hope more folks will start to appreciate the value of this Search Engine, got to pay the bills !, Good luck ! Come on Folks, vote it UP !!
Read More, Reason More ... JTS
It would be really nice to have a search engine dedicated to steemit! Waiting for the tool to go live.
It is live! https://www.asksteem.com :)
Genius!
Nice article!By the way,I have upvoted your post,can you upvote my new post? Thanks
Awesome post, app & tool dear @thekyle. To make the things on steemit flow as they should. With absolute order and harmony!!
Kickass search engine mate. A much succulent delight to be used.
A plain inspiration for my next post. Especially now, that I'm finally able to order and organize My original crazy stuff as God commands. Cheers!! }:)
¡Upvoted & Resteemed!
Glad you like it! :)
Oh! Much more than to like it my dear developer. I'm just beginning to squeeze the real juice out of your excellent creation.
Keep Up with the good work! :)
Very cool! Good job. Upvoted.
Glad you like it. Thanks for the upvote, every little bit helps :)
This seems like a interesting and helpful resource, thanks for sharing.
Hello, i just got here..
But..
If you have time to check my blog,
Maybe you can help with my proposal..
And say something about it.. My pleasure
great post i vote and resteem
Well done, this is much needed to find stuff in steemit
Congratulations on a fantastic project, I have voted and resteemed
What a cool idea! Just the other day I saw a comment from someone complaining about how crude Steemit's built-in search is. I wished I could point them to some nifty tool that improves the experience. Now I can! Going to go try this out right away... you sir have earned another follower!
Glad you like it! There is some room for improvement but it definitely works better than generic web search engines like Google (which is what steemit currently uses) which are not optimized for searching blockchains. Thanks for the follow, and make sure to spread the word if you see anyone else looking for an alternative to the default search. :)
I definitely will. Spent some time playing around with it and it seems to work pretty well so far.
Great job !!!
I like it. Works much smoother than the search on Steemit.com and has some nice features.
Resteemed!
Thanks for the resteem, I'm going to keep improving it, but I'm glad you like it. It should work better than Steemit's default search because it interfaces directly with the blockchain rather than using a HTML proxy layer (like the current steemit search)
Yes the google search function is a bit primitive so this is a great step up :)
Amazing work @thekyle. Hopefully the Whales will upvote. Re-steeming.
Really glad you like it! And thanks for the resteem.
I was just thinking today I f*cking hate the search bar on Steemit. Glad someone else thought the same and did something about it. Resteemed. Hope you get the funding! Cheers mate!
Thanks, good to see I'm not the only one who saw this as a problem that needed solving.
Kyle, Thanks for a great too. As a trader in all markets, your search tool helped me find two other Steemians who also post market and trading data.
Glad you found the tool useful. :)
Thank you, resteemed and have a question:
Is it possible to combine 2 search tags like author and exact search in one search request?
Glad you like it and yes. You can type
author:thekyle AND bitcointo get posts created by @thekyle (me) about bitcoin. This works with any two types of search. For more info read the part about boolean search in the above post.Thanks for answering!
Great job, a search engine is sorely needed!
Woow....genius, here to lend all the needed support
Awesome! I'm gonna need all the support I can get. :)
Solid!
Awesome.... Just upvoted and Resteemed...
nice work, though it would also be great if an author's comments could be searched as well!
Thanks, I haven't really had any plans for allowing users to search authors comments, but if people start making comments that might answer queries then perhaps in the future I will allow comment searching. :)
I just know that for me, I often find myself searching my past comments in reference to a new post or comment (or even in chat).
Just tried it and it works!!
God job.
I had a friend on here @kaptainkrayola who had a nice and effective search engine running for steemit called steemshovel.com until Feb 1st, 2017. He ended it up shutting it down for exactly the reason you indicated. No funding, and he was actually losing money by running it.
Yeah, I looked into SteemShovel before launching AskSteem. Technically in the backend both search engines use ElasticSearch for indexing the blockchain, and think SteemShovel may have also been hosted on AWS. Hopefully, my goal of funding the search engine through upvotes will work out, otherwise, there are some other ideas I have for getting funding.
Yeah my friend didn't post here enough or he likely could have funded it. He is a developer, but didn't really want to blog.
Funny enough it was him talking to me about Eth mining that caused me to stumble on steemit, before him. He was kicking himself and wondering how he missed it.
If you need to use the account @asksteem let me know :)
It's being used to ask questions , but maybe it can be used for people to query right inside steemit to pull up info from your search engine.
Yeah, I could build a bot that allows people to query the index from steemit directly. Are you available on steem chat?
I am good sir. Steemitqa under steemit.chat.
Yes, it would allow people to query the index from steemit directly and come back with the results.
Boolean + tag search + everything else = Winner! We have been needing something like this, and perhaps it can be implemented on the blockchain itself in a future hard fork? Got to replace the current search bar with this!
It unlikely that you could include a full search engine inside the of the blockchain and make it as fast as AskSteem, however its possible in the future that they could include the ability to natively sort posts by tag (I know I was surprised the blockchain didn't have that already too)
Out f'n standing!
This looks great. I'm sure it will be way better than current searching methods. Following for new updates. ^^
You're the man! Thank you very much for your great work! It's really fast :) Would be nice to have the filter possibilities in a html form. Maybe with a link to an "advanced search" or so... I'll resteem ;)
Thanks, I've been planning to create an advanced search page since I launched AskSteem so that is definitely on the to-do list.
Perfect! Do you have access to the number of views or are they tracked by Steemit only? If that would be possible, you could add this for searching "lost content", which didn't get many views (for whatever reason), but nevertheless could be interesting content for some of us...