Link to the github repository
https://github.com/steemit/steem
0. INTRODUCTION
Steemit currently does not have a direct messaging feature like that in other social networking services such as Twitter and Facebook. Instead, we can use the MEMO field of a TRANSFER as a DIRECT MESSAGE.
So, on the Steem platform, MEMO means a short message that can be associated with the transfer of Steem tokens. It was inherited from BitShares and is a great feature not implemented in the main cryptocurrencies such as Bitcoin, Litecoin and Ethereum.
Using MEMO as Direct Message:
- At least 0.001 STEEM/SBD is required to send memo.
- MEMO can be encrypted by the memo private key. The encrypted memo can be read by only receiver, so it serves as private message. To encrypt memo include # as a first character of your memo.
In this analysis I wanted to answer some questions like:
- How familiar is the use of transfers as direct messages?
- What has been the evolution over time of this use?
- What kind of accounts are using it and for what purposes?
Other previous related analysis:
1. ACQUIRING THE RAW DATA
I have used the Steem database of the Steem Blockchain Database Service (SBDS) of PRIVEX.IO. (sbds.privex.io) On the date that I obtained the data for this analysis, the date of the most recent records was April 25th, 2018.
The raw data used to perform this analysis is contained in the table TxTransfers.
So, the TEXT of the so called Direct Message can be found in the memo field:
EXAMPLE:
"block_num:3.945.257","transaction_num:1,","operation_num:1","timestamp:2016-08-10 00:10:21","from:alitas","to:silver","amount:"0,001"","amount_symbol:SBD","memo:"Hi Silver, maybe you want to take a look at my Steemit City Tour in Buenos Aires."","operation_type:transfer"
Therefore, a query to extract all the transfers than can be candidates to be considered like Direct Messages more than regular transfers, selecting the most relevant fields for this analysis would be:
SELECT
sbds_tx_transfers.timestamp,
sbds_tx_transfers.from,
sbds_tx_transfers.to,
sbds_tx_transfers.amount,
sbds_tx_transfers.amount_symbol,
sbds_tx_transfers.memo
FROM sbds_tx_transfers
WHERE sbds_tx_transfers.amount = 0.001
2. ANALYSIS
2.1 How many transfers as 'Direct Messages' have been made?
A first approximation to obtain the transfers that we can classify as candidates to be 'Direct Messages' are those transfers whose AMOUNT field are equal to 0.001 SBD/STEEM (I refer as milli-transfers)
Although It makes sense to make a payment or a deposit worth 0.001 seems reasonable to think that in general they will be higher than this value, so with the previous query almost all the transfers using MEMO as a description of payment or deposit to a exchange account are filtered out.
Considering the entire historical time period from April-2016 to the April-2018 there have been a total of 2,416,709 transfers with AMOUNT=0.001 that represent the 24% of the total number of transfers (10,067,879) in that period of time. 73.14% using SBD and 26.86% using STEEM.
I wondered if there could be transfers that can be considered Direct Messages that have Amounts greater than 0.001, (although still low such as 0.002, 0.003, ... to 0.006.) The answer is affirmative but the number of those transfers are in much smaller proportion as shown in the following table:
It is quite impressive to know that the milli-transfers (0.001 SBD/STEEM) are 24% of the total, so analyzing their behavior can be considered relevant in the Steem(it) ecosystem. I will investigate the nature of these transfers with the initial motivation to know the use as Direct Messages but without discarding any information that may appear.
2.2 What is the evolution over time of these milli-transfers?
Grouping the data by months in the time period from April-2016 to the April-2018 and also calculating the number of unique transmitters and receivers of the milli-transfers.
COUNT = Number of transfers of 0.001 SBD/STEEM
NUM_FROM = Number of unique transmitters accounts
NUM_TO = Number of unique receivers accounts
TABLE 1
To visually analyze the data of that table I have created the following graphs.
Number of transfers of value equal to 0.001 SBD/STEEM
This curious shark teeth pattern, with bursts or impulses, seems to me like 'cambrian explosions' of increasing magnitude followed by corrections or extinctions of 30% until the next explosive growth.
What generates these exponential increases and their subsequent falls?
Number of NUM_FROM (black) and NUM_TO (yellow)
The curve of the number of transfers has the same form as the number of unique Receivers (NUM_TO) that is, the number of transfers increases and decreases following the pattern of the number of unique receivers even when the number of issuers (NUM_FROM) remains fairly constant. Therefore, the behavior in the form of impulses and falls is due to the more or less active state of the issuing accounts of these transfers.
Can be estimate when the next peak will appear?
Just as a hypothesis I wanted to make an estimate using the analysis of stock market curves and using the concept of fractal, although in a simple and intuitive way since I am not an expert in those subjects. I have enlarged proportionally in magnitude the own curve of known values to realize a projection you can see in the chart..
Estimate of number of transfers of 0.001 SBD/STEEM
According to this simple estimate, the number of milli-transfers (0.001 SBD/STEEM) would be reduced until 2018-10 and then go back up stratospheric until 2019-06. I have not calculated the values in number of transfers because I think the graph gives a good idea of the proportion and because this is just a hypothesis
Once we have an overview of the behavior in the volume of the milli-transfers (0.001SBD/STEEM) we will investigate the accounts that are related with these transfers.
2.3 Who are sending and receiving these kind of transfers?
At this point I have made a query that covers the period of time from December 2017 to April 2018 in order to obtain the most updated behavior obtaining a total of 1,810,190 transfers made by 32,629 accounts and received by 129,288 accounts.
Note that the same account can be both transmitter and receiver of transfers as it happens with some of those that appear in the two following lists.
TOP 20 :: FROM or ISSUERS :: Accounts
TOP 20 :: TO or RECEIVERS :: Accounts
As you can see these accounts are mainly bots which indicates that the volume of the milli-transfers (0.001) is due in great part to the interaction between bots and their users.
2.4. Categorization of the milli-transfers by the content and/or use of the MEMO field
Without judging, at the moment, the behavior of these bots I am interested in finding those transfers that can be considered Direct Messages for what I have proposed this classification of transfers by their MEMOS.
- Empty + CODES
- URL - RESQUEST to bots
- TO @NULL (promotion)
- ENCRYPTED
- SPAM
- REMAIN (*)
(*) Remain category represents the remaining records after filtering the raw data eliminating those belonging to the other groups.
Let's briefly describe the characteristics of each category and filter the data sequentially.
2.4.1. 'Request to bots'
This type of transfers contains in the MEMO field the URL of the post to promote, ReSteem, etc. The total number of transfers whose MEMO start with http are: 218,427 , that is 12.6%of the total and represents the effort of post promotion by users in services offered by bots.
It can be seen that the num_FROM is much larger than the num_TO since many users send many URLs to a relatively much smaller number of bots.
CHART. Request to bots
In March-2018 there has been a peak in the number of users using these services. The striking thing is that some increases in the number of transfers (in green) have occurred when the number of users did not grow significantly, which indicates a great increase in the activity, that is, the same accounts sending more requests.
2.4.2. Memo Spam
The general strategy of MEMO SPAMMERS is to send the 'transfer - direct message' as if it were "A gift" or a "Donation" and offer their services where usually includes a @ accountname, URL and words like: Promote, Send, Resteem, Delegate, Follow, Upvote, Bid, Bet, Send , Offer, SBD, STEEM in the memo field.
According to this filtering, 1,349,572 (75.55%) of the milli-transfers can be classified as SPAM. Which is equivalent to 18% of all transfers of any value.
CHART. Memo Spam.
In this case the num-FROM (almost imperceptible in red in the graph) is much smaller than the num_TO since the SPAM is directed from a few accounts to many receiving accounts of those unsolicited messages or milli-transfers.
There was a peak in the Memo Spam Activity in 2018-03 and now seems to be tending to drop.
Maybe because MEMO SPAMMERS have not achieved the profitability they were looking for or maybe because they are preparing for another peak of activity.
2.4.3. Encrypted Memo
The milli-transfers (0.001) that were better candidates to be genuine direct messages between users are those in which the MEMO field is encrypted since this provides privacy.
The data shows that there were only 30,047 encrypted transfers which represent a very small percentage (1.66%) of the total of milli-transfers.
CHART. Encrypted Memo
No more than 1k unique users a month have used them and no sustainable growth is appreciated. I imagine that the reason for this small numbers is that only a few users know this possibility.
2.4.4. Remain
The remaining data, after filtering the previous components, represent 5.83% of the milli-transfers.
The contents of the MEMOS are basically welcome messages, registrations confirmations, greetings, acknowledgments, tests, requests for votes, phrases in other languages such as Russian, Korean, Chinese, Arabic.
CHART. The Remain
Regarding the proportion of the accounts of issuers and transfer recipients, we can see the disproportion between num_FROM and num_TO starts to grow from December-2017 to remain more or less constant later.
Summary of percentages by MEMO category
With these results it can be said that the use of the transfers as direct messages would be those belonging to the categories of
- Encrypted 1.66%
- Remain 5.83%
- SPAM 75.55%
This would relegate to a very small percentage what could be qualified as genuine direct messages formed by those belonging to the Encrypted + Remain categories, which add up to 7.49% compared to 75.55% of SPAM category.
Is MEMO SPAM a problem?
Beyond the fact that the transfer history can be full of memo SPAM preventing the users who receive the spam from seeing their own transfers I wonder if it also implies a cost in sending bytes to through the network and storage in the blockchain.
Top 25 Users by Cumulative Bytesize in Transfer Memos by @netuoso.
What is certain is that when SPAM is associated with SCAM, it is a serious problem as I comment in the last section of this analysis.
A case of SCAM using MEMO SPAM
In the table of the top20 of issuers of milli-transfers I was able to detect two accounts @steempromos and @alesno1 that during the period [2018-03-11] to [2018-03-25] sent 20k transfers of 0.001SBD (20SBD) promoting false services that promised stratospheric returns like:
Send from 1 to 10 SBD or STEEM to @steempromos to subscribe to 🐋SuperWhale🐋 service which gives out 700% to 850% returns! Spots are limited so act fast! (TIER 1 = 1 - 3 SBD OR STEEM, TIER 2 = 4 - 9 SBD OR STEEM, TIER 3 = 10 or more SBD / STEEM)
Using a very old and known bulk-email strategy in which it is expected that the small percentage of responses will be compensated by a very high volume of attempts to obtain a benefit from the few that fall into the trap.
Although its appearance could be detected as SCAM for an average user, there were 55 users (as some say it clearly) who were scammed, obtaining 69 SBD from them. Fortunately, these accounts have stopped performing this activity because perhaps they thought that a response of 0.275% is less than expected. These figures may seem ridiculously insignificant but they represent a benefit of 345%.
Some possible actions to reduce spam could be:
- Raise the minimum amount necessary to send a transfer.
- Limit the number of transfers per account.
3. SUMMARY AND FINAL CONCLUSIONS
This is an analysis of the use (and abuse) of the MEMO field available in the transfers as a direct message between accounts.
As a first approximation, I analyzed the behavior of milli-transfers whose AMOUNT = 0.001 SBD / STEEM
In volume of transfers represent 24% of the total and observing its evolution over time, a very particular behavior is detected in the form of bursts or impulses of increasing magnitude.
These impulses are due to a large increase in the activity of the issuing accounts.
I have performed an estimation exercise on the volume of these transfers in the future, which shows that after a hypothetical decrease until 2018-10, another much higher explosion period can be initiated that reaches 2019-06.
Knowing who are the most active accounts in this behavior, it is explained that these impulses (that grow exponentially and fall by 30%) in the activity are mainly the result of the interrelation between different bots and their users that seem to pass through fluctuating stages of activity.
Subsequently, I classified the content of the MEMO of the transfers in categories to analyze them separately observing a very high proportion of the SPAM category. which can be used as a way of SCAM, as I have shown with an example that achieves 345% yields.
In conclusion, it can be said that the use of what I have called Genuine Direct Messages is minor. Maybe tools like Steem Chat-Wallet (by @therealwolf) can help popularize since it makes use of this feature.
4. SCOPE, TOOLS & CODE
SCOPE
- Raw Data 1 from 2016-3-30 to 2018-4-25
- Raw Data 2 from 2017-12-01 to 2018-4-25
TOOLS
To get and process data from the Steem database I used dBeaver a free and open source multi-platform database tool for developers, SQL programmers, database administrators and analysts
Infogram to create charts.
CODE
SELECT
sbds_tx_transfers.timestamp, sbds_tx_transfers.from, sbds_tx_transfers.to, sbds_tx_transfers.amount, sbds_tx_transfers.amount_symbol, sbds_tx_transfers.memo
FROM sbds_tx_transfers
WHERE sbds_tx_transfers.amount = 0.001 and sbds_tx_transfers.timestamp > '2017-12-01 00:00:00'
Hey @lokomotion
Thanks for contributing on Utopian.
We’re already looking forward to your next contribution!
Contributing on Utopian
Learn how to contribute on our website or by watching this tutorial on Youtube.
Want to chat? Join us on Discord https://discord.gg/h52nFrV.
Vote for Utopian Witness!
Congratulations @lokomotion! You have completed some achievement on Steemit and have been rewarded with new badge(s) :
Click on any badge to view your Board of Honor.
To support your work, I also upvoted your post!
For more information about SteemitBoard, click here
If you no longer want to receive notifications, reply to this comment with the word
STOP.
hi @crokkon,
The notifications, SBD giveaways, etc. DO NOT appear in the "spam" category but in the category that I have called REMAIN (after filtering out the other components) that represents only 5.83% of the transfers of 0.001 SBD/STEEM
In the SPAM category there are all those that clearly are unsolicited direct messages and try to convince the receiver in a more or less direct way to send some amount of SBD/STEEM.
The peaks of increase in the number of transfers of 0.001 SBD/STEEM do not coincide exactly with the waves of new accounts. Although they must be related, there is at least a temporary delay. I believe that this is due to the fact that Spam is aimed especially at new accounts but take at least a few weeks a user to start to investigate the possibility of obtaining profits through the use of bots, and the "memo Spammers" probably knows this.
And, of course, my prediction about the next wave of milli-transfers is a daring in this uncertain world that should be taken as a hypothesis.
.
Hey @lokomotion
Here's a tip for your valuable feedback! @Utopian-io loves and incentivises informative comments.
Contributing on Utopian
Learn how to contribute on our website.
Want to chat? Join us on Discord https://discord.gg/h52nFrV.
Vote for Utopian Witness!