EOS.IO Transaction Structure - Developer's Log, Stardate 201707.9

[dan]

Today I would like to take a moment to explain the current structure of an EOS.IO transaction so that developers can better understand the concurrency model. Below is a JSON representation of a transaction that will transfer currency from sam to alice. In this case, currency, sam, and alice are all account names; however, they are used in different ways.

{
  "refBlockNum": "12",
  "refBlockPrefix": "2792049106",
  "expiration": "2015-05-15T14:29:01",
  "scope": [
    "alice",
    "sam"
  ],
  "messages": [
    {
      "code": "currency",
      "type": "transfer",
      "recipients": [
        "sam",
        "alice"
      ],
      "authorization": [
        {
          "account": "sam",
          "permission": "active"
        }
      ],
      "data": "a34a59dcc8000000c9251a0000000000501a00000000000008454f53000000000568656c6c6f"
    }
  ],
  "signatures": []
}

When serialized to binary with a single signature, this transaction is about 160 bytes in size which is slightly larger than a Steem transfer which is about 120 bytes or a BitShares transfer which is about 94 bytes. Much of the extra size comes from having to explicitly specify recipients, authorization, and scope which collectively add 51 bytes to the message.

TaPoS - Transactions as Proof of Stake

Those of you familiar with Steem & BitShares will recognize the first 3 fields of the transaction; they remain unchanged. These fields are used by TaPoS (Transactions as Proof of Stake) and ensure that this transaction can only be included after the referenced block and before the expiration.

Scope

The next field, "scope", is new to EOS.IO and specifies the range of data that may be read and/or written to. If a message attempts to read or write data outside of scope then the transaction will fail. Transactions can be processed in parallel so long as there is no overlap in their scope.

A key innovation of the EOS.IO software is that scope and code are two entirely separate concepts. You will notice that the currency contract is not referenced in the scope even though we are executing a transfer using the currency contract's code.

Messages

A transaction can have one or more messages that must be applied in order and atomically (all succeed or all fail). In this case there is exactly one message, so lets look closer at the message:

code:

Every message must specify which code it will be executing, in this case the currency contract's code will be executing resulting in the following method being called:

currency::apply_currency_transfer(data)

type:

The type field defines the type of message (and implicitly the format of data). From an object oriented programming perspective you could view type as a method "name" on the "currency" class. In this example the type is "transfer" and hence explains the naming of the method being called:

${namespace}::apply_${code}_${type}( data )

In case the "namespace" is the currency contract; however, this same method apply_currency_transfer may also be called in other namespaces.

recipients:

In addition to calling currency::apply_currency_transfer(data), the method apply_currency_transfer(data) will also be called for each recipient listed. For example, the following methods would be called sequentially in this order:

currency::apply_currency_transfer(data)
alice::apply_currency_transfer(data)
sam::apply_currency_transfer(data)

The notation account:: specifies the contract which implements the method. alice and sam may choose to not implement this method if they don't have any special logic to perform when currency::apply_currency_transfer is executed. However, if sam was an exchange, then sam would probably want to process deposits and withdraws when ever a currency transfer is made.

The person who generates the transaction can add any number of recipients (provided they all execute quickly enough). In addition some contracts can require that certain parties be notified. In the case of currency both the sender and receiver are required to be notified. You can see how this is specified in the currency contract.

void apply_currency_transfer() {
   const auto& transfer  = currentMessage<Transfer>();
   requireNotice( transfer.to, transfer.from );
   ...
}

authorization:

Each message may require authorization from one or more accounts. In Steem and BitShares the required authorization is implicitly defined based on the message type; however, with EOS.IO the message must explicitly define the authorization that is provided. The EOS.IO system will automatically verify that the transaction has been signed by all of the necessary signatures to grant the specified authorization.

In this case the message is indicating that it must be signed by sam's active permission level. The currency code will verify that sam's authorization was provided. You can view this check in the example currency contract.

void apply_currency_transfer() {
   const auto& transfer  = currentMessage<Transfer>();
   requireNotice( transfer.to, transfer.from );
   requireAuth( transfer.from );
   ...
}

data:

Every contract can define it's own data format. Without the ABI the data can only be interpreted as a hexadecimal data; however, the currency contract defines the format of data to be a Transfer struct:

struct Transfer {
  AccountName from;
  AccountName to;
  uint64_t    amount = 0;
};

With this definition in hand we can convert the binary blob to something similar to:

{ "from" : "sam",  "to": "alice",  "amount": 100 }

Scheduling

Now that we understand the structure of an EOS.IO transaction, we can look at the structure of an EOS.IO block. Each block is divided into cycles which are executed sequentially. Within each cycle there are any number of threads that execute in parallel. The trick is to ensure that no two threads contain transactions with intersecting scopes. A block can be declared invalid without reference to any external data if there is any scope overlap among the threads within a single cycle.

Conclusion

The single biggest challenge to parallel execution is ensuring the same data is not being accessed by two threads at the same time. Unlike traditional parallel programming, it is not possible to use locks around memory access because the the resulting execution would necessarily be non-deterministic and potentially break consensus. Even if locks were possible, they would be undesirable because heavy use of locks can degrade performance below that of single threaded execution.

The alternative to locking at the time data is accessed, is to lock at the time execution is scheduled. From this perspective, the scope field defines the accounts a transaction wishes to acquire a lock on. The scheduler (aka block producer) ensures no two threads attempt to grab the same lock at the same time. With this transaction structure and scheduling it is possible to dramatically deconflict memory access and increase opportunities for parallel execution.

Unlike other platforms the separation of code (currency contract) from data (account storage) enables a separation of locking requirements. If the currency contract and its data were bundled then every transfer would have to lock the currency contract and all transfers would be limited by single threaded throughput. But because the transfer message only has to lock on the sender and receiver's data the currency contract is no longer the bottleneck.

Removing the currency contract from being a bottleneck is incredibly important when you consider an exchange contract. Every time there is a deposit or withdraw from the exchange the currency contract and the exchange contract get forced into the same thread. If both of these contracts are heavily used then it will degrade the performance of all currency and all exchange users.

Under the EOS.IO model two users can transfer without worrying about the sequential (single threaded) throughput of any other accounts/contracts than those involved in the transfer.

Comments

[pal]

Dan, thank you, you brought me in programming world. I am value so much that I was early involved with BitShares, because I was introduced to your talks and writings about what technology can do. It was like some catalyst for me, I started to learn programming, two years latter I left my sales job and became full time web developer. It was great gift, no appreciation of token will compare with acquired ability to earn money with your own brain. In the end, stake in crypto not so important, price could decline for years, but skills is much more tengible. Human mind creating these systems to serve humanity, we should not be slaves to them by depending on price of the token. I hope someday I could contribute code to projects like STEEM, BitShares or even EOS.

[tushar2017]

Really a great work i have seen so far.keep continue ur gd work

[williambatu]

@dan This architecture is same as AWS LAMBDA. Pay how much you use.I found serverless computating architecture may helps devs save some time to reinvent wheels.

[almost-digital]

What happens if sam is evil and implements a sam::apply_currency_transfer(data) that never returns?

[troglodactyl]

That depends on how much EOS sam buys. If he bought the entire supply of EOS, he could freeze the network until all the witnesses abandoned him. The previous holders of EOS could all thank him for the massive piles of cash he gave them, and then fork the network with sam excluded.

If he only has a small stake in EOS, his code should execute briefly (proportional to his stake) on the witness nodes and then be abandoned with minimal inconvenience to anyone.

[jonuzi]

https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSFVRDSU9IYoBFbGZ2NbhYUVCG807y3WUBKMpIO4DFm5mm79cz3

[almost-digital]

I was thinking more along the lines of that he could block any transfers involving him, but I reread that part and it seems the sender could just exclude him from the recipients list and still have the transfer go through?

[dan]

No, currency contract requires both sender and receiver to be in recipients list. Sender or recipient could implement apply_currency_transfer to auto reject every transfer.

An exchange would only block transfers that exceed user's balance with exchange contract, meanwhile other contracts may block all incoming transfers that they do not expect.

[almost-digital]

Ok I think I got it, it would be up to the contract to make sure any required recipients hook cannot be abused by a third party and in the currency case it's not a problem if either of the parties would like to stop the transaction from happening.

That's really powerful! I can't wait to try this stuff out on the testnet :)

[eikr]

Very nice answer !

[dan]

An transaction that cannot execute in a couple of milliseconds is abandoned and never included in a block.

[richguy]

I have no doubt that EOS should take centre stage in cryptocurrency discussion in a few years to come for obvious reasons.

Unlike bitcoin, EOS is a product of years of experience. @dan has successfully pulled off a number of project that are individually a success.

Steem and bitshare are obvious success story of a man' s vission.

I can bet that EOS will be a standard for anything crypto in the next 5 years.

That said, I want to bring @dan attention to the current EOS Token ICO. Many of us who are not vast in blockchain cannot dare to participate in the sale for obvious reasons.

The procedure for participation is so complex for an average investor.

I really feel something needs to be done because alot of people have been waiting for this ICO, so so seeing it passing by like this just for not having enough technical knowledge to buy is not good.

This should be the people's coin so make it accessible to the people. Thank you.

[shareme]

Here is a steemit link on how to participate https://steemit.com/eos/@thimom/how-to-use-myetherwallet-for-the-eos-ico

[bitminter]

For this exact reason i only buy on exchanges, bought a few days ago at $3, ouch hahaha :-) Thanks for the comment.

[richguy]

Which exchange did you buy from

[cryptonfused]

EOS is an upgraded like an ugrade to blockchain tech . Lets be clear here: Bitcoin, was built on the ingenious blockchain tech. Bitcoin is what led the way for more innovation and every other variation of the blockchain. To imply that bitcoin is not a product of years of experience is misleading.

Ps you can buy tokens after ICO. If EOS delivers in its promises and based on activity and consistent update report from @dan , it is most probable that it will. If it does, the future value of EOS tokens can increase significantly. In that case; purchasing it on exchanges early may prove to be just as profitable in the nxt 3 to 4 years.

[cocokoala]

I made the right choice investing on EOS,this is what brought me on Steem.

[avilsd]

Which do you believe in more?

[cocokoala]

I have faith in EOS and I have just discovered Steemit so it's really hard to decide at this point. But steemit is freaking awesome!

[avilsd]

Which do you think is the better option in your opinion?

[cocokoala]

Steemit, no doubt.

[barrydutton]

Dan you are great.

I feel I haz the dumbz every time I read one of your posts LOL.

Have a good week my man.

Ironically I wrote a post today again about Poloniex and their nonsense, I think it is my best one yet with balance and fun and facts....

I honestly thought about you the whole time I was writing that post, which took me about 2 hrs.

[hamzaoui]

Good post thanks for sharing

[booster]

This comment has received a 0.14 % upvote from @booster thanks to: @hamzaoui.

[cryptoman01]

Way to go @Dan. We wil be cheering you!

[frost04]

Parallel Execution seems very intricate. Good luck @dan keep up the good work!

[isjw84]

good!

[creativephoto]

great work

[tsteem]

Hello @dan... im beginner in steemit and I still do not know about digital currency. And some time I see your post on my homepage about eos, what is eos? thank you!

[theofilos13]

It's really hard to understand what Dan has brought in the community and how much improvement will come in the next months because of him. Thank you for the EOS, Dan!

[moneymaster]

thanx for this post ! but i have really no clue what this is about :)

Sorry, SteemON

[banglasteve]

Amazing work Dan! I know a lot of people say the technology is unproven but with you at the helm you provide the confidence we need in the success of EOS. The project is exciting and many of us are willing to see this project through. Keep up the great work!

[snowy1nl]

I am Dutch, but most of this is double dutch to me. I think it's great that you are updating the community on a very regular basis. Keep up the good work, i am following the project with much interest.

[doncute]

Nice one thanks

[robjc]

Interesting ! opening my eyes!

[midgeteg]

Nice developer log :D

[dan]

something tells me you didn't read it in less than 2 minutes.

[midgeteg]

I skimmed over it, I dont know much about logs

[timcrypto]

Great post Dan👍🏼, definitely a bright future for Eos in my opinion.

[elegente]

It's an absolutely brilliant post.

[dobro88888888]

Thanks for the positive post. Retweet.

[elegente]

Sir, may I ask why my previous reply was flagged? I left a sincere reply regarding this article.

[sirdre]

I admire this model. Very innovative.

[td4me80]

Nice breakdown! Keep steeming!

[dinasapitri]

Thanks for share @dan , although I do not really understand, because I am still a new user in this steemit.

[outhori5ed]

Great post Dan

[sam999]

really good work dan..

[dssb]

Thank you @dan, it was a very clear explanation. You made me think about getting in, altough it was not in my roadmap!

[vinylshopus]

Hey Dan please check out our steem art wallet. Let us know your thoughts!

[leobliss]

Awesome work. I really admire your work

[jrcar]

i like your post.

[kevbot]

hey @dan, i dont know why you downvoted my comment. :O, it made me lose rep, i support your channel ,work and everything you post, but that was a little disappointing to be on the blacklist. I didn't think it was very nice. all i stated was i have 1 eos :O, which is true. Been using steem for a short time period and have been an avid user since. I didn't know what i said was inappropriate. sorry if i offended you in any way ..., i'll continue to support your work even so.

[naphy]

Nice post :D I just bought EOS today waiting for my token :D

[wavemaster]

Interesting code

[aaronaugustine]

Its TRUE.....

[spicydavid2500]

Ouch you lost 1200 dollars!! Steemit you need improve their system.

[inversio]

really nice work, seems like fastest transcriptional option it has.

[beekeyyy]

This is a very good article! Still surprise what decent byte size Steem has!

[shahmaqsood]

Really a great work

[krytonika]

Very very well written and your understanding of EOS makes it much easier to relay all of its components using terms that the rest of the Steemit Community can understand - and make investing decisions more on EOS properties rather than trend graphs that have no rump or dump reasoning

[anandpoddar27]

Fantastic

[jaey-designs]

Thank you for EOS dan... 👏👏👏👏👏

[fortified]

Hello Dan. Pleased to meet you.
First, thank you for the amazing platform.

I check up on your feed from time to time to get the latest news on Steemit. I admit most of it is above my knowledge level, but I like to learn. Since signing up to Steemit in February I've really enjoyed my time here and have learned a lot, not just about Steemit itself but also about crypto-currencies in general.

One thing I did not expect though, and surprised me the most about Steemit, was how much I'd learn about myself and the world around me. This in turn has lead to fundamental changes in my life. I won't spam your comments here with it but I recently wrote a post in which I thank the Steemit community and also outline the positive impact it's had on my life. It's a great testament to your creation and I'd be happy to link you to if you'd like a read.

Sorry if this is a bit off topic here but I just wanted to take this opportunity to thank you for everything you've given us so far.

Thanks again..
@fortified

[sanees]

I have to read this multiple times to understand this.but overall the scheme looks very similar to OOPS concept.
"The alternative to locking at the time data is accessed, is to lock at the time execution is scheduled". How does the scheduler keep track of the lock used inside the program..Does it look for any specific "lock or synchronization object" tags and make sure threads sharing them are not accessing them simultaneously ...May be the threads are of same priority, otherwise wouldnt this cause deadlocks?

[sanees]

I realize lot of information is available in github.Will read up the documents and provide any comments and concerns we have.My team started out with Ethereum.But scalability and parallelism offered by EOS is very intriguing. For now exploring both

[arcange]

Congratulations @dan!
Your post was mentioned in my hit parade in the following category:

• Pending payout - Ranked 1 with $ 1259,69

[nakedchef89]

Hey, good luck on your journey in our fine community! I'll follow your account to see how you doing :). Please follow me @nakedchef89.

[oddreality]

This makes so much sense now! Thank you so much!

[lazytrades]

Thanks for keeping us updated! :)
Nice to have someone who cares for quality content!
Keep it up