Before we dive deep into to the magical world of the Boinc client communication protocols, I must tell you some things. Maybe it will be boring, I don't know. If you aren't interested in stories, then feel free to jump to the next part.
TL;DR - We will be looking into the depths of the Boinc communication. If you aren't interested in tech things from a coders perspective, then this post is not written for you.
The story
I've met Boinc firstly (exactly SETI@HOME) in about 2006. I was absolutely amazed, and I thought it's the best idea ever in the history of mankind. The idea behind the project was so fresh and new and cool. I was so proud of my SETI@HOME screensaver, and all of my friends was just blinking stupidly at me :D They absolutely didn't understand what's so cool in these colored lines...
Lots of years passed, and I have forgotten all of this thing. I found the crypto world around 2010-2012, and mined some Litecoin, because all of this new ideas and technology seemed so interesting to me. I was so impressed by seeing the computations, seeing the separate nodes forming mining pools, and sharing the interests. All of this was so... new and exciting :)
Another years passed. Meanwhile I was still forgotten about the crypto world. But at the end of 2017 I have found the Gridcoin project, and I was amazed again. Hey, it combined the two things what burned my brain out previously. WOW!
I thought I must participate.
It was harder than I thought. Gridcoin has almost zero documentation, which analyzes or describes things in depth. The Gridcoin.us website has a good base point to start as a newbie. But if you want more understanding... Then you must dig into the infinite depth of obsolete forum comments from 2015, forgotten reddit posts, etc.
Hey I didn't even know what was the difference between PoW, PoS, PoR... It was not so easy to learn the Gridcoin world from 0.
I'm an IT guy, a programmer, who is always interested in exciting new/weird/unusual technologies. I'm always curious about how things work, and/or if I'm able to reproduce it or not.
So this is where our story begins. I've found Gridcoin, I've first registered to BAM, and of course staked nothing. Then I joined to grcpool, and started to get some Grc.
And I wondered, how all these things work. I've tried several methods to 'listen in' to the communication between Boinc client and Account Managers.
And finally when it was successful, I decided to share it with the world. Because unfortunately the Boinc online documentation is only useful, when you exactly know what are you searching for. It doesn't give you any directions, where to start to write your account manager. Or maybe I was too lame to find it ;)
So... This is (partially) why this post was born. I would like to lay some foundation for my later posts.
(Yes I'm planning some series, but it's a secret now :)))
Boinc client basics
The basic idea around the whole Boinc communication is very simple.
There is the Account Manager what communicates with the Boinc Client which communicates with the Projects... This communication is flowing in both directions, and the Boinc Client is always in the center, and it maintains the connection between the Projects and the Account Manager.
Let's see out first image here:
1 - Query
So when you first try to connect to an Account Manager (AM in the following), the client first connects to http://myaccountmanager.com/get_project_config.php and it gets some basic infos about the AM, like the name, if it accepts new user registrations through Boinc, the minimum length of passwords, etc.
All of the communication is done with XML files, so it's very easy to reproduce/parse it with programs. :)
I think I will explain the exact XML queries and responses in my next post. (Maybe with some code examples?? o_O)
2 - Send data
When this response was received successfully, the client must send the user login datas to the AM. Email/username, password hash, host details (OS, number of processors, GPUs, etc...)
In good case, the login credentials are correct, and the AM should check if the host's data already exist in the database. If not, it should store it as a new host.
In the response the AM sends back the list of the assigned projects to this host (address, projects login data, etc) to the client.
3 - Login to projects
The client sends the previously received login credentials to the project servers one by one, and tries to login there. If it's successful, the project server sends back the work units, with some additional datas. (Internal ID within the project, CPID within the project, etc)
Also if there is already some finished work, then it's sent back in this phase.
4 - Send "other data"
The client login again to AM, and reports some "other data" (received from projects) to its AM, like the Internal hostID, which makes the host clearly identifiable within the project. Unfortunately the internal CPID can change within the project, so it's not a best practice to use them for host identification... o_O
This phase is almost the same as step 2, except the client sends some extra infos back. I splitted it to 2 different steps to see things clearer.
And all the things will be start again from step 2, and you'll have now a persistent connection between the AM and the Projects through your Boinc client.
What the docs didn't say
The most important thing, what all the docs forgot to mention is that the client stores the queries and the replies in simple text files... Which can be examined to see what happens in the background... I think it's the most valuable information about all of this Boinc thingy...
It makes so much easier to understand the whole communication when you can have a look at these files.
Especially if you want to make some AM, or would like to imitate its functions.
On Ubuntu linux they are located in: /var/lib/boinc-client
On windows, please see Boinc wiki
1 - AM query
get_project_config.xml - The most basic infos about AM
2 - Send user auth data
acct_mgr_request.xml - It contains the request to AM, with login credentials, host data, and possible known projects
acct_mgr_reply.xml - What the AM sends back, with project's URLs, and login datas, and signing keys
3 - Login to projects
account_<project-url>.xml - Login datas sent to projects. Separated files for each project.
sched_request_<project-url>.xml - Sent data to project
sched_reply_<project-url>.xml - Reply get from project
4 - Send "other data"
It starts again with the acct_mgr_request.xmlwhich now contains the <hostid> which uniquely identifies your host for the particular project, and some other datas.
Ok, so what's the meaning of this whole post?
I would like to give some helping hand to those guys who will find Boinc and Gridcoin in the future. It wasn't so easy to figure these things out alone, because of the messy documentation. And when you first contact with these things, you are a full newbie, and don't know where to start. There aren't any places with collected informations...
And if you are wondering, yes I will post a series of similar guides about how to connect/communicate/fetch data in this Boinc-Gridcoin world.
I think there are smart coders out there who are able to participate in the Gridcoin project, and all what they need is just pure information. A post like the above should spare me about 2 month of investigation to finish my project.
But it'll be the part of a future post. :)
Keep up the good work
Thanks, I'll try what I can :)