[Guide] Steem Windows Mining in a Linux Virtual Machine

in #steem8 years ago (edited)

Full Steem Ahead!

mining

Originally I wrote this guide because the early Windows builds were very slow at mining. Now they are just as fast as Linux mining, but because they are made by volunteers and not officially by the Steem team (at least not yet), some might want to be secure by downloading and compiling the Steem code straight from github while still running Windows. Another reason to run a steemd node in a Linux VM is that the updated Windows builds are not always available past a hard fork. Furthermore, Part 2 of this guide serves as a step-by-step guide to using Ubuntu 16.04 which differs slightly from 14.04 in that we install libboost from the Ubuntu repository instead of compiling it ourselves, which saves time. Now to the guide.

Right click-> View image for any image that you'd like to see bigger.

Part 1: Install Ubuntu Server 16.04 in a VM and connect to it with a terminal

Step 1.

We're going to use a virtual machine (VM) within Windows, so first things first, verify your CPU's virtualization features are enabled. Most CPUs made in the last decade have hardware virtualization but sometimes they are not enabled in your BIOS configuration. Intel calls it VT-x, and AMD calls it AMD-V. To check this, Microsoft provides the Hardware-Assisted Virtualization Detection Tool to check this, and you can also use CPU-Z portable to see if VT-x or AMD-V is enabled as well.

HAVDT CPU-Z
Success, hardware virtualization is enabled

HAVDT fail
Fail, virtualization needs to be enabled or your CPU does not support virtualization

If your CPU supports it but virtualization isn't enabled, you likely need to enable it in your computer's BIOS/UEFI. Refer to your computer or motherboard manual for how to do this, but it's generally an easy process of rebooting your computer, pressing the Del or F2 key, and hunting around the menus for something that mentions virtualization.

Step 2.

Next, begin downloading the Ubuntu Server 16.04 iso:
ubuntu-16.04-server-amd64.iso (torrent link) [655MB]
SHA256 sum: b8b172cbdf04f5ff8adc8c2c1b4007ccf66f00fc6a324a6da6eba67de71746f6

Then download and install VMware Workstation 12.1.1 Player for Windows [70MB]

Step 3.

With VMware Player installed and the Ubuntu Server .iso fully downloaded, launch VMware Player and click Create New Virtual Machine. The first screen of the wizard asks you for an installer disc image file, navigate to and select the Ubuntu .iso you downloaded. Enter in the user and password for the linux installation, (the pass doesn't have to be particularly strong, also write these down just in case you forget!) [next], and select where you want the virtual machine files to be stored (ensure there's plenty of space on the disk) [next]. Set a maximum disk size, if you have the space 20GB will be fine for a while. You can set it lower to conserve space and if you don't plan on mining long-term [next].

click customize

Stop right there! Click Customize Hardware. If you have plenty, set your memory to 2GB. The VM process will only use as much RAM as it needs. Set the number of processors to the number of threads CPU supports. This is equal to the number of cores any AMD or non-i7/Xeon Intel CPU has. If you have an i7 or Xeon you have hyperthreading, so set the processor count to the number of threads your CPU supports. For AMD and Intel, if you're not sure how many cores to set, press CTL+Shift+ESC for the Task Manager, click the Performance tab, and count how many boxed graphs you see under CPU usage history.

Customize

When you're done click Close. Leave the "Power on this virtual machine" option checked and click Finish.

Step 4.

The VM will start and the Ubuntu Server installer will boot. VMware will do an "Easy Install" which will breeze through it and reboot the VM. Eventually you will be greeted with a black terminal with the login prompt.

terminal
Yay! Installed and ready to go

Step 5.

You can run steemd and use cli_wallet straight from the VMware terminal but for a few reasons I recommend using a terminal client. KiTTY is an updated version of PuTTY, a simple but powerful oldschool Windows terminal client. Download it here. Run it, and it will open a Configuration prompt that needs a hostname or IP. To connect to the VM you'll first need to install OpenSSH, go back to your VM terminal and enter sudo apt-get -y install openssh-server and enter your password. Then type ifconfig

ifconfig

I've highlighted in the red box what you're looking for, the local IP address for your VM. [1] Enter that IP address into the KiTTY configuration and make sure the SSH button is selected. [2] Under Load, save, or delete a stored session type in Steem (or whatever you like) at the top of the box and [3] click Save. Here you can retrieve the session by clicking Load next time. [4] Finally click the Open button.

KiTTY config

The first time you connect you will get a "KiTTY Security Alert" prompt regarding the SSH key, click Yes. You will have a terminal screen open that you need to log in to. Enter your username and password.

Part 2. Install and configure Steem

Note with PuTTY/KiTTY and many other terminals, right-clicking will paste text.

Step 6. Update + install dependencies

Logged in to your terminal, enter sudo apt-get update and enter your password. You might see some errors about the CD-ROM, ignore this. Then enter

sudo apt-get -y upgrade && sudo apt-get -y install git cmake g++ python-dev autotools-dev libicu-dev build-essential libbz2-dev libboost-all-dev libssl-dev libncurses5-dev doxygen libreadline-dev dh-autoreconf screen

This downloads all the software that Steem needs to build and run, plus screen which makes interacting with multiple programs easier.

Step 7. Build

Now we're ready to build and run steemd! Next enter this:

git clone https://github.com/steemit/steem && cd steem && git checkout master && git submodule update --init --recursive && cmake -DCMAKE_BUILD_TYPE=Release -DLOW_MEMORY_NODE=ON . && make && cd programs/steemd/ && ./steemd

You will see some warnings while it's compiling but it's safe to ignore them.

Step 8. config.ini

When the build is done with the above command, steemd (the program that runs the Steem node) will start but it can't do anything without a configuration file yet. Press CTL+C to stop steemd and then enter cd witness_node_data_dir.

Optional but recommended: Witness @steemychicken1 offers an up-to-date blockchain download here: http://steemitup.eu/ which can make syncing much faster. If you somehow find yourself out of it, navigate to your data dir cd ~/steem/programs/steemd/witness_node_data_dir then wget http://steemitup.eu/witness_node_data_dir.tar.gz && tar xvzf witness_node_data_dir.tar.gzand the blockchain will be in the right place.

To edit your config.ini file type nano config.ini which opens it in the command line text editor nano. Tip: the terminal will auto-fill the directory or filename when you press Tab. So to perform the above commands you can simply type cd wit [tab] [enter] nano con [tab] [enter] or even abbreviate the names to just the first letter.

config.ini

Within nano you can navigate using your arrow keys, home, end, page up/down etc. Again you can paste with right-click. Here's what you'll want to enter in config.ini:

First copy and paste the whole list of Steem seed-nodes, available from steem-id's post here https://steemit.com/steemhelp/@steem-id/updated-steem-seed-node-list

Then add rpc-endpoint and witnesses:

rpc-endpoint = 127.0.0.1:8090

witness = "youraccountname"
witness = "yourotheraccountname"
witness = "etc.etc"

Because of the mining queue system, you will want to make at least a few accounts for mining. Once a miner account gets queued, the machine can't mine for it anymore and will otherwise sit idle, so you have a handful of accounts as secondary, tertiary, etc accounts for when the above accounts are already queued. Something to keep in mind is steemd prioritizes accounts by alphabetical order. So if you want your primary account to mine the most SteemPower, make sure your secondary accounts come after it in alphabetical order. In the example above, the etc.etc account would be first to be submitted when steemd solved a proof-of-work despite being listed last. If you are making a new account by mining, make sure it is not already taken by visiting https://steemd.com/@your-desired-account and seeing "Page not found".

When mining, for every witness account you set, you need a miner = line as well.

miner = ["youraccountname","5WIFKEY"]
miner = ["yourotheraccountname","5WIFKEY"]
miner = ["etc.etc","5WIFKEY"]

You may use the same keys for different accounts as long as you are mining them on the same machine. Personally I use different keys for every account.

For all of your keys be sure to back them up, print them, etc.

If you have an account already, made with a Facebook or Reddit verification, you can get the key you need to mine to that account with by going to your Permissions page on your Steemit profile: https://steemit.com/@yourusername/permissions and clicking show. Side note on Steemit password security: How to login to Steemit with your posting key (and why this is important).

For new accounts, you can obtain secure WIF keys by going to https://bitaddress.org and saving the HTML to your computer. You then open it locally, do the mouse entropy, then click Bulk Wallet, uncheck the compressed address option then click Generate. The WIF keys are on the right, below.

bitaddress

Finally you must set mining-threads = X where X is the number of threads your CPU supports discussed earlier. Don't set it for less, we're going for max performance remember?

mining-threads = X

Press CTL+O to save, press enter to keep the same config.ini filename, then press CTL+X to exit.

Part 3: Run steemd and mine!

Step 9.

You'll be in witness_node_data_dir so type cd .. to go back to the steemd/ directory and then enter screen -t steemd ./steemd and hit enter. If all goes well steemd will start and it will begin to sync or replay if you downloaded the blockchain via .zip. You will see many messages scroll by that include hash rate: 1 hps. When scrolling fast, this means it is syncing the blockchain.

syncing

From here you can either wait or if you want to check on the progress on the sync you can make another screen by pressing CTL+A then C. Then start the cli_wallet with cd ~/steem/programs/cli_wallet/ and ./cli_wallet. If you wish to use the command line wallet you can make a password, but it's useful just to see where you are in the sync. Type info and look at the line near the bottom that says "head_block_age":. This is how far behind in the chain you are. Once that time is down to the seconds, you should be synced. To navigate to the previous screen in console, press CTL+A twice in a row. For navigating more screens, press CTL+A then " and select from the list.

Once synced, your node should start mining and look something like this:

mining
Yay mining!

Step 10. Optionally set CPU priority

To keep your Windows machine more usable when running a mining node, you can optionally set the CPU process priority for the virtual machine process. Press CTL+Shift+ESC to open Task Manager. Find the virtual machine process then right-click -> Set Priority -> Below Normal (or) Low. When idle, your machine will mine as fast as it can, but when programs need the CPU, lower priority will allow them a little more breathing room.

priority


That's it! Easy huh? :P

When you solve proof-of-work you are entered into the mining queue. You must leave your node on while queued to actually get the SteemPower reward when you generate the block and leave the queue. To track your mining you can visit your user page on steemd.com and also see the https://steemd.com/witnesses page to view the whole miner queue. Be sure to track how many SteemPower you accumulate on your mining accounts as well to make sure they are actually generating the blocks.


Thanks to the many Steem users who helped me accumulate the knowledge of this process and thanks to @tuck-fheman for prompting me to write this guide.

If you found this or my other posts helpful, click here for my blog page and hit the follow button in the upper right!

Sort:  

⚠️ This post is out of date!

Steem mining, proof-of-work, is no longer part of Steem's block creation. It is purely a delegated-proof-of-stake blockchain network now.

Why steem cannot be mined anymore?

Mining (PoW to be queued for the 21st witness slot in each round) was removed in version 0.17.0 because it was being exploited and it was decided that the work to fix it was not worth the effort.

@pfunk Can I mine steem if I become steem witness now?

Best guide ever!!! It still works as of my post (Windows 7 AMD). I would suggest adding a FTP\GIT download of the blockchain if possible.

Great effort looks like a lot of effort went into writing this guide, i will give it a shot when i get home! :)

Nice job pfunk!

Thanks :) I only wish I didn't get dissuaded from writing it earlier :P

-snip-

Never mind, it synced 5 minutes after asking. :)

Interestingly, I get a much better hashrate using bitcube's Windows miner than going this route. I expected this to be on par with arhag's Windows miner. Is there some setting in the VirtualBox or Ubuntu that I need to set perhaps? For instance, is the VirtualBox or Ubuntu limited to less threads than I would be using versus the Windows miner?

Update: I tried the "Hardware-Assisted Virtualization Detection Tool" but it does not work in Windows 10, so I had to use CPU-Z (and it's on).

Update 2 : DOH! When I setup VirtualBox I did not tell it to use more than 1 processor. Fixing. See, I should have waited for your guide to do this. ;)

Update 3 : Hellz yeah! Now we're cooking with Crisco! ;) 12500 hps! Thanks for the guide pfunk! I will be sending big SBD tip on 7/4! I added a link to your guide from the Windows guide for those who are having issues with arhag's Windows miner but want more speed than bitcube's Windows miner offers.

Thanks I really appreciate that! Glad you're running at full Steem :p

what did you get through the windows route?

That comment was when we were mining on v0.5.0 so is no longer applicable. But my hps via the Windows miner is on par with my previous results under the VM.

wow got 100% increase.
http://imgur.com/3cAUQNp
mining-threads = 4

Excellent guide, thank you for posting it.
Once you have mined a block and have a new account it shows 1 steam here on steamit. In the cli_wallet it shows 7700+ vests in the account. I tried the withdraw_vesting command, but that says it will power them down and takes time. Is there a way to just transfer the vests to steamit?

Steemit displays your SteemPower, or VESTS as the cli_wallet calls them, as STEEM, the liquid currency that is tradeable. What it really means is what your SteemPower's current value in STEEM is, because it's not liquid as you found out. If you go to steemd.com you can see on the top of the right-hand box that steem_per_mvests = 129.229. 1 million divided by 129.229 equals 7,738. So that's where that VEST number comes from. https://steemd.com/@hickorysmoked shows your actual SteemPower/VESTS in million-units.

What should I do with it, do I need to move it or power it down to keep it without that wallet file?

The wallet file only keeps your keys imported so you can perform blockchain actions. The key(s) for your account(s) is/are what you need to keep safe and backed up.

What you do with your SteemPower is up to you but the more you have the more you'll get in curation rewards from upvoting good posts so that's a consideration. And so far you have ~1 STEEM worth of SP (soon to be 2 though since you entered the mining queue again!) so 1/104th divestment weekly wouldn't amount to much.

re: @hickorysmoked above

You don't need to move or transfer anything with the cli_wallet. It's safe in your account which is controlled by your key(s). The keys can be used to log in to Steemit.com and you may go from there.

I think I understand but the vesting ones do not show in the steemit wallet, so I don't quite understand how it will benefit me. I just wanted to make sure I wasn't supposed to move it\transfer it\withdraw it before turning off the miner.

Thanks very much. But one quastion. What about NVIDIA cards? Can I mine with nvidia cards?

There are as yet no GPU miners for Nvidia or AMD. Be very wary of anyone saying they've made one too.

This guide is incredible -- very well laid out and organized (and pictures too!). Very noob friendly. I'm going to give this a shot in the morning , since I'm a total linux noob. Great work pfunk!

yup, it's windows user quality and by that i mean idiot proof (i am a windows user by the way)

It's very satisfying to get it going. I've been so intimidated for years about installing Linux that I've never done it. Hanging with all these smart dudes lately gave me the itch and confidence (that I had someone to ask questions of) to finally give it a shot. Everyone in the Steem slack is very helpful if you run into any issues.

Wow. Fantastic! Thank you for taking the time to put this together.

Warning! mining STEEM is not possible anymore.
Save your time.

Can I mine STEEM?
No. Proof of work mining has been removed from Steem.
https://steemit.com/faq.html#Can_I_mine_STEEM

thanks !!! justed wested 4 housrs for settings in config file and learn about mining .

now I am getting this when it was building the blockchain http://prntscr.com/bu3ha1 do I just wait?

unlinkable block could be due to hard forck. Check witnesses on steemd.com and see what version they are running. If there is a new version, you need to grab it from git and reinstall.

I'm just going to start over again (facepalm)

Indeed need to start over, 0.11.0 is the latest right now, no error this time.

Same problem

I went through all comments, and it looks like I'll be the first one to ask: which mining algorithm is used to mine steem? Is it SHA256 or scrypt?

It can be found here: https://steem.io/documentation/consensus/

It's neither Bitcoin's SHA256d or Scrypt, so no existing ASICs can be used to mine Steem.

Fantastic Article, currently loading the blockchain to my VM. Hopefully my i7 4790 still packs a punch!

hi pfunk, i followed your guide and make it work, but the problem is i 've never get any POW for 2 days, and i'm running Core i7-5820K 3.3Ghz

It also happens to me the same, it seems that the queue number always gets pushed back to the end, and never seem to produce a POW.

how long does it takes to sync the blockchain?

it took about an our

nice, i have been on this for three hours now. It's a mess, so many errors one after another. If i can finish it , i'll do it again but on video for others.

Nice!

I'd be interested to see if you actually see a performance improvement, however, from setting the number of cores lower than the maximum number (at least when you're using the host machine at the same time)... Reason being that in most hypervisors (I could be wrong here, so someone please correct me if I am...) the VM will only be scheduled CPU time when all cores are available concurrently.

If you're using the host machine at the same time, this is going to mean that whenever you're running even a single process with a single thread, the entire VM is off-CPU because in order for the guest OS to do that multithreading, it needs its full contingent of cores simultaneously (having no real awareness of the host system).

Again, my previous disclaimer about possible factual inaccuracy stands (I don't know what work has been done on hypervisors here within the last few years, and I wouldn't be surprised if VMWare player runs completely in userspace and therefore does some crazy weirdness), but it would be nice to see a comparative load test if you're set up for it!

All I know that on my 4690K I get exactly the hashrate I should be getting in the VM (compared to the same generation chip running only Linux) while everything else is idle.

Ya I noticed very little slowdown speed wise.

Cool! I wonder if VMWare player is doing anything particularly clever or if the Windows scheduler is just doing a really good job... Probably the latter to be fair.

If I'm trying to use an existing account, which WIF key/permission should I use? Owner, Active or one of the others?

If you have an account already, you can get the key you need to mine to that account with by going to your Permissions page on your Steemit profile: https://steemit.com/@yourusername/permissions

Sorry for not being more specific there. Use the active key.

Quick question how or where do I get another account from without actually making one, if I just input "will-zewe1" will it automatically make a new account for mining? I know how to get the additional keys off bitaddress I'm just not sure where the account name comes from...

if I just input "will-zewe1" will it automatically make a new account for mining?

Yes, so long as nobody's made the same account name yet. I think you're safe.

Awesome! Thank you, I thought I saw that somewhere else but I couldn't retrack it down to verify so figured I'd just ask.

Every time there's a problem with the network (like when hacking occurred or this morning when there was interruption for an hour) my miner stops. When i try to run it again, it's not working. Instead it says on the first line error parsing logging config from the config file (config.ini). I haven't changed that since initial setup. And then the miner starts, but it gives all errors.
It says error when pushing block, unlinkable block exception, block does not link to the known chain. Once again I didn't change anything, just restarted the miner after the miner stopped working. Is there anything that I can do to fix that?
Thanks

same here- everytime I stop and restart I have to delete the witness node data dir directory - wait for it to build all OVER again.... before it mines- otherwise it's stuck on block "0" with errors

what blok are steem right now ?

Problems with this error code with building steem:

FC_ASSERT( hardfork == hfp.last_hardfork + 1, "Hardfork being applied out
   ^

What to do?

I had these issues too... what I did was delete the directory, start over, follow this https://steemit.com/steem/@pfunk/guide-maximize-your-mining-hashrate-in-windows-by-mining-steem-in-a-vm follow the step 6, 7, and then on step 8- download the blockchain BUT make sure you are in steem/programs/steemd NOT the one he says .... then type wget http://steemitup.eu/witness_node_data_dir.tar.gz next get 7z type apt-get install p7zip-full then type 7z x witness_node_data_dir.tar.gz then edit the config with the rest of the directions

Good catch, I need to edit the post but I can't right now. I'll update when I can.

thanks, cool :)

Thankyou! Syncing it now

I have the same thing, while I am reading your reply and try to find a solution, the build process continues.

hi i have followed these instruction to the t several times and i can not get this to work for me the program starts then immediately crashes the the first time it start to load blocks but they were all red after that it just crashed

I had running miner. But tell me where to see how many Steem that miner is working?

Hi pfunk.
When i check cli_wallet ==> info. I find the line "current_witness=xxx" but it is not my account name. Explain me about this line. Thanks

Great guide, It has been fun to get it all together.
If anyone has any advise? I have made it to step 9 and entered "screen -t steemd ./steemd" It did not go so well. When I click the enter key the screen flashes and I get the message [screen is terminating]
I did start over twice and received the same flash and message.
Aloha and Mahalo!

The same problem and just sloved it. Check the "" at line witness name, or WIF key

Bilder hochladen can someone help me to fix it?

I have done this, have a version thats up to date, but when its replaying the blocks its not applying HARDFORK 12.
It's just stuck on 96 % of replay.
Whats going on?

same here.

Windows mining works. Follow this guide https://steemit.com/steem/@bitcube/steem-mining-in-microsoft-windows-a-miner-s-guide-part-2
I have managed to start it successfully . Just download the latest steem release from github (0.12.2), not the 0.8.5 version as it says in the text.

If I made a new account for mining on the Steemd program, do I also have to create an account on the actual website for it to store my Steem Power gained from mining?

No, it's automatically created when POW is found

Noob question: The output of the make command creates steemd and cli_wallet; if I took this files and copy them to different computer in /usr/bin/

  1. will steemd work correctly ? I guess there will be no problem but I want to confirm
  2. will I see decrease in hbs?

Very easy to digest. Great post. Moving from windows host to ubuntu server mining now.

nice guide

Thank you. It's entirely irrelevant and outdated now though. Steem does not have any PoW any more. :)

I know this post is old, however, I am having some issues with step 8. I have done all of the steps before it however when I enter...

 cd witness_node_data_dir

I get the error saying "no such file or directory found"

Can you please help me?

Ok first of all, mining is no longer possible with Steem, and these instructions are also out of date in a couple ways.

It may point in the general direction of successfully building still, but understand that if you want to build a node now, you should do a git checkout of the latest release tag (not master) before doing the git submodule update step.

The witness_node_data_dir is created after running steemd for the first time. So run it, give it a second, and then quit it with CTRL+C. Then there will be that directory with config.ini within.

Really nice guide, thanks for this!

Excellent! Thank you so much!

Guide is outdated/no longer valid because there is no mining in Steem anymore, but you're welcome anyway. Is this guide being linked to somewhere? There have been some newer comments on it.

very good post. thanks for sharing.

is this guide still up-to-date or usable ? thanks for your tutorial

@pfunk thanks for such amazing documentation and help.

Quick question - I am following the process mentioned here for windows mining of steem. I have configured the file as stated and once I run the exe file it shows some red text and immediately closes out. Any pointers on this? I have used only 1 CPU thread for mining, and I am using it on my Laptop.

Steem PoW mining is no longer possible. The network has moved to a purely DPOS consensus model. Though the old top comment thread was about this, I've made a comment at the top now that makes it more clear.

Thanks for your response @pfunk. Is virtualbox mining still available?

Mining is not possible.