Hive Engine witness servers are usually ever-growing. We cannot do much if the disk space is consumed for actual data. But we can make sure the logs and other backup files don't consume too much space. Even if they do so, we can make sure those are monitored and cleared or the logs are rotated. In this article, I'm going to be explaining some of the basic checks and steps that we can take to reduce the space occupied by the log files.
At the time of writing this article, 85GB disk space should be sufficient to run the witness node. It will anyway keep growing but it is wise to check the unwanted log files that are occupying the space and remove/rotate them.
PM2 log rotate
This is the first utility we are going to install to keep the pm2 logs under control. Especially when we are running a witness node, there will be lots of logging happening that can keep growing over the course of time. Though the logs are important, we will need them only when something goes wrong. Otherwise, in an ideal scenario, we don't really have to keep the old logs.
pm2-logrotate is a utility that comes along with pm2 that can be used to rotate the logs. The npm page has more details on the configurations we can do with that. I personally think that the default configuration itself should be sufficient. It can be installed as a module to pm2 and it will appear like the above.
Rotating Mongo DB logs
The mongo DB logs will also keep growing occupying a lot of space on our witness server. The same applies here as well. We need them only if things go wrong. On all other days, we are good if we just keep rotating the logs.
mongo
> use admin
> db.adminCommand( { logRotate : 1 } )
CTRL-D
The above commands will help in keeping the logs in rotation. In addition to this, if you already have log files that are old and occupy a huge amount of space, it will mostly be in the default directory cd /var/log/mongodb. You can check this directory and remove the old log file. It will usually be a file that comes with a DateTime stamp and it can be removed with the rm command. Sometimes it might require permissions to delete this file so using sudo in front of the command should do. sudo rm mongod.log.2021- ...
Check the size of individual files inside a directory
The below command can be used to check the individual file size inside a directory. This will give you an idea as to what is occupying more space. This command will also check the subdirectories, so it is better to run the command in your root directory.
find -type f -exec du -Sh {} + | sort -rh | head -n 5
After checking the file size you can remove the biggest file that is occupying the space. Inside the steemsmartcontracts folder, there is this file called node_app.log. This is a log file created by the application. We can also remove that file safely.
In addition to this if you would like to get the drive space or an overall disk space inside the VM. The command df -h will be helpful.
Alternatively, you can also use this find command to check if there are files greater than 10MB inside a directory. sudo find . -xdev -type f -size +10M.
 |
 |
 |
 |
 |
 |
Posted with STEMGeeks
I use ncdu(
apt install ncdu) for finding out whats using my storage. Really nice and easy to use.Wow, this is great. Thanks. All I had to do was to navigate to
/and doncduto get full VM's disk usage.A cool tool. Thanks!
!BEER
Well written. Now I don't need to write it :)
Current disk usage at my witness node:
It's very similar to yours.
@tipu curate
Great. That's one of the reasons why I wanted to write this article. I don't have to write it somewhere safe in my notes. It is safe in the blockchain. 😉 !BEER. Thanks for the votes. !BRO
Logrotate usually deletes old log files? Or am I missing something?
Logrotate will keep rotating the logs and not let the size of the log file grow. Old log files might need manual deletion.
You might find this worth a read through.
https://docs.rackspace.com/support/how-to/understanding-logrotate-utility
Thanks, I will give it a read.
wow lots of code my head is spinning but good luck mate
lol, take it easy. Cheers!
You as well 😉
Congratulations @bala41288! You have completed the following achievement on the Hive blockchain and have been rewarded with new badge(s) :
Your next target is to reach 9500 replies.
You can view your badges on your board and compare yourself to others in the Ranking
If you no longer want to receive notifications, reply to this comment with the word
STOPCheck out the last post from @hivebuzz:
Support the HiveBuzz project. Vote for our proposal!