HafSQL in beta, I noticed index building is becoming a serious drag. Thought I'd share my last conference video as possibly helpful: Speedrunning the Open Street Map osm2pgsql Loader
Greetings Hive minds! I'm a long time PostgreSQL contributor and author specializing in performance tuning. Now that @mahdiyari has the PostgreSQL based
That took the roughly terabyte sized Open Street Map Planet data set and re-tuned everything for NVMe to drop loading time, which on current hardware I now have down to just over 4 hours. All the config changes to PG and Linux are documented. HafSQL's starter postgresql.conf probably needs less shared_buffers and more maintenance_work_mem to speed all its index builds up; few disk GB of max_wal_size first would help too. Hoping to get the full Hive data set running here so I can test myself.
Day job at Crunchy Data is 100% open source work like multi-cloud PostgreSQL. I run a benchmark lab and the Hive blockchain makes a nice sized data set for my upcoming work.
Wow!
So technically your first post is supposed to be a introduction post. And that is so that we can reward your content enough so that you can be self-sufficient to operate on blockchain independently.
However this actually is an incredible first post. So thank you very much for your hard work and your knowledge!
Thank you very much for being here and this definitely looks like an amazing opportunity for the entire community.
@freecompliments
The content of this post is flying so far over my head, I think it collided with an aircraft!
Just goes to show how little I know in certain areas lol.
Basically it means that we have really amazing friends....
Just wait. A lot of people believe in you and a lot of people are doing some really good work.
Here pretty soon when you have an idea and you ask for something...
This person might be able to say that they can get it done.
Our world is opening up and it's absolutely amazing!
Love it! I already have general ideas gradually brewing for what can be added onto the blockchain, but then we have the actual knowledgeable developers with the know-how of execution and potential pitfalls.
Oh yeah I got a bunch of different friends that know how to do this however this guy right off the bat comes walking in swinging some really good work...
And right off he attracts some pretty interesting attention. I have a feeling he's going to fit in with the programmers and developers. If not become one of the leaders of that sector.
!LOLZ
lolztoken.com
I can do it with my eyes closed.
Credit: reddit
$LOLZ on behalf of spoonies
(1/10)
ENTER @WIN.HIVE'S DAILY DRAW AND WIN HIVE!@freecompliments, I sent you an
Thank you, and yeah, I know, but technically I'm not supposed to write things like this without going through a corporate process--one which might have failed had I only written a standard introduction post Some days there's no way to avoid breaking the rules, you just gotta decide which rules you're gonna break. Keep up the good work on the troll swatting BTW, once I'm better oriented here I'd enjoy building tools for that job; reminiscent of fighting the spammers when I used to run an ISP.
You absolutely should sit in on my secrets class...
So yeah just ask your wife and I'd be very happy to plug you both in.
It seems like you speak code. And you happen to be able to make computers be able to listen to you.... A modern day pied Piper!
Don't worry about anything and technically you can do what you want although I would advise a couple of things...
First off yeah writing the introduction post is one.
Setting up your account to be automated is another one.
Beginning trading and investments on our decentralized platform.
Those three techniques and when you implement them early allows compound interest to begin right away. That compound interest will absolutely grow out of control very soon.
I really like enabling people here on the blockchain to do all sorts of amazing stuff. It's really incredible that even a caveman like me understands how this entire blockchain works.
Besides that there is a huge amount of different communities as well as jobs here! The entire financial reward system is pretty cool. The decentralized investing? It is absolutely revolutionary.
Congratulations on being here and I can't wait to chat!
Thanks for the starter checklist. I started high speed database work with PostgreSQL for trading. Purchased second resolution history on how high alpha stocks move, wrote a Java Automated Trading System that looked for patterns in how the whale banks staggered their trades. Back tested it over and over on that data set, thus the need for a fast database. Executed day trades against the live market for just over a year. It made money. Only shut it down because my mom needed help after an injury and it took round the clock work to keep the system going. The R&D on that project had me read every major book on trading and speculation theory then on the market. Tossed about 100 obsolete trading books last week.
Part of why I'm here on Hive is to explore monetizing my writing while reviving my trading. Alternatives like Substack are interesting, I just try to avoid business with Venture Capital controlled software companies or ones with tightly controlled and/or closed source code. Hive checks most of my boxes.
Dude I really can't wait to have you as a friend!
I have my own style of trading it's really not rocket surgery and I'm very happy to share it with the entire world. Basically the entire premise is I just don't care. I just place the trades and let it all roll. 100 small positive trades beats a couple of small ones. Lots of small steady steps forwards really helps!
And we have automated trading bots here! So I'm really excited to check out what you are doing as well as to learn more about trading.
I sure do love the diesel pools because I can trade and make a pretty good chunk of funds and then put them into these liquidity pools and leverage that to make even more.
As for the rest of it my class definitely covers all this information. Monetizing and being a part of Hive is an amazing thing!
Glad you are here!
That dog is so adorable . I love this pic
Yes he is most days!
You expressed brilliance and knowledge right off the bat!! I'm sure that Hive will forgive such an insignificant social transgression. 😉
Thanks for joining, and I hope that you find as much value in Hive as you're certain to bring.
Welcome to Hive !
!DHEDGE
!LOLZ
lolztoken.com
Acorny one.
Credit: reddit
$LOLZ on behalf of cryptounicorn420
(1/10)
ENTER @WIN.HIVE'S DAILY DRAW AND WIN HIVE!@postgres, I sent you an
We generally don't need much shared_buffers to begin with because ZFS is already doing the caching. I probably will have to include my
postgresql.confin the repository.I remember experimenting
maintenance_work_memand I do already hit the max disk utilization pretty much on gen4 NVMe raid 0.I will certainly take a look at your video and try your suggestions. Thank you.
I saw how you're using ZFS for compression and therefore prioritizing ARC, that all seemed quite sensible. That Open Street Map loader application will gobble up to 90GB of RAM for its cache, and on 64GB servers I was surprised how well PostgreSQL continued to work even with
shared_buffersand the working memory settings all dropped to 1GB. I hope you're able to find a useful trick or too in the talk, things like turning WAL off can be helpful when you're cycling on local development and just want the load time down. I was doing NVMe RAID0 for a while, I have a 4TB Firecuda 530 I'm planning to use for testing your code out. Keep up the good work and we're really rooting for you here. Migrating apps from SQL Server to OSS has been a good sized chunk of my life.Congratulations @postgres! You have completed the following achievement on the Hive blockchain And have been rewarded with New badge(s)
Your next target is to reach 50 upvotes.
You can view your badges on your board and compare yourself to others in the Ranking
If you no longer want to receive notifications, reply to this comment with the word
STOPCheck out our last posts:
Post manually reviewed and approved for an Ecency boost. 😊
Yay! 🤗
Your content has been boosted with Ecency Points
Use Ecency daily to boost your growth on platform!
Support Ecency
Vote for new Proposal
Delegate HP and earn more, by @cryptounicorn420.
Welcome to Hive!
I'm HIVE friends with your wife and she told me you had joined Hive recently. Welcome!
When she told me what your username was I was like, hmmm.... isn't that the database? Seems my memory hasn't completely failed me yet! !lolz
Would love to hear more about your plans for a HIVE database. I'm a HIVE dev wanna-be (I'm the owner of The LOLZ project) and creator of @memebot !meme and would love to have ways to search the blockchain in more efficient ways.
lolztoken.com
I don't bereave it!
Credit: reddit
$LOLZ on behalf of captaincryptic
(10/10)
Delegate Hive Tokens to Farm $LOLZ and earn 110% Rewards. Learn more.@postgres, I sent you an
Credit: fonestreet
Earn Crypto for your Memes @ HiveMe.me!
Buy DHEDGE on Tribaldex or earn some daily by joining one of our many delegation pools at app.dhedge.cc.This post has been selected for upvote from our token accounts by @cryptounicorn420! Based on your tags you received upvotes from the following account(s):- @dhedge.bonus- @dhedge.pob- @dhedge.cent@cryptounicorn420 has 14 vote calls left today.Hold 10 or more DHEDGE to unlock daily dividends and gain access to upvote rounds on your posts from @dhedge. Hold 100 or more DHEDGE to unlock thread votes. Calling in our curation accounts currently has a minimum holding requirement of 100 DHEDGE. The more DHEDGE you hold, the higher upvote you can call in.