You are viewing a single comment's thread from:
RE: Updates on steem-python, SteemData, and the node situation
So, we need a healthy and well organized community run network.
In my opinion, Steemit Inc. should have a top priority in establishing this network. With planning, support, providing resources and rewarding participants. Full package!
Ps: self-upvoted to push the comment up.
Which?
We already pay for and run a high-availability cluster of steemd nodes, at https://steemd.steemit.com.
What exactly are you proposing we do?
What I do miss are clear and precise instructions / documentation on how to set up and run a community node. Hardware resources needed, recommended, software needed, set up procedure, day to day operations needed. The n the why 's of running a node - benefits, not only financial.
Perhaps all this is already written and published somewhere?
All this to distribute the load from your cluster to the community.
Next, looking into the future - one word comes to mind - sharded.
If it really is a requirement to have 128 GB of RAM to run a node then that's way too much. Somehow the load should be lowered to come into a reasonable range. I am not telling you anything new, I guess.
OK, I realize that my suggestions are not very constructive. I can't propose an out of the box solution. I can only voice my concern. I believe that you are working on a long term solution to keep the platform functional and scalable.
Good luck!
I'm starting to understand the architecture more (I think), and am feeling more confident about the long-term future than I was, but here are some of my thoughts about what you could do:
a) Allow it to be run to provide only the last n days worth of vote, comment and transfer data. This is all I would need for running my services on it, and I expect many others could also manage with less historic data. A smaller index surely reduces the RAM requirements significantly.
b) Allow it to run in a 'ledger only' mode, where the index could be smaller (as it would exclude all other operations). This is almost all the exchanges would need I think.
I realise this will need some work on steemd, but over time, these increasingly demanding running requirements are centralising the network unnecessarily, making it more vulnerable, stifling community innovation and leading to doubts about scalability.
Thanks for your time.
They are working on several pieces of infrastructure to address these issues. I hope that the community can also help, and add some resilience to it.