Programming Diary #19: Follower network strength and preparing for Open Source publication

in Steem Dev8 months ago (edited)

Summary

Once again, I didn't have much progress to report at the two-week mark, so I skipped that update and it has been about a month since Programming Diary #18: WhaleAlert inspired activity notices. For this interval, my activities mostly shifted from the Steemometer programming to alignment of voting weights with the strength of an author's follower network. I also made some progress towards getting the Steemometer ready to publish as open source.

Background

Here are the two goals that I set for myself in Programming Diary #18: WhaleAlert inspired activity notices.

  1. Implement algorithm changes in automated voting so that voting percentages start incorporating information about an author's follower network;
  2. Start preparing the existing Steemometer code for Open Source release by adding comments, removing clutter, and trying to make sure that it doesn't leak details about my development environment.

I expect to revisit the first goal many times in order to continually improve the voting alignment between blockchain content and the human attention that the content receives. I also expect the second goal to take more than 2 weeks of effort.

I made progress on both goals during the last couple of weeks, but as expected, I have work remaining before I can call either one complete.

Progress

Aligning voting weight with follower network strength

For the first goal, aligning voting weight in my autovoter with the strength of an author's follower network, here's a visualization of the rules that I implemented. (each line is based on the number of followers that an author has)

Here are some characteristics of the "Follower Score" as implemented:

  1. If an author's follower network has a median follower reputation of 25 or less, it doesn't matter how many followers they have. The follower score is 0.01 (to avoid multiplying by 0).
  2. If an author has more than about 375 followers, and the median reputation is above 25, then the median reputation doesn't matter. They get the maximum follower score, which is 1.
  3. In between 1 follower and 375 followers, the follower strength increases from 0.01 to 1 with the number of followers and with the median follower reputation.
  4. To get the follower score above 0.5, the author must have somewhere between 64 and 128 followers.

I don't really like the shape of these curves, but it's a first pass. I ran into trouble with API responsiveness for a few days, so I'm still observing the impact of the changes.

As previously mentioned, once I find a curve that I'm happy with, I'll use that to inform the vote suggestions in the Steemometer tool.

Preparing Steemometer for Open Source publication

In order to do this, I am leaning about javadoc, and how it is implemented in Netbeans. I also learned about the SLOC maven plugin, so that I can contextualize my progress through the various Java class files. At present, it seems that my project currently contains about 2,790 lines of code.

Package NameFile NameTypeBlankJavaDocCommentCodeTotal
module-info.javasrc002911
steemometerApiServer.javasrc72915289
steemometerAuthorInfo.javasrc446122133260
steemometerCountOps.javasrc344134227336
steemometerMedianCalculator.javasrc20391108168
steemometerPostDisplayManager.javasrc1543391152
steemometerPostInfo.javasrc421228227399
steemometerStartPoll.javasrc61002137
steemometerSteemData.javasrc4511211192360
steemometerSteemPriceFetcher.javasrc2191265107
steemometerSteemometer.javasrc1901212110781401
steemometerSteemometerConfig.javasrc1441050105
steemometerSteemometerController.javasrc51012339
steemometerSystemInfo.javasrc4120925
steemometerTransactionNotifier.javasrc306014217321
steemometerUrlValidator.javasrc1644168129
steemometerVoteSuggestor.javasrc51701537
steemometerXferMemoInfo.javasrc23931126243
steemometerXferMemoManager.javasrc1542379139
2 package(s)19 file(s)java53679723527904358

I learned how to use Netbeans to insert javadoc comments in all files, and I have reviewed the auto-inserted comments in the ApiServer.java file and the AuthorInfo.java file. I also went through all files and removed auto-generated comments that I thought were unnecessary.

Next up

Since my goals from the last cycle are still in progress, I plan to continue them for the next two weeks.

  1. Experimentation and improvement towards aligning voting weights with follower network strength.
  2. Getting Steemometer ready for Open Source publication.

As with last time, I don't really expect to finish either goal in the next two weeks.

Reflections

I want to follow-up on three topics from the Reflections section in Programming Diary #18: WhaleAlert inspired activity notices, and also add a new topic:

SBD print threshold / Steem inflation

The Steem price has now remained above the SBD print threshold for a little more than a month, so I have continued to keep an eye on Steem inflation. The first quarter was deflationary, but the first week of the second quarter reversed that trend. It will be interesting to see what happens in coming weeks since the BTC halving is expected on April 20.

Blockchain activity

In the last post, I wondered if SBD printing would lead to an increase in blockchain activity. It appears that the answer is "yes". In that post, the blockchain activity was reported at about 185 operations per minute. Now it's up to about 207.

Steemit.com stability/API performance

The overall stability problems that were mentioned in Programming Diary #18, have been solved. API reliability still has occasional problems, however. As noted above, my autovoter fell behind by about two days for some reason. I didn't notice it for a while, so I have no idea what the cause might have been. Also, simple python activities are often still blocked by the rate limits that were implemented last fall.

Witnesses

Here's the new topic for today's post. I have some thoughts about witnesses in the Steem environment:

  1. The top-20 witnesses have more to do than just validate blocks. The task of setting the price feed seems to be in good order, but they also need to consider the SBD interest parameter. I've discussed this topic here, here, and here. It would be nice if the witnesses would weigh in with their perspectives. Is there some reason why 0 is the optimal percentage?
  2. I could be wrong, but I think there's at least one top-20 witness who has disengaged considerably from the ecosystem. It might be good for people to reevaluate their votes.
  3. Since I don't control enough stake to influence the ranks of the top-20, I often focus my witness voting further down in the ranks. There are a number of disabled witnesses who are still in front of real witnesses, so - again - it might be good for people to reevaluate their votes. For those of us who have smaller stakes, moving votes that aren't needed from the top-20 section to higher ranked witnesses might have a substantial impact towards downranking the disabled witnesses.
  4. I've mentioned it before, but IMO, @moecki should be ranked much higher than #58. It seems to me that he provides more to the ecosystem than several of the witnesses in top-20 positions, and many of the others, too. I hope readers will consider allocating a vote towards his witness.

Conclusion

If slow and steady wins the race, then I'm on track for a big programming win. 😉 It seems like publishing the toy Steemometer program as Open Source is in sight now, even if it's still not imminent.

My purpose for breaking out of iterating new features for it was to force myself to develop some other skills that will be useful for eventually implementing a peer to peer social media client. Based on the experiences learning about javadoc and SLOC, that seems to be proceeding as hoped.

I hope to see you in two weeks with Programming Diary #20!


Thank you for your time and attention.

As a general rule, I up-vote comments that demonstrate "proof of reading".




Steve Palmer is an IT professional with three decades of professional experience in data communications and information systems. He holds a bachelor's degree in mathematics, a master's degree in computer science, and a master's degree in information systems and technology management. He has been awarded 3 US patents.


image.png

Pixabay license, source

Reminder


Visit the /promoted page and #burnsteem25 to support the inflation-fighters who are helping to enable decentralized regulation of Steem token supply growth.

Sort:  
 8 months ago 

I wonder if I have any painful suggestions to make?

Thinking about the Follower count to assist voting percentages, I wonder how hard (or necessary) it would be to check "Follower Activeness" - not necessarily their "Last Update" but perhaps checking that their STEEM Power is above a minimum threshold for acceptance. For example, older users might have a "higher reputation" bias from inactive users. As I type this, I'm asking myself "so what?" and perhaps the answer is that you're checking if the user's attracting new followers and therefore still posting good quality content. Although high power users naturally attract new followers irrespective of how good their content is.

Can you tell I'm thinking out loud.

I've mentioned it before, but IMO, @moecki should be ranked much higher than #58. It seems to me that he provides more to the ecosystem than several of the witnesses in top-20 positions, and many of the others, too. I hope readers will consider allocating a vote towards his witness.

!00% agree. Not only his activity, but his willingness to put himself in the community for all to see. I know some witnesses working behind the scenes that haven't talked about what they're doing (not that I've seen anyway).

 8 months ago 

I wonder how hard (or necessary) it would be to check "Follower Activeness"

I've thought about that, too (great minds think alike? 😉). Also median SP of followers. I think both values might be useful, but the only ways I know to implement them would be to query each follower account separately at the time of voting, or to store it in my own database and update it regularly. Both implementations require more compute than I want to put into it right now.

TEAM 1

Congratulations! This comment has been upvoted through steemcurator04. We support quality posts , good comments anywhere and any tags.
Curated by : @o1eh



 8 months ago 

Thanks @o1eh 🙂

 7 months ago 

I just stumbled across this and thought of you and @moecki - https://peakd.com/steem/@steemitblog/introducing-the-steem-recommendation-page

I wonder why it was removed - presumably it required too much processing power.

Have you ever scrolled to the bottom of steemitblog on steemit.com? It ends prematurely which I found interesting 🫢

 7 months ago 

Nice :-)

I recently saw something in the Hivemind code that amazed me. I can't remember what it was right away, but it must have once been available or intended as a function.
Perhaps you can actually still find remnants of the implementation of the recommendations at that time. In any case, I think that this was implemented at Hivemind level.

It ends prematurely which I found interesting 🫢

Strange. On steemitdev.com it doesn't end prematurely...

 7 months ago 

I think hivemind came along later. I ran a couple quick searches of steemit's github space, but didn't find anything. I didn't spend a lot of time at it, though.

 7 months ago 

You are right. Thanks for the link. I really need to have a look at all the old posts from steemitblog. Quite a lot was explained there.

I found two commits:
https://github.com/steemit/steem/commit/a01b04ed439681528c61b3b7e318092c8b98067f
https://github.com/steemit/steem/commit/f3877959f55d30e1ba2815eeb437dabc0e3d70fd

These were probably the first implementations. I don't yet know whether it was changed before it was removed again.

 6 months ago 

I recently saw something in the Hivemind code that amazed me. I can't remember what it was right away, but it must have once been available or intended as a function.

Whilst looking for some documentation for steemit.com, I stumbled across a few interesting discussions that were prompted by the dev team. I can't remember what any of them were either 😆 but I did think about digging some of them out. One of them was a page view counter and there's a lot of condenser code scattered about for that.

 6 months ago 

page view counter

This would be a great favour to some (well known) users :-)
However, there are a few obstacles here:

  • The way it was implemented at the time, only the calls via the Steemit condenser could be counted.
  • The data are obviously still sent to the overseen server, but requests to it do not work (I tried this once with the old code).
  • Even if the server were to answer requests again, this would only work with the api.steemit.com node. Meaningful results are obtained with other nodes (if the changeover works) only if they redirect the requests to api.steemit.com.

Will you accept the challenge? ;-D

 6 months ago 

Ha ha - no chance! I'm grateful that you've looked into this already. I'm not a fan of page view counters and if you managed to convince me otherwise (😉), I'd investigate a Google Analytics API that could pull the data in and produce some more meaningful data. One day, I'll get access to Steemit's GA account!

 6 months ago 

Hopefully it won't take so long ;-D

 7 months ago 

Interesting. That even predates me by a a couple of months. I have no recollection of ever seeing that feature in operation, so maybe they removed it quickly. If I remember, I'll ask @cmp2020 if he remembers it when he gets home from college. I didn't find any other references to it in github or google after a couple quick searches.

I agree that it was likely removed due to performance considerations. That definitely strikes me like it would have trouble scaling. Also, when the automated voting came along, I'm not sure how accurate it would be. I think useful recommendations would probably require something more sophisticated (i.e. language processing & machine learning). There's a whole field of AI research that's dedicated to this purpose.

If I scroll to the bottom of steemitblog, I get there. If I scroll to the bottom of steemitblog/posts, it ends prematurely. Both go all the way to the bottom from steemitdev. Maybe another bug that you fixed with your work on condenser. 😉

 7 months ago 

I think it was the first post that came from the steemitblog account. There was a latter post about Steemit going open source so my suspicion is that the code didn't even hit GitHub. I haven't really looked though.

I think useful recommendations would probably require something more sophisticated (i.e. language processing & machine learning). There's a whole field of AI research that's dedicated to this purpose.

Good point. I suspect that it won't be long before an AI model can analyze a user's posts, review who they vote and comment on and subsequently find authors with a similar profile. I would say that using tags (for example) but few people use tags correctly now. Ignoring that downside, it actually feels like a fairly straightforward algorithm to do.

  1. Review my comments and posts.
  2. Review the people I've commented on (tags used, reputation, date joined, etc.) with pre-determined weightings.
  3. Find people who use those tags, have that reputation, etc. with a narrow to broadening search. It could target 3 months data so as not to be overwhelming.

Maybe another bug that you fixed with your work on condenser. 😉

My guess - it's hard coded on the steemit.com domain 😉

Upvoted. Thank You for sending some of your rewards to @null. It will make Steem stronger.

TEAM BURN

Your post has been successfully curated by @kouba01 at 40%.

Thanks for setting your post to 25% for @null.
We invite you to continue publishing quality content. In this way you could have the option of being selected in the weekly Top of our curation team.

Team Burn (1).png

Burning STEEM by sending it to the @null account helps reduce the supply of STEEM and so increase its price.

 8 months ago 

Thank you @kouba01!

This post has been featured in the latest edition of Steem News...