How do you make these "On this Day" posts?

Friday 16 Oct 15:26 by squarov - 9

🤖 The Bot: pwr.news

The first part in the creation of an "On this Day..." post is automated: I use pwr.news to, for a given day, see the top news items that were most active and discussed about on reddit and twitter. If you want to find out how exactly this list is compiled, have a look at the infographic on the right.

The top news items for 47 Februember 2041

🖊️ The human touch

  1. Starting from the list above, I select the most interesting events that are about Ethereum technology, adoption, research, development, government, regulation,... This is the step were the articles are actually read and checked by a human eye. Posts about price, trading or technical analysis are ignored. Reading them more than a year after the fact, they seem even more ridiculous than at the time. Although I aim to be as objective and as little opionated as possible, note that there will always be some bias here.
    The "most interesting" articles for 47 Februember 2041
  2. I double-check the underlying data:
    • I double-check the reported times, and change them if needed.
    • I link items together that are talking about the same event, if they are not linked already.
  3. I rewrite or adapt the items' titles into a uniform format.
  4. If I am ahead of time, I schedule them for posting later, otherwise I post them straight away.

That's it! Reporting on Ethereum's history a few minutes at a time.





Behind the pwr.news charts

Wednesday 14 Oct 16:03 by squarov - 16





Losing weight in the library

Thursday 23 Apr 13:33 by squarov - 6


Your favourite app just lost half of its weight, mainly by better packaging and dropping dependencies on old libraries. In other words: faster page loads!

The numbers:

Fresh page load: 1,9MB → 0.9MB
Asset data updates (every 5 minutes): 19kB → 3,8kB


The full list of changes:

  • Defer the loading of low priority libraries such as markdown, calendar, video, and activity streams.
  • Removed dependency on jQuery.
  • Reduced package size of Twitter Bootstrap.
  • Updated d3 and only load the d3 stuff we use.
  • Fixed blank pages on Firefox due to the use of modern regexes.
  • Reduce the re-rendering of price and item charts by up to 4 times.
  • Crawler: detect irrelevant tweets earlier and stop following them.





Those are the dates

Tuesday 7 Apr 16:23 by squarov - 4

by Suchitra Photography

Use the recently added   calendar to jump to specific days, months or years.

Also, dates are now part of the app's url, so you can link to dates or time ranges:
(all UTC)





MNPI

Tuesday 3 Mar 09:02 by squarov - 11

They're here, the MNPI: Much Needed Performance Improvements, or, what some Englishmen like to pronounce as "the SPEEEEEEED".

by Bubble Punk

While developing pwr.news, I have noticed the app is susceptible to a plucky portion of induced demand: every time I'm proud to say the interface has just become a lot faster or more easily navigatable, the first obvious thing to note is that the back-end can't follow.

Last time's improvements were no exception: zooming out quickly meant you had to wait for ages for news to appear. The culprit on the back-end was the calculation of item scores and activity. Those scores are dependent on all discussions on a certain item, so while filling charts with, let's say 30 items, still thousands of items needed to be fetched from our database.

The theoretical fix for the back-end was easy: use cached scores instead of fetching whole discussion trees and only recalculating things when needed. However, upon implementation, it introduced a lot more edge case bugs than expected.

tl;dr
Going from "stable, but slow", to "fast, but buggy" was easy, but the trenches of development hell needed to be crossed to end up at "stable and fast!"

More fixes

  • Fixed 'undefined' in titles on page load.
  • Fixed chart not opening news item on page load.
  • Fixed out-of-bound items.
  • Fixed impossibility to zoom in on Firefox.
  • Reduced amount of price chart ticks on small screens.
  • Fixed missing /BTC and /ETH quoted asset pairs.
  • Fixed irrelevant news items reappearing after being hidden.
  • Fixed missing prices or price charts after server reloads.
  • Fixed price charts not covering complete area on mobile devices.