@p almost done my live log parser complete with status code filtering, requests over a buffer limit highlighted in a different color, ip anonymity toggle (in this case it's the nitter log), the ability to search the entire log, pausing live to investigate, themes, ua agent filtering/sorting, live (shitty hemispheric map made with :: (going to redo this). might be ready to send a copy if you want to try it out since I know you like your mon2.sh. sorry it's written in python
@tobias@p pete has a really cool shell script tool that will print out http errors and how frequently they occur each second. looks like attached image (for poast). I wanted something similar but that I can have immediate access to the info instead of having to switch tabs and check the logs (see second image for what I'm working on). ex in mine, red are failures (in this case mostly 429 or throttles), the pink line is somebody who is hitting the buffer and will likely get throttled/temp banned as he's got 4k requests and 20 over the span of about 20 seconds
@ailatan@p sorting in burst mode properly supports user agents now as well as IPs. just going to try to draw an ascii map or just strip it out altogether and ship it I think. I'm satisfied unless anybody else who has to view logs on the daily for long periods of time has anything else to chime in with
>inb4 Goyfeed
again this is the log for the nitter so the one for poast will be a lot more interesting (but its a lot larger and takes forever to buffer since it loads the backlog while tailing the current incoming)
@tobias@ailatan@p the backend software has the ability to generate thumbnails and provide the full scale image when it is opened but back in 2021 people complained and it caused a lot of constraints on the hardware we had at the time so we disabled it. its something we can look at again, but the amount of emojis that get loaded when you load poast and open the emoji picker are far larger in terms of overall size than what you'd get just casually browsing the timeline lol
at the UX end of things, i would be fine with limits like one post per minute and one reload per 15sec. storage-wise, i feel that images need enough detail to be understood. imho, high res is for autistes.
btw, are we seeing "web optimised" images different from the upload? because i would be fine with seeing thumbs in the thread and original quality via the dl button. (ie, images >120kb get an embedded thumb for the inline view).
i haven't done my homework so, if these are real rookie takes, just say so.
@graf@tobias@p Navigating the emoji picker is indeed quite painful, because the less frequently used pictures take forever to load. Also, some of the file sizes are quite absurd for such small images.
For example https://poa.st/emoji/stolen/AngryGlitch.png is 253 kilobytes. The attachment is an AVIF conversion at only 39 kilobytes: ffmpeg -i AngryGlitch.png -crf 50 AngryGlitch.avif
@eee@ailatan@p im done for now. it's 534 lines incl 22 lines of comment at the top about usage and why its python3 only. 31kb is as compact as I can get it with all the features currently present. I could do without themes but I know people like pete want to burn their retnas out with white backgrounds so
@eee@ailatan@p Looks like its 89-90% one guy with Inoreader paying for a subscription that updates /rss feeds every couple hours and it happened to be around midnight UTC
some have 16 subscribers, but its mostly that. wonder why they chose bulgaria for their hardware
@striderpod@ailatan@p I need to add a --since cli argument, parsing a month worth of poast logs every time I change the code is going to drive me insane. nearly 50 million lines and we ignore media and all requests from other instances in the logs so thats just users :/
@striderpod@ailatan@p look at the time of the day mate, thats in UTC. they are sorted by requests at that time. highest amount of requests at 00:00 NZ was #4 which checks out since thats noonish around where you are or 1pm somewhere around there
@p it's got a memory leak for large logs (48GB ram usage for poast's 16GB access log minus instances for example) so I'm fixing it but maybe I should take the opportunity to rewrite it in Go so I can learn
@graf That is one reason it's convenient to use awk: stream in, stream out. It is not quite as slick as a single-button scoreboard but it's like, whatever question I'm interested in, I can ask the question in the form of a one-liner and get a sensible answer out of awk.