Notices by practically without merit (pwm@gh0st.live), page 2
-
Embed this notice
practically without merit (pwm@gh0st.live)'s status on Monday, 22-Jul-2024 22:12:38 JST 
practically without merit
9:11 make a wish -
Embed this notice
practically without merit (pwm@gh0st.live)'s status on Sunday, 21-Jul-2024 08:16:03 JST
practically without merit
Restless. Feeling the need to do something retarded. -
Embed this notice
practically without merit (pwm@gh0st.live)'s status on Sunday, 21-Jul-2024 05:06:13 JST
practically without merit
@kirby stop pissing into your usb port when you sleep -
Embed this notice
practically without merit (pwm@gh0st.live)'s status on Saturday, 20-Jul-2024 23:16:05 JST
practically without merit
~ $ echo "wham bam shamalam" | ./freakify 𝔀𝓱𝓪𝓶 𝓫𝓪𝓶 𝓼𝓱𝓪𝓶𝓪𝓵𝓪𝓶 -
Embed this notice
practically without merit (pwm@gh0st.live)'s status on Saturday, 20-Jul-2024 23:16:04 JST
practically without merit
@prettygood neither. more to follow when I clean it up a bit -
Embed this notice
practically without merit (pwm@gh0st.live)'s status on Saturday, 20-Jul-2024 23:16:03 JST
practically without merit
@prettygood -
Embed this notice
practically without merit (pwm@gh0st.live)'s status on Saturday, 20-Jul-2024 12:40:30 JST
practically without merit
have I mentioned lately that I DESPISE unicode -
Embed this notice
practically without merit (pwm@gh0st.live)'s status on Saturday, 20-Jul-2024 06:47:51 JST
practically without merit
@0 @pernia @11112011 @DiamondMind @DigitalCheese @Doll @Forestofenchantment @Sui @Terry @Waerloga @dcc @dick @ins0mniak @j @p @syzygy @thatguyoverthere @thebitchisback @threat fuck now I'm hungry -
Embed this notice
practically without merit (pwm@gh0st.live)'s status on Friday, 19-Jul-2024 19:29:29 JST
practically without merit
@pernia
okay so scenario a:
You need json from a row in a database (one of your posts) because someone wanted you to serve it so that it federates or some shit. we also suppose the posts are indexed by like, id and that we have that from the request. The database has to check an index for the id of that post, which is pretty quick, BUT then it has to actually go get the json (it probably wouldn't be in the index -- this is called a covering index, if it has all the information you need --, because that would effectively double up the size of the object data) so it looks at the page number and row id pointed to by the index (this is the logical location of the data in the database).
Then the database asks the page manager for the page it needs. In this scenario that page is not in memory, and the page manager must read it from disk. with this page loaded into memory, we then grab the tuple we wnat BUT WAIT, the json data is bigger than the remaining size available in the page and spills over into another page so we have to ask the page manager for that page, and any subsequent spillover pages until we are done reading all the json we want (each page fetch necessitates a new disk read if that page is not already in memory, and, since these are pages full of nothing but json from one row, it's highly likely that they are not already in memory).
THEN after all that we can stream the json data out to whatever is handling the http response
OR
scenario b:
We receive a request for an object. We have cleverly named all our objects to that the file path maps to the url path, and have told nginx about this mapping and where to look.
nginx just serves the file (it is very fast at this), and the request never touches our backend. -
Embed this notice
practically without merit (pwm@gh0st.live)'s status on Friday, 19-Jul-2024 19:29:27 JST
practically without merit
@pernia
> i assume by page manager u mean the mmu?
the page manager is the component of the database (it's part of the software, not the OS) responsible for reading and writing pages. It usually has a LRU cache of pages which it has recently fetched from disk so it can sometimes return them quicker. Pages can come in several types that indicate what information is stored in them (data tuples, table definitions, indexes, mappings of tables to which pages contain data for that table) but the big one here is the data page. Pages are addressed by their page number, which is literally just the order they are in (usually). A data page holds data tuples. Data tuples are the rows in a table, and they can be logically addressed by (page_number, row_id).
> wouldn't moving json from disk to memory have to happen anyway? why would it be slower in a db than from disk?
It does have to happen anyway but when you get it from a database instead of ripping it straight from a file, first you have to go and find the data you want, and then call fopen. If you just know which file you want to rip json out of, then you can skip all the work of locating it, and just call fopen.
> and wouldn't reading the data from disk be faster since its a B tree, rather than reading the file sequentially?
indexes are b trees, data tuples are just sort of chucked in there in the order they are created usually unless you are doing something fancy like maintaining a physical sort order within the pages, which would be really expensive for CRUD operations as you would have to shuffle potentially your entire table around for every insert.
> then in scenario b, that would mean reading the file sequentially to load it from disk to memory,
nginx does this and it does it in fancy optimized ways that stream the file, rather than load the entire file into memory in one big buffer and then flush it out.
Scenario b is faster if you engineer the files to be laid out in such a way that you don't have to look for them. Placing them strategically means that you just know where they are based on filename. If you did have to search them with like grep and shit then yes that would be much slower.
You have some misconceptions about where exactly the b-tree comes into play. The b-tree powers indexes. To fetch indexed data you first consult the index by traversing its b-tree (fast), and then you still have to fetch the data from its data page if the index tuple wasn't indexing the field you wanted in the first place (which it wasn't in our scenario). The index IS way faster than doing a sequential scan of every datapage that has data for a given table, and checking each tuple in it for the one or however many your query wants. With an index you know the address of the data you want but you still have to fetch it off the disk (unless it's cached by the page manager but let's pretend it isn't).
The database CAN'T be faster than simply reading off a static file. It is simply more work to be done, work that is a superset of the work done by just ripping the file off the disk and out onto the network.
The limitation is that not every scenario allows you to engineer the database out of the picture. This is not a universally applicable strategy. The database offers flexibility and makes difficult things possible, but the realization here is that all you're really doing is serving a static file, and that this isn't necessarily a difficult thing (if you're clever about it). -
Embed this notice
practically without merit (pwm@gh0st.live)'s status on Friday, 19-Jul-2024 19:29:20 JST
practically without merit
@pernia @vic caddy is for zoomers who are scared of config files longer than 6 lines -
Embed this notice
practically without merit (pwm@gh0st.live)'s status on Friday, 19-Jul-2024 13:50:20 JST
practically without merit
@pernia
1: @MercurialBlack
2: @MercurialBlack -
Embed this notice
practically without merit (pwm@gh0st.live)'s status on Friday, 19-Jul-2024 13:50:19 JST
practically without merit
@MercurialBlack @pernia I think you two have not federated yet, he couldn't tag you in a post -
Embed this notice
practically without merit (pwm@gh0st.live)'s status on Friday, 19-Jul-2024 13:50:18 JST
practically without merit
@MercurialBlack @pernia it's him, not you I think -
Embed this notice
practically without merit (pwm@gh0st.live)'s status on Friday, 19-Jul-2024 13:50:17 JST
practically without merit
@MercurialBlack @pernia does snac2 have plugins? you should patch it in -
Embed this notice
practically without merit (pwm@gh0st.live)'s status on Friday, 19-Jul-2024 13:50:16 JST
practically without merit
@MercurialBlack @pernia to cc him nuntil federation is complete -
Embed this notice
practically without merit (pwm@gh0st.live)'s status on Friday, 19-Jul-2024 13:50:15 JST
practically without merit
@MercurialBlack @pernia it would be simple, take where the message in and output things are, and add a registry of plugins,
on startup, have the executable load up libraries with dlopen, which can register themselves into the message processing pipeline (it can just be a linked list)
each plugin need merely implement a single function that accepts a message, and farts it back out the other end, for the next thing in the chain.
message flow can resume to normal once you hit the end of the plugin linked list -
Embed this notice
practically without merit (pwm@gh0st.live)'s status on Friday, 19-Jul-2024 13:50:14 JST
practically without merit
@MercurialBlack @pernia it's just a couple function calls and some function pointers.
I have explained it clumsily. -
Embed this notice
practically without merit (pwm@gh0st.live)'s status on Friday, 19-Jul-2024 13:50:13 JST
practically without merit
@MercurialBlack @pernia dlopen man page would be about all you would need to read. -
Embed this notice
practically without merit (pwm@gh0st.live)'s status on Friday, 19-Jul-2024 13:50:12 JST
practically without merit
@pernia okay compiled plugins written in c are a valid use case imo
All GNU social JP content and data are available under the