August 15, 2021
AnyNowhere — daily chat log — back (to logs index)
13:28, Alex> Gligar, I'll probably be able to provide some simple additional features if you need them, yep. There's time, but feel free to take your time to detail what you need.
13:49, Gligar whispers: Well for one, I'd need a way to archive things that doesn't involve OCR (:
13:50, Alex whispers: isn't that something the database dump would help, or is it an entirely different problem and I'm not getting it?
13:51, Gligar whispers: Oh, here, not there.
13:51, Gligar whispers: Honestly, all I'd need is the archive of the thread.
13:52, Alex whispers: ok, I'll grab one so you can see how it looks and what you can do with it...
14:02, Alex> uuuh well, it came out 14.5 megabyte 7-zipped, all posts and the list of message IDs in the relevant Paranoia threads. How's that sounding?
14:03, Gligar> That sounds okay.
14:04, Alex> the thing is... starting from the file holding the list of IDs, you'd have to seek the relevant file among the posts, and then seek the <message>...</message> part of each message
14:05, Alex> it's not straightforward, I must say, but I guess it can be done :/
14:06, Gligar> As long as everything's in order, I'll manage.
14:06, Alex> or I could try and make a small script to fish out the messages and save them into a single file, in case you found that unbearable.
14:07, Alex> for the moment, I'll upload the file to that generic "personal" folder I had here, so you can give it a look
14:09, Alex> about 5 minutes for my sluggish connection to upload...
14:14, Alex>
14:14, Alex> there, click and download :)
14:16, Alex> the threads' message ID lists are in the "forums_4500-4799.txt" file, the relevant thread IDs are 4528 and 4763: from there, locating those lists, one may search the posts_... files to find each ID
14:16, Alex> but boy it'd be tedious...
14:17, Alex> lemme see if I can do something better in the meantime, ok? :)
14:18, Gligar> At least 4528 is easy to find (:
14:18, Alex> yeah, it's supercraplong :D
14:24, Gligar> And I can open everything in VS Code and do a multi-file search. Yep, that's doable.
14:26, Alex> uh... ok, but lemme just try and sort everything out, it's not that much of a hassle to give it a try...
14:28, Gligar> Go ahead.
14:32, Alex> gimme some time eh? I'll let you know how it went...
15:00, Alex> b-boy! are there really... 25595 posts? WHOA!
15:01, Alex> ok I've isolated the array of IDs, now to search all of them in sequence and fish out the author and <message> part, and serialize into a single text
15:49, Alex> think I finally got it right, but lemme check...
15:56, Alex> it's... working on that. :D maaan it's slow...
16:01, Alex> ssssiiiiixty percent put together... *yawns*
16:03, Alex> crossing the 10-megabyte output mark at 20000+ posts
16:06, Kristos salutes the long-term paranoia players
16:07, Albeyamakiir> Gosh, I'm constantly amazed that's still going.
16:08, Alex> yeah, ok, I've taken the whole dump, right now... it's slightly less than 3 HUNDRED THOUSAND lines. :/
16:09, Albeyamakiir> Wild
16:10, Alex> and sigh... I replaced doublequotes with the wrong sign, the less than...
16:11, Alex> bah, let's redo the whole thing... it'll take about 12-13 minutes, I think
16:13, Alex> 'least this time it might be perfect, to the extent it's possible to dump those posts into text. pseudo-html markups like <e>, <i> etc could be easily replaced if needed.
16:14, Gligar> It's the <e> and the <i> (and the rest) that I'm after :)
16:15, Alex> You've got a MASSIVE kind of work to do, I'll tell you. Aside from making it more comfortable, it's just a BUNCHLOAD of text.
16:16, Alex> Kudos for all the effort, in advance.
16:20, Alex> Almost there with the final version of the dump...
16:23, Alex> done, looks fine... but that's the IC thread. You need the OOC one? it wouldn't take much.
16:23, Gligar> Sure
16:23, Alex> k!
16:25, Alex> er that took 20 seconds maybe
16:28, Alex> Here it is, 3 megabytes 7-zipped ->
16:29, Alex> the IC thread, uncompressed, is 13.1 MB
16:31, Alex> good luck, oh and... don't feel absolutely compelled to host it on the successor of anynowhere, consult with the others and feel free to decide, I'm ok with whatever decision.
16:32, Alex whispers: it's your creation, not mine :)
16:33, Gligar> Those dumps look fine, thank you :)
16:34, Alex> That's nothing, you deserved help for all the time you've been caring for this place by enticing people to stay and play. :)