The End of Shrinemaiden As We Know It

[Lebon14]

Can an admin make the login only forums and subforums visible for everybody so we can preserve them?
Thanks.

EDIT:
To be more precise, I'd love to archive, with Wget, the Rules section, Nuclear Furnace, the Caf? and Daiyousei's Cold Storage and all their subforums.
Additionally, I'd love to be able to get the profiles of posters too.

I can do this by injecting my cookies into Wget and get a "logged in" page but that would be I would have to archive all reply/Edit/preview/etc pages for every forums/subforums, threads and posts which slows down the process a whole lot.

[DX7.EP]

I'm going to hazard that we've until 22 April at latest to do any backups, going by the mention of the domain registration.

If so there will be quite a bit to archive here, mostly old stuff from Genji's Battle Arena where TF2 and Minecraft servers were run and active once upon a time. Some of their contents continue to remain relevant in one spinoff clique.

Aside from that and the 2013 Guilty Alice stuff, I don't know if I'd back up any of my other things.... :V

[Agent of the BSoD]

This is pretty unfortunate, but given mentions from the staff here that they could only work on so much stuff on the forum, it's not a big surprise. I'll get to work on backing up what is important to me. Also I guess MotK v3 will be a good time to make a fresh guide on the PC-98 emulators.

[CapTengu]

Godspeed, everyone. I'll miss all of the fanfictions and guides posted here.

[HTFCirno2000]

Ok so I did some research, did some digging and actually have created a local instance of Simple Machine Forums 2.0.15. I have found out that, by default the fresh out of the box SMF has database backup enabled. I am unsure if it was disabled at one point but only people who have Forum Administrator rank can do this. It actually allows an administrator to Download the database as an SQL file with a sequence of commands to restore the data on another site. (Also full disclosure, I tracked down the old ModernDark64 theme on the Wayback Machine and used it for the local instance for nostalgia reasons. This is entirely only accessible within my local area network.)

MoTK > Admin Center > Forum Maintenance > Database

The steps are screenshots below:





Like i said, i have no idea if it was actually enabled for those who have Administrator role, but if it is there no root access to the server platform is actually required to get the database. However this does not account for the other wikis that exist on this server.

[Seventh Holy Scripture]

FYI, I have full control over the shrinemaiden.org domain name, and the domain renewal date is irrelevant for the purposes of guessing how long the current forum will remain functional.

[CapTengu]

Thank goodness. I guess we have until whenever the database decides to call it quits to back everything up.

[WindyKitsune]

Ok so I did some research, did some digging and actually have created a local instance of Simple Machine Forums 2.0.15. I have found out that, by default the fresh out of the box SMF has database backup enabled. I am unsure if it was disabled at one point but only people who have Forum Administrator rank can do this. It actually allows an administrator to Download the database as an SQL file with a sequence of commands to restore the data on another site. (Also full disclosure, I tracked down the old ModernDark64 theme on the Wayback Machine and used it for the local instance for nostalgia reasons. This is entirely only accessible within my local area network.)

MoTK > Admin Center > Forum Maintenance > Database

The steps are screenshots below:





Like i said, i have no idea if it was actually enabled for those who have Administrator role, but if it is there no root access to the server platform is actually required to get the database. However this does not account for the other wikis that exist on this server.

That's very good news, at the end there is a simple way to make a full content backup of the forum.

[HakureiSM]

Welp, looks like it's time to make use of that "download PMs" feature at last.

I don't see such a feature.

It seems there's a handful of people working on scraper scripts, thank you all for the work.

[Kurzov]

What's the status of the High Score threads/attached replays, will they be manually moved over or ...?

[Kilgamayan]

Since forum archive crawlers are being put forth, while I will make no administrative effort to block it, I would personally request any and all crawlers avoid recording LettyJournal, due to the personal nature of many of the threads there.

[aListers]

Well, I guess it's been a good run. As soon as I heard news of Helvetica passing I backed up my posts so I'm at no loss of history here. My dream diary is safe. That being said, I will be personally backing up my own topics on lettyjournal anyway - they're a good indication of what I thought about my life at the time. I don't want to lose any personal history of mine.

This website has had a huge impact on me as a person. From my dreams of Alice and Gensokyo to the somewhat biased political discussions we had, I've loved my time here. I really needed this place to be able to keep track of the brightest light of my life at a dark time - and while that light may now be put out, I will never forget it and the time I spent chasing it. Thank you to everybody who put up with my creepy and wierd postings on this fourm in my time here. I've cherished every positive message I've recieved on here and wish all the people here the best! Cheers to MoTK! May Touhou and Gensokyo live on forever!

[N-Forza]

Oh, for some reason I assumed PM archiving was a feature but maybe I was thinking of another forum.

[Barrakketh]

Oh, for some reason I assumed PM archiving was a feature but maybe I was thinking of another forum.

How important is the stuff in your sent/outbox? I have a "works for me" scraper to backup PMs, but it does require some setup on the users behalf. And it doesn't work on sent messages because I have none to test with :V

[Sparen]

Where is that feature? I can't seem to find it. If there's a built-in method of downloading your PMs, I think it belongs in the first post.

Edit:
It was recently been brought to my attention that there's a Danmakufu Wiki hosted on this site. Has it been archived?

The Danmakufu Wiki doesn't change much so the entirety of the wiki is available on the Wayback Machine. In addition, since it has already experienced multi-month periods of downtime, I did contingency planning starting a few years back so a good portion of the important parts are already on my website.

Ref: https://sparen.github.io/ph3tutorials/docs.html

[Karisa]

What's the status of the High Score threads/attached replays, will they be manually moved over or ...?

The attached replays have already been archived manually, and can be found in my scoreboard archive. The scoreboards will be recreated on v3, with gensokyo links and MotK v2 attachment links replaced by their archived versions, though it might take a while (I plan to make all the first posts consistent, this time around).

[Arcorann]

Looks like someone added MotK to ArchiveBot, so at least all the public material will be saved.

[Evil_Nazgul0616]

I have a question about the status of Rumia's Party Games, particularly the Mystia's Stored Games subforum: The quests archived there tend to span multiple threads of 1000 posts per thread, which makes them a bit difficult and tedious to back up and preserve. I also heard that there were archive crawlers going around through the site, so what I would like to know is if the archive crawlers intend to back up MSG as well or if that would solely be our responsibility?

[HTFCirno2000]

I have a question about the status of Rumia's Party Games, particularly the Mystia's Stored Games subforum: The quests archived there tend to span multiple threads of 1000 posts per thread, which makes them a bit difficult and tedious to back up and preserve. I also heard that there were archive crawlers going around through the site, so what I would like to know is if the archive crawlers intend to back up MSG as well or if that would solely be our responsibility?

I'm not sure the specifics of each bot but I think the crawlers are indiscriminate in that they are essentially just archiving anything within the forum structure. So if they are crawling the whole site, they'll get to mystia's stored games too.

[N-Forza]

How important is the stuff in your sent/outbox? I have a "works for me" scraper to backup PMs, but it does require some setup on the users behalf. And it doesn't work on sent messages because I have none to test with :V

Kind of important because it's how I've been keeping track of orders/requests (although I shouldn't be, admittedly) but most of it's already settled so it's probably the more recent stuff that's actually necessary to have on hand. I'll let you know if I need to make use of it, thanks.

[niektory]

How important is the stuff in your sent/outbox? I have a "works for me" scraper to backup PMs, but it does require some setup on the users behalf. And it doesn't work on sent messages because I have none to test with :V

I do have some so perhaps I could help with that, just send me the code. But wait... then you'll have a sent message so you won't need my help anymore.

I have a question about the status of Rumia's Party Games, particularly the Mystia's Stored Games subforum: The quests archived there tend to span multiple threads of 1000 posts per thread, which makes them a bit difficult and tedious to back up and preserve. I also heard that there were archive crawlers going around through the site, so what I would like to know is if the archive crawlers intend to back up MSG as well or if that would solely be our responsibility?

I'm backing up all threads I can access with my account, so yes, that includes MSG.

[Helepolis]

Late to the thread because of time zones and IRL.

RikaNitori contains quite some threads which need to be archived or moved. The wiki is definitely something that needs to be mirrored, but Sparen seems to already have taken care of that.

I will see if I can backup stuff from the sticky threads. The number of danmakufu activity did decline a lot over the past years. I can somewhat see the same happening over at the Eastern Communities so it is not just us.


@ Forza,  right I somewhat promised you to build that app for easier order tracking and such in the past. . . orz

[Nemo★Ma]

... ...

Just give me a call (via the forum PM, or discord (nemoma#0001) )if you guys need any help on moving the website (server choices, domain name related stuff -but that seems to be taken care of, backup/restore the forum, etc.).

From my experiences, while moving a site like MotK is a tedious process due to its size, it's certainly doable depending on how much one can backup.

[Lebon14]

@admins
https://old.reddit.com/r/Archiveteam/comments/f44mxj/touhou_forum_maidens_of_the_kaleidoscope_is_going/fhqfkia/

Hey u/ajshell1,

I work at Linode, and I'm hoping you can deliver a message for me to the operators of shrinemaiden.org's forum so that we can prevent the site from going down at all. I tried to register on the forum, but it appears registration is currently disabled.

First and foremost, I'm sorry for your loss. From the bit of reading I did, it sounds like TSO / Helvetica was a great friend who was a tremendous benefit to the Touhou community.

Through context from your forum I'm lead to believe that one of the administrators has access to a Linode user account, but they are having difficulty accessing the server via SSH because they do not have the root password. If they have permissions, they should be able to reset the root password of the server, and from there they can do anything they need in order to keep you up and running.

Thanks for forwarding this along if you can.

-Brian

[Alfred F. Jones]

@admins
https://old.reddit.com/r/Archiveteam/comments/f44mxj/touhou_forum_maidens_of_the_kaleidoscope_is_going/fhqfkia/

First off, happy birthday (to me). (This might be my last birthday on this site and I'm not sure we'll even get to the site's 11th birthday at this rate.)

Second, thank you for passing that on. Unfortunately, it doesn't work. The linode login that I used can only be used to pay the bill and update the payment information used for the site and that is it. It can't even be used to open a ticket with linode, much less reset the root password.

I'm sorry. I wish I had good news with which to reply. Please thank that person for taking the time to try and help, at least.

[Edible]

Yeah, if we had full access to the linode account it wouldn't be an issue.  Unfortunately that's not the case.

[Infy♫]

I spent the past few days scraping the site.
Here is every post on shrinemaiden.org that's accessible through an account.

it's all in a .csv file of about 800mb. I hope someone else can figure out a way to make it all easy to access.

[HakureiSM]

127mb compressed

Thanks my man, I got a copy

[nav']

I spent the past few days scraping the site.
Here is every post on shrinemaiden.org that's accessible through an account.

it's all in a .csv file of about 800mb. I hope someone else can figure out a way to make it all easy to access.

Grabbing a copy too. Myself I have downloaded every thread I could access in print mode, which in total is of similar size when compressed with 7zip Ultra setting. Lightweight, but not so great because links to other posts and attachments don't work... Gotta put together something better.

Edit: I sort of wish your csv file also preserved the id number of every post.

[Barrakketh]

Grabbing a copy too. Myself I have downloaded every thread I could access in print mode, which in total is of similar size when compressed with 7zip Ultra setting. Lightweight, but not so great because links to other posts and attachments don't work... Gotta put together something better.

Edit: I sort of wish your csv file also preserved the id number of every post.

I'll probably have finished writing my scraper by tomorrow, the next function is just grabbing the main page to get all the boards to crawl. Each thread is combined into one page, quote anchors are corrected to link to the right post, and attachments are saved (the high score entry board was my first test). Images hosted on shrinemaiden.org and in a post are downloaded (covers the staff badges and such, avatars, emoticons, etc.), and replay links pointing to gensokyo are rewritten to so they go to the lunarcast archive.

It's not a mirror of the board. More of a copy of the important content.