Author Topic: The End of Shrinemaiden As We Know It  (Read 5902 times)

Lebon14

  • 椛ちゃん、助けてぇぇぇぇぇ!
Re: The End of Shrinemaiden As We Know It
« Reply #30 on: February 14, 2020, 06:57:01 PM »
Can an admin make the login only forums and subforums visible for everybody so we can preserve them?
Thanks.

EDIT:
To be more precise, I'd love to archive, with Wget, the Rules section, Nuclear Furnace, the Caf? and Daiyousei's Cold Storage and all their subforums.
Additionally, I'd love to be able to get the profiles of posters too.

I can do this by injecting my cookies into Wget and get a "logged in" page but that would be I would have to archive all reply/Edit/preview/etc pages for every forums/subforums, threads and posts which slows down the process a whole lot.
« Last Edit: February 14, 2020, 07:29:15 PM by Lebon14 »

DX7.EP

  • Red shroom
  • *
  • Smooth Charisma
    • Sanmaat (site of personal tat)
Re: The End of Shrinemaiden As We Know It
« Reply #31 on: February 14, 2020, 07:55:59 PM »
I'm going to hazard that we've until 22 April at latest to do any backups, going by the mention of the domain registration.

If so there will be quite a bit to archive here, mostly old stuff from Genji's Battle Arena where TF2 and Minecraft servers were run and active once upon a time. Some of their contents continue to remain relevant in one spinoff clique.

Aside from that and the 2013 Guilty Alice stuff, I don't know if I'd back up any of my other things.... :V
C:DOS> ayayaya.mid
Bad command or file name

C:>_

Agent of the BSoD

  • Takeminakata Invocation
  • *
  • Never forget
Re: The End of Shrinemaiden As We Know It
« Reply #32 on: February 14, 2020, 08:32:31 PM »
This is pretty unfortunate, but given mentions from the staff here that they could only work on so much stuff on the forum, it's not a big surprise. I'll get to work on backing up what is important to me. Also I guess MotK v3 will be a good time to make a fresh guide on the PC-98 emulators.
I figured out how to play midi in games with a different device on Win7 ^^
TF2 Backpack
Embodiment of Scarlet Hair English Patch is almost completed.
^ I didn't forget about this. I don't know what you're talking about. >_>

Re: The End of Shrinemaiden As We Know It
« Reply #33 on: February 14, 2020, 09:08:01 PM »
Godspeed, everyone. I'll miss all of the fanfictions and guides posted here.
Je pense, donc je suis.

Re: The End of Shrinemaiden As We Know It
« Reply #34 on: February 14, 2020, 09:34:45 PM »
Ok so I did some research, did some digging and actually have created a local instance of Simple Machine Forums 2.0.15. I have found out that, by default the fresh out of the box SMF has database backup enabled. I am unsure if it was disabled at one point but only people who have Forum Administrator rank can do this. It actually allows an administrator to Download the database as an SQL file with a sequence of commands to restore the data on another site. (Also full disclosure, I tracked down the old ModernDark64 theme on the Wayback Machine and used it for the local instance for nostalgia reasons. This is entirely only accessible within my local area network.)

MoTK > Admin Center > Forum Maintenance > Database

The steps are screenshots below:





Like i said, i have no idea if it was actually enabled for those who have Administrator role, but if it is there no root access to the server platform is actually required to get the database. However this does not account for the other wikis that exist on this server.
Cirno the Ice Fairy~

Seventh Holy Scripture

  • Err... d'ya need somethin' from me?
  • *
Re: The End of Shrinemaiden As We Know It
« Reply #35 on: February 14, 2020, 09:36:50 PM »
FYI, I have full control over the shrinemaiden.org domain name, and the domain renewal date is irrelevant for the purposes of guessing how long the current forum will remain functional.

Re: The End of Shrinemaiden As We Know It
« Reply #36 on: February 14, 2020, 09:39:36 PM »
Thank goodness. I guess we have until whenever the database decides to call it quits to back everything up.
Je pense, donc je suis.

WindyKitsune

  • Lumin(ifer)ous Touhouism
Re: The End of Shrinemaiden As We Know It
« Reply #37 on: February 14, 2020, 10:05:13 PM »
Ok so I did some research, did some digging and actually have created a local instance of Simple Machine Forums 2.0.15. I have found out that, by default the fresh out of the box SMF has database backup enabled. I am unsure if it was disabled at one point but only people who have Forum Administrator rank can do this. It actually allows an administrator to Download the database as an SQL file with a sequence of commands to restore the data on another site. (Also full disclosure, I tracked down the old ModernDark64 theme on the Wayback Machine and used it for the local instance for nostalgia reasons. This is entirely only accessible within my local area network.)

MoTK > Admin Center > Forum Maintenance > Database

The steps are screenshots below:





Like i said, i have no idea if it was actually enabled for those who have Administrator role, but if it is there no root access to the server platform is actually required to get the database. However this does not account for the other wikis that exist on this server.

That's very good news, at the end there is a simple way to make a full content backup of the forum.

HakureiSM

  • Reimu is all of it
  • I suddenly feel like I ate a crowbar.
Re: The End of Shrinemaiden As We Know It
« Reply #38 on: February 14, 2020, 10:12:50 PM »
Welp, looks like it's time to make use of that "download PMs" feature at last.

I don't see such a feature.

It seems there's a handful of people working on scraper scripts, thank you all for the work.
[20:45:19] Ciryano: come and behold why they call it the Panzerfaust
[20:45:39] Hakurei Reimu: ... because it shoots once and then you throw it out?
                                                                                   .

Kurzov

  • All good things must come to an end sometime.
Re: The End of Shrinemaiden As We Know It
« Reply #39 on: February 14, 2020, 11:00:05 PM »
What's the status of the High Score threads/attached replays, will they be manually moved over or ...?
« Last Edit: February 14, 2020, 11:06:03 PM by Kurzov »
"Better think awhile
Or I may never think again
If this were the last day of your life, my friend
Tell me, what do you think you would do then?"
~ Genesis (Undertow, 1978)

Kilgamayan

  • True
  • *
  • The Real Treasure Is You
    • Let's Play Super Marisa World
Re: The End of Shrinemaiden As We Know It
« Reply #40 on: February 14, 2020, 11:01:34 PM »
Since forum archive crawlers are being put forth, while I will make no administrative effort to block it, I would personally request any and all crawlers avoid recording LettyJournal, due to the personal nature of many of the threads there.
[22:40:12] <Drake> "guys i donwloaded esod but its not workan"
[22:40:21] <Drake> REPORTED
[22:40:25] <NaturallyOccurringChoja> PROBATED
[22:40:30] <Drake> ORGASM
[22:40:32] <NaturallyOccurringChoja> FUCK YEAH

[22:28:39] <Edible> Mafia would be a much easier game if we were playing "spot the asshole"

aListers

  • Vocaloid for the win!
  • Gensokyo forever!
    • Newgrounds
Re: The End of Shrinemaiden As We Know It
« Reply #41 on: February 15, 2020, 12:28:42 AM »
Well, I guess it's been a good run. As soon as I heard news of Helvetica passing I backed up my posts so I'm at no loss of history here. My dream diary is safe. That being said, I will be personally backing up my own topics on lettyjournal anyway - they're a good indication of what I thought about my life at the time. I don't want to lose any personal history of mine.

This website has had a huge impact on me as a person. From my dreams of Alice and Gensokyo to the somewhat biased political discussions we had, I've loved my time here. I really needed this place to be able to keep track of the brightest light of my life at a dark time - and while that light may now be put out, I will never forget it and the time I spent chasing it. Thank you to everybody who put up with my creepy and wierd postings on this fourm in my time here. I've cherished every positive message I've recieved on here and wish all the people here the best! Cheers to MoTK! May Touhou and Gensokyo live on forever!
???-2004?=dark ages, 2005?=atomic betty era, 2006=red dwarf era, 2007-2009=newgrounds era, 2009-2014= anime era.

Been good talking to you all. Gensokyo gu braith!

N-Forza

  • Information Superhighway Robbery
  • *
  • I said it was a steal, but not for whom
Re: The End of Shrinemaiden As We Know It
« Reply #42 on: February 15, 2020, 01:20:47 AM »
Oh, for some reason I assumed PM archiving was a feature but maybe I was thinking of another forum.

Barrakketh

  • You're suddenly Director of Fixing That Shit!
  • Vice President of It's Your Problem Now.
Re: The End of Shrinemaiden As We Know It
« Reply #43 on: February 15, 2020, 01:42:01 AM »
Oh, for some reason I assumed PM archiving was a feature but maybe I was thinking of another forum.
How important is the stuff in your sent/outbox? I have a "works for me" scraper to backup PMs, but it does require some setup on the users behalf. And it doesn't work on sent messages because I have none to test with :V
Cheating? I cannot even wrap my head around the point of it. Wouldn't you know you had cheated? How on Earth could you maintain crisp certainty of your superiority to all others? And if you're unable to do that, what's the point of anything?

Sparen

  • Danmakufu Artist
  • Git ready, git set, PUUSH!
    • AFCDTech
Re: The End of Shrinemaiden As We Know It
« Reply #44 on: February 15, 2020, 01:44:21 AM »
Where is that feature? I can't seem to find it. If there's a built-in method of downloading your PMs, I think it belongs in the first post.

Edit:
It was recently been brought to my attention that there's a Danmakufu Wiki hosted on this site. Has it been archived?

The Danmakufu Wiki doesn't change much so the entirety of the wiki is available on the Wayback Machine. In addition, since it has already experienced multi-month periods of downtime, I did contingency planning starting a few years back so a good portion of the important parts are already on my website.

Ref: https://sparen.github.io/ph3tutorials/docs.html

Karisa

  • Extend!
  • *
  • High scores are meant to be broken.
Re: The End of Shrinemaiden As We Know It
« Reply #45 on: February 15, 2020, 02:19:01 AM »
What's the status of the High Score threads/attached replays, will they be manually moved over or ...?
The attached replays have already been archived manually, and can be found in my scoreboard archive. The scoreboards will be recreated on v3, with gensokyo links and MotK v2 attachment links replaced by their archived versions, though it might take a while (I plan to make all the first posts consistent, this time around).
« Last Edit: February 15, 2020, 01:29:38 PM by Karisa »

Arcorann

  • hey catgirl
  • Surprisingly unkempt
Re: The End of Shrinemaiden As We Know It
« Reply #46 on: February 15, 2020, 03:47:12 AM »
Looks like someone added MotK to ArchiveBot, so at least all the public material will be saved.

Evil_Nazgul0616

  • Seg Fault
  • ERROR
Re: The End of Shrinemaiden As We Know It
« Reply #47 on: February 15, 2020, 06:02:28 AM »
I have a question about the status of Rumia's Party Games, particularly the Mystia's Stored Games subforum: The quests archived there tend to span multiple threads of 1000 posts per thread, which makes them a bit difficult and tedious to back up and preserve. I also heard that there were archive crawlers going around through the site, so what I would like to know is if the archive crawlers intend to back up MSG as well or if that would solely be our responsibility?

Re: The End of Shrinemaiden As We Know It
« Reply #48 on: February 15, 2020, 06:12:44 AM »
I have a question about the status of Rumia's Party Games, particularly the Mystia's Stored Games subforum: The quests archived there tend to span multiple threads of 1000 posts per thread, which makes them a bit difficult and tedious to back up and preserve. I also heard that there were archive crawlers going around through the site, so what I would like to know is if the archive crawlers intend to back up MSG as well or if that would solely be our responsibility?

I'm not sure the specifics of each bot but I think the crawlers are indiscriminate in that they are essentially just archiving anything within the forum structure. So if they are crawling the whole site, they'll get to mystia's stored games too.
Cirno the Ice Fairy~

N-Forza

  • Information Superhighway Robbery
  • *
  • I said it was a steal, but not for whom
Re: The End of Shrinemaiden As We Know It
« Reply #49 on: February 15, 2020, 06:22:23 AM »
How important is the stuff in your sent/outbox? I have a "works for me" scraper to backup PMs, but it does require some setup on the users behalf. And it doesn't work on sent messages because I have none to test with :V
Kind of important because it's how I've been keeping track of orders/requests (although I shouldn't be, admittedly) but most of it's already settled so it's probably the more recent stuff that's actually necessary to have on hand. I'll let you know if I need to make use of it, thanks.

Re: The End of Shrinemaiden As We Know It
« Reply #50 on: February 15, 2020, 06:54:50 AM »
How important is the stuff in your sent/outbox? I have a "works for me" scraper to backup PMs, but it does require some setup on the users behalf. And it doesn't work on sent messages because I have none to test with :V
I do have some so perhaps I could help with that, just send me the code. But wait... then you'll have a sent message so you won't need my help anymore. ;)
I have a question about the status of Rumia's Party Games, particularly the Mystia's Stored Games subforum: The quests archived there tend to span multiple threads of 1000 posts per thread, which makes them a bit difficult and tedious to back up and preserve. I also heard that there were archive crawlers going around through the site, so what I would like to know is if the archive crawlers intend to back up MSG as well or if that would solely be our responsibility?
I'm backing up all threads I can access with my account, so yes, that includes MSG.

Helepolis

  • Charisma!
  • *
  • O-ojousama!?
Re: The End of Shrinemaiden As We Know It
« Reply #51 on: February 15, 2020, 07:55:04 AM »
Late to the thread because of time zones and IRL.

RikaNitori contains quite some threads which need to be archived or moved. The wiki is definitely something that needs to be mirrored, but Sparen seems to already have taken care of that.

I will see if I can backup stuff from the sticky threads. The number of danmakufu activity did decline a lot over the past years. I can somewhat see the same happening over at the Eastern Communities so it is not just us.


@ Forza,  right I somewhat promised you to build that app for easier order tracking and such in the past. . . orz

Nemo★Ma

  • Have Faith.
  • *
  • And keep it.
    • NMforce
Re: The End of Shrinemaiden As We Know It
« Reply #52 on: February 15, 2020, 07:31:37 PM »
... ...

Just give me a call (via the forum PM, or discord (nemoma#0001) )if you guys need any help on moving the website (server choices, domain name related stuff -but that seems to be taken care of, backup/restore the forum, etc.).

From my experiences, while moving a site like MotK is a tedious process due to its size, it's certainly doable depending on how much one can backup.

Break anything that stands between you and them!

Lebon14

  • 椛ちゃん、助けてぇぇぇぇぇ!
Re: The End of Shrinemaiden As We Know It
« Reply #53 on: February 15, 2020, 10:56:22 PM »
@admins
https://old.reddit.com/r/Archiveteam/comments/f44mxj/touhou_forum_maidens_of_the_kaleidoscope_is_going/fhqfkia/
Quote
Hey u/ajshell1,

I work at Linode, and I'm hoping you can deliver a message for me to the operators of shrinemaiden.org's forum so that we can prevent the site from going down at all. I tried to register on the forum, but it appears registration is currently disabled.

First and foremost, I'm sorry for your loss. From the bit of reading I did, it sounds like TSO / Helvetica was a great friend who was a tremendous benefit to the Touhou community.

Through context from your forum I'm lead to believe that one of the administrators has access to a Linode user account, but they are having difficulty accessing the server via SSH because they do not have the root password. If they have permissions, they should be able to reset the root password of the server, and from there they can do anything they need in order to keep you up and running.

Thanks for forwarding this along if you can.

-Brian

Alfred F. Jones

  • Estamos orgullosos del Batall?n Lincoln
  • *
  • y de la lucha que hizo por Madrid
Re: The End of Shrinemaiden As We Know It
« Reply #54 on: February 16, 2020, 07:50:00 AM »
@admins
https://old.reddit.com/r/Archiveteam/comments/f44mxj/touhou_forum_maidens_of_the_kaleidoscope_is_going/fhqfkia/
First off, happy birthday (to me). (This might be my last birthday on this site and I'm not sure we'll even get to the site's 11th birthday at this rate.)

Second, thank you for passing that on. Unfortunately, it doesn't work. The linode login that I used can only be used to pay the bill and update the payment information used for the site and that is it. It can't even be used to open a ticket with linode, much less reset the root password.

I'm sorry. I wish I had good news with which to reply. Please thank that person for taking the time to try and help, at least.

Edible

  • One part the F?hrer, one part the Pope
  • *
  • It's the inevitable return, baby
Re: The End of Shrinemaiden As We Know It
« Reply #55 on: February 16, 2020, 08:15:41 PM »
Yeah, if we had full access to the linode account it wouldn't be an issue.  Unfortunately that's not the case.

Infy♫

  • Demonic★Moe
  • *
Re: The End of Shrinemaiden As We Know It
« Reply #56 on: February 17, 2020, 10:01:56 PM »
I spent the past few days scraping the site.
Here is every post on shrinemaiden.org that's accessible through an account.

it's all in a .csv file of about 800mb. I hope someone else can figure out a way to make it all easy to access.

HakureiSM

  • Reimu is all of it
  • I suddenly feel like I ate a crowbar.
Re: The End of Shrinemaiden As We Know It
« Reply #57 on: February 17, 2020, 10:19:11 PM »
127mb compressed

Thanks my man, I got a copy
[20:45:19] Ciryano: come and behold why they call it the Panzerfaust
[20:45:39] Hakurei Reimu: ... because it shoots once and then you throw it out?
                                                                                   .

nav'

  • nothing to see here
  • definitely not a Ditto
Re: The End of Shrinemaiden As We Know It
« Reply #58 on: February 17, 2020, 10:42:26 PM »
I spent the past few days scraping the site.
Here is every post on shrinemaiden.org that's accessible through an account.

it's all in a .csv file of about 800mb. I hope someone else can figure out a way to make it all easy to access.
Grabbing a copy too. Myself I have downloaded every thread I could access in print mode, which in total is of similar size when compressed with 7zip Ultra setting. Lightweight, but not so great because links to other posts and attachments don't work... Gotta put together something better.

Edit: I sort of wish your csv file also preserved the id number of every post.
« Last Edit: February 17, 2020, 11:24:46 PM by nav' »
Рабинович глядит на плакат ?Ленин умер, но дело его живет!?
? уж лучше бы о он жил!

Barrakketh

  • You're suddenly Director of Fixing That Shit!
  • Vice President of It's Your Problem Now.
Re: The End of Shrinemaiden As We Know It
« Reply #59 on: February 18, 2020, 01:31:02 AM »
Grabbing a copy too. Myself I have downloaded every thread I could access in print mode, which in total is of similar size when compressed with 7zip Ultra setting. Lightweight, but not so great because links to other posts and attachments don't work... Gotta put together something better.

Edit: I sort of wish your csv file also preserved the id number of every post.
I'll probably have finished writing my scraper by tomorrow, the next function is just grabbing the main page to get all the boards to crawl. Each thread is combined into one page, quote anchors are corrected to link to the right post, and attachments are saved (the high score entry board was my first test). Images hosted on shrinemaiden.org and in a post are downloaded (covers the staff badges and such, avatars, emoticons, etc.), and replay links pointing to gensokyo are rewritten to so they go to the lunarcast archive.

It's not a mirror of the board. More of a copy of the important content.
Cheating? I cannot even wrap my head around the point of it. Wouldn't you know you had cheated? How on Earth could you maintain crisp certainty of your superiority to all others? And if you're unable to do that, what's the point of anything?