How the Annotation Worker ...Worked

So the annotation thing. You remember that, right? Well, here is how the worker seemed to function. Note that I'm getting this information from a brief cursory glance (and chatting with one of the devs). I know it works because I had three of them running at any given time. But how? Uh, *shrug*

Let's get started, shall we? So the worker (at omarroth/archive) the code starts by creating a new Worker class. This is our basic worker.

The run function creates a BatchProcess and calls its run. *sigh* So what does that do? Well it asks the server for a batch, pulls it up from a database, and retrieves the annotations for each of them ...which is done in yet another class, this one called AnnotationProcess.

So what does AnnotationProcess do? It does a request to YouTube to get the annotations. (The URL in the repository was changed after the fact. By me. Interesting.) How it gets those annotations is interesting: to make sure the worker is functioning properly, there is a trust system. A fresh worker won't actually get a new batch; it'll get one that's already been verified. As it gives more valid responses, it's more likely to get a new video. This way, the likelihood of getting garbage data is minimized, which is important for an archival project.

Once all the videos in a batch have been downloaded, they're verified with the server and then uploaded to DigitalOcean Spaces, a cloud storage service. This goes on ad infinitum until YouTube decides to pull the plug.

And that is what (I think) the annotation worker did.

- glmdgrielson

Weekly Summary 02/24/2020

This week was rather calm, with mostly positive developments.

A quick update on the Bethesda forums from last week: The shutdown has been rescheduled to March 9th, so that's 2 more weeks from now to grab anything you still haven't backed up.

Besides that, the Internet Archive recently digitized a large collection of Russian books. You can read more on their blogpost: https://blog.archive.org/2020/02/21/russian-book-covers/.

Stay tuned for next week's summary, where we'll hopefully see each other again same time next week at EST 9 AM!

Weekly Summary 02/17/2020

The focus of this week is a new Archiving Project on documenting information around us about the recent Coronavirus outbreak.

The Internet Archive and the IIPC are working together to collect resources regarding the recent  Novel Coronavirus (Covid-19) outbreak. The goal here is not just to collect academic research information but also to document containment efforts and social effects. Any of the following are of interest:

• Coronavirus origins 
• Information about the spread of infection 
• Regional or local containment efforts
• Medical/Scientific aspects
• Social aspects
• Economic aspects
• Political aspects

So go ahead, feel free to share news or shares on social media as well even if you don't have in-depth information on the matter.

If you would like to nominate any websites or web content covering the outbreak for archiving you can use this form here: https://forms.gle/iAdvSyh6hyvv1wvx9. The collection will later be made available to all at the IIPC's archive-it page: https://archive-it.org/home/IIPC, where you can already find some interesting examples from previous collections on the Olympics or the European Refugee Crisis to give you an idea of the goal here.

If perhaps you would like to share offline content such as photos of pamphlets you may have encountered at school or in the workplace, the Archive Internet archive encourages you to upload to a collection of your own, preferably tagging it as 'coronavirus' so people will be able to find it in the future. Here's a featured example: https://archive.org/details/2019-nCoV

For more information see https://blog.archive.org/2020/02/13/archiving-information-on-the-novel-coronavirus-covid-19/ and https://netpreserveblog.wordpress.com/2020/02/13/cdg-collection-novel-coronavirus/.

In other news, the old Bethesda forums (https://forums.bethsoft.com) are expected to be shutting down in a few hours. The forums had been read-only for the last 2 years or so, and there are quite a few snapshots. The community has already moved onto the aptly named bethesda.net/community.

That's about it for this week.

Stay tuned for next week's summary, where we'll hopefully see each other again same time next week at EST 9 AM!

Keep Circulating the Tapes

If you recognize the title, chances are you know what I'm about to talk about. For those of you who don't, the phrase appeared in the credits of the first four seasons of a show called Mystery Science Theater 3000. If you're not aware of the series' premise and wondering why they're encouraging something like this, note that one, it was on a network that at the time was just barely getting started, namely, Comedy Central. Two, the premise of the show is that it features actual B-movies. Naturally, this means that getting the rights to episodes of their own show can be a bit of a hassle. Reasons for this range from Toho deciding no, you can't have your movie to the rights holder being so paranoid she won't even let it be shown at conventions.

So recording episodes and sending them to your friends became a part of being a fan of the show. There's even a mention in the episode covering The Magic Sword of a parent sending tapes to their son as a means of connecting with him. And of course, there are plenty of uploads all over YouTube.

- glmdgrielson, a big nerd

Weekly Summary 02/10/2020

The main event this week has been recovery efforts following the fire that broke out near the Museum of Chinese in America (MOCA) back in January.

MOCA is deeply saddened and shocked by the devastating fire at Chinatown’s beloved 70 Mulberry. The MOCA team stayed on site until hoses stopped last night. We have reached out to emergency conservators. Thank you for outpouring of community support re: MOCA archives. pic.twitter.com/QqD1vFU5kO

— Nancy Yao Maasbach (@YaoMaasbach) January 24, 2020

Initial estimates of how much material had survived the ordeal were rather pessimistic, with reports claiming that if the fire was unable to reach any artifacts the water used to put it out was. Conservators from museums across New York and other volunteers have however decided to give Sun Zu a run for his money by attempting to recover the likely drenched documents.

The current status report: An estimated one third of the archives has been retrieved from the building, and 80% of this retrieved batch have been afflicted with moisture but appear salvageable. In addition 35,000 digitized objects have been recovered from back-ups. For further updates you can visit MOCA's own page http://www.mocanyc.org/visit/ or follow MOCA's Twitter.

This damaged scroll was recovered from our archives. It says Museum of Chinese in the Americas - MOCA’s name before 2009. During transfer to our recovery area, it was accidentally placed upside down. Seeing it that way, it feels like the fire has turned MOCA’s world upside down. pic.twitter.com/Q0hAEi14bv

— Museum of Chinese in America (@mocanyc) February 10, 2020

MOCA is an active advocate for archive digitalization, and in fact they had an event "Digitilzation  Days" scheduled for this week -which has been obviously cancelled- where attendees would get a chance to digitize family photos, documents, VHS's, cassettes tapes, and even vinyl!

A GoFundMe has been set up and for readers in the New York area those of you who might be interested in volunteering should contact firerecovery@mocanyc.org .

In other news, this week The Eye had quite an unusual hardware test. A fairly large (over 500 GB) collection of assorted magazines is being served "temporarily", with it being stated that they would likely later be handed off to the Internet Archive for safekeeping. 

These aren't anything too old, mostly magazines from 2019 but also few from this year. You can expect to find anything from Vogue to three different regional editions of the Financial Times. 

Stay tuned for next week's summary, where we'll hopefully see each other again same time next week at EST 9 AM!

MSN Messenger is Back!

Think about the first instant messaging service that you used. Chances are it has been discontinued many years ago and got replaced by today's giants such as Skype or Discord. Except that there is a chance someone revived it!

Introducing the Escargot MSN Server. Since 2017, you can use your favourite (supported) version of MSN Messenger once again whether if it's for nostalgic reasons or just for fun. Follow the instructions, insist your friends to use Escargot and voilà! You're back in 2005.

Some versions has more bugs than others, and not every function is supported yet but regular instant messaging is in working order.

If you want to find out more, you can visit their website: https://escargot.log1p.xyz/

Weekly Summary 02/03/2020

A warm welcome to a cold February!
This week we have some updates on Yahoo Groups, and a couple of shutdowns.

We recently passed the deadline for the shutdown of Yahoo Groups, but as of me writing this it seems that the "get my data" services are still functional. Currently Archive Team's own Tracker reports 2.76 TB of data to have been saved. The "Save Yahoo Groups" team (aka Yahoo Groups Fandom Rescue Project, aka Yahoo Gedden) hasn't fully tallied up their data yet, but have counted the number of groups they've downloaded so far to be about 84,655 and the number of groups they are still retrieving from to be about 38,000 for a total of 123K groups! We did a more detailed report on the situation earlier this week which you can read here.

The SRPG/Card Game Hybrid Duelyst was announced to be shutting down later into February a few days ago: 

We are saddened to announce that the Duelyst servers will be shutting down permanently on February 27th, 2020 at 3:00 PM PST. Saying goodbye to any game is never easy, and it’s always hardest when you love the game as much as we all love Duelyst. We’re incredibly proud of the effort our developers have put in on Duelyst over the years, and even more proud of the amazing community of friends and gamers that have enjoyed Duelyst.

We want to thank all of you for being a part of the Duelyst family, and for making our time working on Duelyst the amazing adventure it has been.

As a final thank you, the prices of in-game items have been lowered significantly and refunds are being offered for recent in-game currency purchases.

Seeing as the game requires a server connection it will be as good as dead. There is already discussion on fan revivals/sequels on the r/duelyst subreddit for anyone interested.

Finally, the goth social network Vampire Freaks was shut down two days ago, on February 1st. The site's shutdown had been announced months ago, so it likely didn't come as a surprise to much of the community. A portion of the diaspora of users seem to have migrated to the Discord Server of r/goth for anyone who might be looking to find a few old friends.

Stay tuned for next week's summary, where we'll hopefully see each other again same time next week at EST 9 AM!

Why is this important? Or, the story of Victor Trembly

So let's start off with a story. It's 2013 and I've just found out Homestar Runner made a reappearance. Strong Bad gets his name wrong and calls him Homestuck. I check that out and eventually I decide to check out the forums. I get acquainted and even start my own little thing called NetHacked, starring Victor Trembly. (Female, by the way. Name and gender were picked by two different people.) This of course died a quick and swift death. Then I retool it into "Victor has gotten loose and I need to get him back". This is moderately successful in the sense that I have some sort of audience. I had just dropped forcing him to watch MM8 cutscenes, MST3K-style, before all of a sudden, the forums disappeared. As far as I know, there was no backup. This is not a good thing. Preventing that sort of nonsense is why we're here. Because people put effort into things and they deserve to stick around.

@
Victor says hi.

- glmdgrielson