Backing Up the Petabyte Server

wot

Jesus Christ, that title.

4 minutes ago, BachChain said:

Jesus Christ, that title.

They are embracing the clickbait

I love it

You should use Wasabi. They don't charge for pull and are like 80% cheaper than AWS and Google Cloud.

We use them for video storage and streaming. 

Why not try something like a GCP physical appliance to transfer the data? 

https://cloud.google.com/transfer-appliance/

Ooooh, I loved that reference to Nicky V at around 3:00!

Also, is it just me.. Or was the 'fire' vocal added in post at 0:24?

And now you may have made some data hoarders upset. Now more people might do this and Google may crack down on the loophole.

For my off-site storage needs, I simply went with a sharing an extra NAS with family in another state. Basically you mirror 2 NAS devices, and you then limit yourself to use like half of the available space, thus without any extra monthly fees, you get offsite storage for 2 different users, as you each act as each other's remote backup. Compared to a cloud service, it pays for itself within a year if you have a lot of data, e.g., each user gets ~10TB of backup space on a 20TB NAS setup, best of all, you get more flexibility as your network storage can be accessed by all of your media streaming devices, e.g., saving a media collection and being able to stream it to any of your devices, as well as simply treat the backup as any other drive on your PC, and have 1 Gbps access (or more with teaming) to your files instead of dealing with a slow download over the internet.

I guess I don't understand how those tapes you could find were so small When this article https://www.popularmechanics.com/technology/a27602/330-tb-magnetic-tape/ came out just under a year ago how is this possible? I'm just a little confused about the whole thing.

There's Amazon Glacier  and stuff like Snowball, where Amazon sends a box to you, you connect it to your network, transfer files to it and ship it back to them.

It is expensive, paying something like a few cents per GB ... so you'd still pay probably hundreds of dollars per month just for archival... and yeah, you would be hit with a retrieval fee is the data is archived (after a few months) to get it back.

6 minutes ago, mariushm said:

There's Amazon Glacier 

The 3 thousand per month solution they were looking at WAS Amazon Glacier

47 minutes ago, Verond said:

You should use Wasabi. They don't charge for pull and are like 80% cheaper than AWS and Google Cloud.

Still 3 thousand a month with their amounts of data

50 minutes ago, benjamin238 said:

Why not try something like a GCP physical appliance to transfer the data? 

https://cloud.google.com/transfer-appliance/

That is for uploading to Google cloud storage, which is an more or less enterprise solution, for them it would cost 5 thousand a month

2 minutes ago, ignaloidas said:

Still 3 thousand a month with their amounts of data

If considering long term use, it is cheaper to just spend a bunch up front and build a fire/water resistant vault, and then stack it full of bare drives as you make more and more cold backups.

Basically at about $200 per 8TB drive, you can afford to buy 15 additional drives or about 120TB worth of storage every month.

I currently use a fire and water resistant safe for storing a bunch of hard drives that I use for cold backups since it is easy to just stick them into an eSATA dock, and backup an image of my drive C + an encrypted backup of any other files on different drives that I want to backup, and then place the drive in the safe, and I just keep them in order and overwrite the oldest backups as needed, thus I maintain around 2 or more revisions of backups, along with the ability to adapt to sudden storage needs by dropping down to 1 cold copy of the data, and then I can slowly build up more drives as I catch different sales and coupon codes for good deals on drives. It takes more manual work, but it is not too hard to start copying a bunch of files before heading to bed or going to work.

Wouldn't this make their 3rd storage location since youtube counts as one, right?

What happened to "Clover Server" for offsite backups? Link:

2 minutes ago, Razor512 said:

If considering long term use, it is cheaper to just spend a bunch up front and build a fire/water resistant vault, and then stack it full of bare drives as you make more and more cold backups.

They were considering tape drives for 15 thousand up front, but if you add things up, for that price to beat their google solution, those tape drives should last for more than 200 years. And tape-drives are one of most cheap means of storage...

Just now, poochyena said:

Wouldn't this make their 3rd storage location since youtube counts as one, right?

YT isn't consider one as it ! not the raw footage and 2 it's not the full video IE not everything recorded

2 minutes ago, yt0097 said:

What happened to "Clover Server" for offsite backups? Link:

 

Way to small for their huge amount of data they have now. As mentioned in the video they have about 700 TB of data so you know Clover Server is tiny. Which means they most likely took it down ages ago.

8 minutes ago, SgtSnipey said:

YT isn't consider one as it ! not the raw footage and 2 it's not the full video IE not everything recorded

Why would they need all of the raw footage though? Tbh, I don't see why they need to backup reviews of 3+ year old hardware anyways.

3 minutes ago, poochyena said:

Why would they need all of the raw footage though? Tbh, I don't see why they need to backup reviews of 3+ year old hardware anyways.

So they can use its new video to look back at the raw footage and use it in new videos it gives them more options for editing. Also, some projects are paid by companies and they have to keep the footage for those.

13 minutes ago, SgtSnipey said:

Way to small for their huge amount of data they have now.

I wonder if they still use it for (smaller) critical data? Seem's like renting out a 4u space in a data center would be cheaper than other solutions, and more legit than what they were doing with google. Just run a dense hdd server with only one or two drives for redundancy (since the sever itself is for redundant).

1 minute ago, yt0097 said:

I wonder if they still use it for (smaller) critical data? Seem's like renting out a 4u space in a data center would be cheaper (and more legit) that what they were doing with google. Just run a dense hdd server with only one or two drives for redundancy (since the sever itself is for redundant).

Using it for critical data seems like a good idea, maybe they are doing it. As mentioned in the video their ISP lets them host servers at their data center. So they are probably not renting space at a data center. But 700 TB of drives and the server would cost 70k. Going through a company means they would not have to worry about running multiple servers at different data centers for 100% data protection. 

You could technically also abuse Carbonite

https://www.carbonite.com/backup-software/carbonite-safe/

or Backblaze

https://www.backblaze.com/business-backup.html