Article: A Digital Audio Drawback - Music Storage Delicacy, Compatibility, and Longevity

2 minutes ago, Jud said:

 

 

Backblaze doesn't use ZFS, it uses a more sophisticated enterprise level system to ensure reliability and availability. See https://www.backblaze.com/docs/cloud-storage-resiliency-durability-and-availability .

 

However, @AudioDoctor is correct that ZFS is almost certainly the most reliable file system for storing files at home to ensure they're not corrupted before you back them up to the cloud.

 

@blaven, if you read the material at the link I sent, you'll see companies such as Backblaze make their living providing reliable backup to enterprise level customers, which is something you would never be able to duplicate in a home environment. Fortunately that level of reliability is available to individual consumers also.

Nice, I wasnt aware. Thanks Jud.

This is really great information and an incredibly helpful article with comments.

My problem with backing up to the cloud is retrieving the data should you ever need it.
 

If you have 6 TB of data and you then try to download it from the cloud at say 500Mbs, it would take about 26 hours or so.  Typically, anytime I’ve tried to do anything that large, I invariably have a failure somewhere in the chain, in the middle of the data stream. So do the companies like I drive and backblaze  send you a hard disk should you need it with your 6 TB?  What about some of these people that have 20 or 30 TB of data backed up in the cloud, it just doesn’t seem practical to download that. 
 

Practically speaking, it’s great that it’s secure and great to know that, but how do you actually get it, Should you need it- Download it from the cloud in bits and pieces and hope you have no interruptions?

3 minutes ago, blaven said:

This is really great information and an incredibly helpful article with comments.

 

My problem with backing up to the cloud is retrieving the data should you ever need it.
 

If you have 6 TB of data and you then try to download it from the cloud at say 500Mbs, it would take about 26 hours or so.  Typically, anytime I’ve tried to do anything that large, I invariably have a failure somewhere in the chain, in the middle of the data stream. So do the companies like I drive and backblaze  send you a hard disk should you need it with your 6 TB?  
 

Practically speaking, it’s great that it’s secure and great to know that, but how do you actually get it, Should you need it- Download it from the cloud in bits and pieces and hope you have no interruptions?

My contracted minimum download speed from my ISP is 1.2Gbps, and actual speed is faster than that, in the 1.6-1.8Gbps range, so I'd be talking more like 8-10 hours. But let's assume your 26 hour time frame is the correct one.

First, yes, I think you can get a drive from Backblaze with your data, though I haven't checked on that in some time so I apologize that I don't have details for you.

Second, the tools I use (like rsync, rclone and their various front ends) work by downloading (or uploading) changes, so whatever you've already downloaded before an error hits (if any does) isn't wasted - the download will just start by redoing the last file that had the error and go from there.

Third, waiting a day or two to get your data back doesn't seem like the worst thing in the world when the alternative is losing it forever.

25 minutes ago, Jud said:

 

My contracted minimum download speed from my ISP is 1.2Gbps, and actual speed is faster than that, in the 1.6-1.8Gbps range, so I'd be talking more like 8-10 hours. But let's assume your 26 hour time frame is the correct one.

 

 

 

Third, waiting a day or two to get your data back doesn't seem like the worst thing in the world when the alternative is losing it forever.

Agree on your points, but not everybody has access to those speeds. I suspect it will only improve in the future, but I don’t have access to that where I live in Sweden.
 

my problem isn’t waiting for the data, as that is a trivial price to pay for the security. My problem is usually I end up having some hiccup in the download with any attempt at a huge data transfer, then need to start over; or move it in bits and pieces. I’ve just never had success downloading anything that large or even transferring it to my Synology over Cat6 when I had that setup. Synology customer service had always told me to move it in smaller chunks. 
 

But where does the madness end ?? I mean how do I know when one of the internal drives on my Aurender has issues?  
 

I guess I need to explore rSynch / Rclone and that’s the greatest learning point for me from this excellent thread. 
 

thanks Jud and all. Super helpful stuff. 

11 minutes ago, blaven said:

But where does the madness end ?? I mean how do I know when one of the internal drives on my Aurender has issues?  
 

Ah, sorry I can't help with the Aurender - my HDDs are simple external units on which I can run the manufacturer's analysis software.

11 minutes ago, blaven said:

I guess I need to explore rSynch / Rclone and that’s the greatest learning point for me from this excellent thread.

They are usually used from the command line and quite powerful, so you might like to practice with "toy" backups until you have some confidence you've got things working as you like. Then it's easy, since as mentioned if there is an error these utilities won't re-upload files that have already been correctly synced, they will resume with the file that had the error and go from there. Rclone is the one that has been made specifically for cloud backup, so you will want to use that unless you use a Mac, in which case Forklift and Chronosync are available to you as nice GUI applications.

41 minutes ago, blaven said:

But where does the madness end ?? I mean how do I know when one of the internal drives on my Aurender has issues?  

When a song wont play.

You could oreder a disk or disks from Backblaze, plug them in via USB or maybe even a network, I am not sure, and copy your data to it locally which is much faster, and then send the disks back to them. Backblaze then copies that to their cloud probably directly and faster.

Or you can buy/build whatever a machine that runs TrueNAS and use it's built in Backblaze B2 sync.

My Backup path looks like this:

Local computer to Truenas to Backblaze. My local computer uses ZFS as does TreuNAS. Backblaze apparently uses a similar but more sophisticated file system. I use Minimserver and JPLAY with HQPlayer because HQPlayer doiesnt play well with ZFS directly. Minimserver does not care, nor does HQPlayer if its just getting a file from Minimserver. At the moment, Minimserver runs on my local machine because getting it working on TrueNAS is proving frustrating.

26 minutes ago, blaven said:

But I only crank up the Milli Vanilli Japanese Import Anniversary Edition every few years or so :)

 

so, In the meantime how would I know if that Aurender file is corrupted 

You dont until a song wont play.

Is there an alternative to Backblaze in Europe?

THX

Torben

8 hours ago, AudioDoctor said:

 

I lost a lot of photos to a Time Machine/iCloud backup scheme. Time Machine and iCloud will happily backup bad data and erase teh good data to make more space for the new, bad data.

 

At a minimum, Backblaze has unlimited personal storage for, I think $10 USD per month, and a Mac App that handles the backups for you. You can use versioning on Backblaze (maybe, I use B2 so it may be different on the personal account) and if you wanted to buy the Forklift App, know beyond any doubt your data is good on Backblaze.

None of my backup disks for Time Machine or CCC are anywhere near full so as far as I know, nothing is getting erased. I replace the HD’s every couple of years so my oldest backups are that old. In any event, are there any backup solutions that won’t backup “Bad Data”? I’d like everything backed up and if there’s a problem I can sort it out from there. If I understand Chris correctly he allows himself about a month to find and correct a problem. I have been meaning to look into Backblaze for several years now, maybe it’s time to get off my butt and do it.

14 minutes ago, daverich4 said:

are there any backup solutions that won’t backup “Bad Data”

Yes, at least two have been mentioned.
 

You can have something as simple as I do, just two off the shelf external HDDs on which you can run the manufacturer’s drive analysis software (most manufacturers offer such software). The chances two of them will have a normal failure mode at the same time are vanishingly small. In that case you get rid of the bad drive and have the good drive to restore from. Or in the case of some disaster - fire, flood, electrical storm that fries your system - you have the offsite backup to restore from. And you can use the everyday tools used by millions - rsync, rclone, an app that incorporates one of these, or similar software, all of which do checksums to ensure offsite backups are bit for bit identical.

Or you can have something as sophisticated as @AudioDoctor’s home setup, and again use those readily available tools to ensure your offsite backup is bit for bit identical.

Good article. Keep in mind, that for most, probably upwards of 90% of their music collection can be replaced by streaming or downloads or re-ripping. It will just cost time, stress, and money. But 100% of your digital photo collection is irreplaceable. Nobody else has your photo files available to replace. So be sure to implement just as robust backup policy for all of your digital files, esp the ones that are cherished memories or your livelihood.  

On 8/23/2024 at 12:37 PM, stefano_mbp said:

The whole streaming service library will be lost … the same will apply with any subscription based service like Roon, Audirvana …

That’s why i bought roon with lifetime  license and not subscription. Second they changed the habits so even the company disappears it works. Hopefully HK will survive since now roon is in there hands.

6 hours ago, daverich4 said:

None of my backup disks for Time Machine or CCC are anywhere near full so as far as I know, nothing is getting erased. I replace the HD’s every couple of years so my oldest backups are that old. In any event, are there any backup solutions that won’t backup “Bad Data”? I’d like everything backed up and if there’s a problem I can sort it out from there. If I understand Chris correctly he allows himself about a month to find and correct a problem. I have been meaning to look into Backblaze for several years now, maybe it’s time to get off my butt and do it.

Disks still go bad, just because they're not full means nothing. It is absolutely not a matter of if, rather it is when.

10 minutes ago, Me2 said:

Second they changed the habits so even the company disappears it works

Never read about that … are you sure? … and what about the need to be internet connected to use Roon?

I own a big qnap NAS with 75 terabytes of disk. I don’t use Backup but i get a message if i a disk start to fail so i can replace before and the Raid will fail. I know that’s not 100% secure but right now i live with that. I have 1530 Albums in Roon right now and around 50TB movies all in 4K for my homecinema. I work a IT Provider and build and maintain our DC Network. Right now backup such a high amount of data in the cloud is still to expensive even with backblaze.

1 minute ago, Me2 said:

I own a big qnap NAS with 75 terabytes of disk. I don’t use Backup but i get a message if i a disk start to fail so i can replace before and the Raid will fail. I know that’s not 100% secure but right now i live with that. I have 1530 Albums in Roon right now and around 50TB movies all in 4K for my homecinema. I work a IT Provider and build and maintain our DC Network. Right now backup such a high amount of data in the cloud is still to expensive even with backblaze.

75 terabytes would be ~450 a month on Backblaze B2. Not a trivial amount but it is at the point where it may make financial sense to get a second NAS offsite and back your current NAS up to that.

7 minutes ago, AudioDoctor said:

 

75 terabytes would be ~450 a month on Backblaze B2. Not a trivial amount but it is at the point where it may make financial sense to get a second NAS offsite and back your current NAS up to that.

This made me think of two things. 
 

1. A new business where people store their backup NAS and pay a small rental fee to keep it there for offsite backup. 
2. Why is it so dang expensive - $450 per month on sites that already have the infrastructure? 

20 minutes ago, AudioDoctor said:

 

Disks still go bad, just because they're not full means nothing. It is absolutely not a matter of if, rather it is when.

My comment was in response to yours saying that Time Machine will delete older files when it needed the room & I pointed out that I have plenty of room on my backup HDs so files are not getting deleted. I’m aware disks will fail which is why I change mine out every couple of years although in my case I’ve only had 2 disks fail over the course of more than 35 years of personal and professional use. It does happen and hopefully I’m prepared for that event.

20 minutes ago, stefano_mbp said:

Never read about that … are you sure? … and what about the need to be internet connected to use Roon?

Roon has gone back to not needing an internet connection to use it. I think it still needs to phone home every once in awhile but they have stated on numerous occasions that if something happens to the company they will make whatever changes need to be done so that the software will be usable although it won’t have any new meta data updates and so on.

49 minutes ago, The Computer Audiophile said:

This made me think of two things. 
 

1. A new business where people store their backup NAS and pay a small rental fee to keep it there for offsite backup. 
2. Why is it so dang expensive - $450 per month on sites that already have the infrastructure? 

1: Interesting idea... Hmmmm

2: I don't know, I ask myself that question every month when Backblaze takes their monthly charge...

40 minutes ago, daverich4 said:

My comment was in response to yours saying that Time Machine will delete older files when it needed the room & I pointed out that I have plenty of room on my backup HDs so files are not getting deleted. I’m aware disks will fail which is why I change mine out every couple of years although in my case I’ve only had 2 disks fail over the course of more than 35 years of personal and professional use. It does happen and hopefully I’m prepared for that event.

okay.