This is not a post on how to choose a NAS box. This is also not a post to provide a step-by-step guide on how to configure one. This is also not a post on how to do enterprise backups, while some concepts apply, i’d say that this is mainly for personal backups.

This is a post to share the choices that i made to create a back up plan for my media such that i minimize the risk of losing the media that has been accumulated throughout my PERSONAL life. Some of these are irreplaceable memories of phases, experiences that are usually nostalgic and you definitely don’t want to lose them.

[This post was inspired by a conversation that i had with a retiring colleague/friend who just ventured to purchase a Synology NAS, with the goal of backing up his photos from his upcoming travels and perhaps to catalog them. As we chatted, i realized that i am repeating the comments, recommendations that was provided, to many people, many times over the years.

My choices has served me well all this while and has been running the same way for almost 10 years. I have not had major issues and failures, even though some outages has induced panic for a while, it has always been recoverable.

I hope that by sharing my backup strategies and rationale here, it will help someone in the future.]

Different storage types available

Before deciding the backup strategy, one must recognize that there are different storage types, and each has its own advantages and disadvantages. i divide them into the following 3 classes.

1. External Drive – This includes USB drives, Thumb drives, even expensive SSD drives
2. Cloud Drives – Any kind. iCloud, Amazon, take your pick
3. NAS Drives – Synology, QNAP, DIY, pick your poison

Each has its drawbacks.

External Drives – Cheap, and no recurring cost. No internet connection required. But generally, you cannot automate the backup and requires the user to KNOWINGLY activate the process. The Backup is only as good as when you last connect the device. If the external drive dies, however, the backup is gone too.

Cloud Drives – Very convenient, but has a subscription cost. If you are travelling, or if you are retiring, every cent counts. Backup time and recovery can be SLOW as it can only be as fast as your bandwidth. The convenience and off-site nature of these are usually the plus points as you don’t have to worry about hardware maintenance. Just the network and as long as the service provider continues to exists.

NAS Drives – Expensive up front and requires knowledge for self hosting. The barrier of entry is the maintenance knowledge required and can be daunting trying to setup. You wholly own the maintenance and you’d generally have to FIGURE IT OUT ON YOUR OWN. But once its done, you generally get the ease of use of a Cloud Drive and better protection than an External Drive.

One recent concern i have had was Legacy planning. One can setup something that is suitable for yourself. But what happens when you pass? How is the media stored supposed to be extracted by your next of kin? I provide my views briefly in the last section.

Needless to say, i believe each of these storage types must be combined to provide a complete backup strategy.

Where do you want to backup from?

In the past, we only had to deal with data or media generated by computers (laptop or desktops). We later had to deal with digital cameras, and over time, this expanded to Smart Phones, tablets (collectively mobile devices), and whatever the future might hold. Regardless of source, the software MUST exist in order to facilitate the backup or retrieval.

On a computer, tons of options exists. A fair bit of options exists for mobile devices as well. But some others will be limited, such as digital cameras. The longevity of the software is also an issue. What good is a backup if the method of retrieval is no longer valid, or overly complicated to use?

Some data sources good option that enables automation. One such example is Time Machine backups on the Mac OS. RSync is also a pretty good option on Linux. (i’m a both a Mac and a Linux user). I am sure similar options exists natively on Windows. Selecting the right software is important as it will help lower the friction of creating backups and therefore increasing protection.

Mobile devices can often rely on cloud backups offered by Apple or Google. These are automatic and pretty easy to activate. Otherwise, you would have to connect (Wifi or Cable) to your computer (i.e. software) and create the backup. iOS does it using iTunes (its since been integrated into Finder), but i’m not sure how Google does it.

Other interesting sources like SD cards from DSLRs or Home video cameras would require some effort to transfer the media. I find this to be inevitable. You can either manually copy the contents of the card, or make use of solutions like WiFi SD cards (Eye-Fi?). However, i have found that the technology doesn’t keep up to date. So manually transferring the data into a computer remains the best. Some cameras offers cloud based backup solutions that requires a subscription. I find those solutions un-necessary, as i prefer to have and own my data. Once the media is transferred either manually, FTP or otherwise, backup can then be proxied by either a Computer, Mobile or directly into the NAS, Cloud, or another External drive type.

Ensuring redundancy

Most people will only start thinking about backing up when their hardware in any of their sources starts showing its age. For others, the sudden loss or failure of their devices would have already left them in a state of panic.

Being able to think about how the media that is generated can be made redundant at various stages of its life-cycle can be very useful. This includes:-

1. The moment the media is created
2. When the media under goes transport from one place to another
3. Storage and archival of the media
4. Retrieval of the media

Each of these stages would require some amount of redundancy as all things can fails. Hardware / Software, even physically like dropping your device or losing it. Take reasonable steps to protect the device from physical damage, but also try to ensure that from the moment the media is created, a separate copy can be made as soon as possible.

When then media is being transported, perhaps when you are moving from one place to another, there should be multiple copies in transit. This prevents media loss in event any one of those devices, or storage, is lost or damaged. Imagine with you dropped your mobile device into a river, or if your luggage is lost during travel.

Once all the media is created and consolidated at some home base, the storage and archival on the media will usually get offloaded into some computer or any of the above mentioned drives. One has to then consider the potential failure AND recovery of any of the above said drives. The main consideration to address is the likelihood of that drive being damaged or destroyed. Something like hard disk failure, natural disasters should be acknowledged (even if not necessarily addressed). Luckily, there are lots of technologies in place to assist in this. Things like RAID, SHR, ZFS, offsite backups, Backblaze all aims to solve this problem. YMMV depending on your willingness to go into the tech, but these are all already fairly well documented.

Not being able to retrieve the media or data in a usable way renders the backup useless. Deciding on the access methods not just for yours self, but your family members is also key. This would not only make the entire strategy usable, but may possible provide the budget, once they appreciate the backup strategy might help them now or in the future. Perhaps an app, or web tool can be very helpful.

Nuts and Bolts – My Strategy

There is no one size fits all strategy as it depends on the nature of each user group e.g. family, house mates etc, but I leverage all three drive types for backups. Depending on the device type and the stage the data is in the life-cycle, one optimizes for cost effectiveness, effort and convenience. i therefore use different procedures to try ensure whatever media generated wouldn’t be lost. Some are automated, some are not. The strategies are little varied for different devices, but progressively gets more consolidated until it finally reaches a central NAS. I’d try to summarize this the best i can without getting too much into specific configurations.

1. All mobile devices will use Cloud drives / services to backup media and / or data. As all my mobile devices at home are iOS. iCloud is used to back all media. This leverages iCloud photo library. It also is able to backup all other content in the mobile device and can also serve to recover the device in event of loss during transportation (see above lifecycle). Cost is SGD$3.98/mth for 200GB. Note: Why not 2TB? As i want to force excess to be off-loaded to other forms of storage which has higher redundancy.
2. All Laptops leverages 2 copies of Time Machine (i am a Mac user). One would be an External Drive (1TB) for both backup during transportation / travel, as well as a Networked Time Machine drive when at home enabled by Synology NAS (See point 5). The External Drive would require a manual step to backup by plugging the device into the laptop. The External Drive is left in the hotel when travelling and is therefore safe from transport loss. It is typically used to back up the laptop at the end of the day. This will serve to protect the laptop during transportation and can act as a checkpoint backup to any media created during the day. i.e. any data loss is limited to the day before it. Otherwise, if the laptop is left in the hotel, it can continually backup throughout the day. When the laptop returns home from travel, it would then automatically perform an entire backup into the Networked Time Machine. This also serves as a checkpoint if anything drastic happen during the course of travel. Similarly, Desktops and workstations leverage the Networked Time Machine. the backups would then occur automatically. But as they are not mobile, that is all that is required. The schedule of the backup is left as the default. Note: I do have a Linux desktop which i use daily. That uses TimeShift, which is very similar to Time Machine, together with the Gnome backup on Ubuntu., pushing the data to my Synology (see point 5).
3. Some Data or Media within the laptop is typically saved into some cloud service. I leverage a self hosted NextCloud instance, but it can also be Box, or Dropbox, or iCloud Drive. This provides a form of off-site data redundancy in event of device loss as well. In an emergency, it ensures that i would have access to data via a web browser if required. I don’t do the same for mobile devices as i already protect its data with a cloud backup via iCloud. However, i do ensure that my mobile devices can access these cloud services to retrieve data if required.
4. A portable NAS device is used to provide media backups on the go. i.e. Western Digital MyPassPort Wireless or a Seagate Lacie Fuel. I use a modded portable NAS device to provide FTP service for all mobile devices to backup its photos and videos via the PhotoSync software on iOS. As these digital media must not be lost (all those precious trip memories!), PhotoSync is used to upload all media into this drive to protect these newly created media and provide redundancy during travel and limits data loss in event a device is lost. While this seems to duplicate point (2) and (3), this allows me to take a vacation without a laptop and still maintain a reasonable level of redundancy.
   
   Note: PhotoSync is really powerful and it allows auto-creation and sorting of media into folders, which is a huge time saver when combined with the portable NAS. Once configured, it is a 1-click operation. This software is also configured for the whole family to survive the trip. I would also not encourage to delete the photos from the Photo Album on device, during a trip. Unless really necessary, it allows a higher level of convenience for photo browsing during the period of travel. Cost of PhotoSync is Free (or paid if premium required).
   
   Note: I have moved away from those WD and Segate devices as they no longer seem to be updated by the manufacturers. I have since developed a RaspberryPi that does the same. But any similar device would work.
   
   Take note that even if a user forgets to perform a PhotoSync backup, the created media is also reasonably protected by the cloud storage in event of device loss. But if the user does the backup, it provides an alternate copy immediately available if its required.
5. All media gets copied / archived into a Synology NAS when the devices is home from travel (see life-cycle). This is once again done via PhotoSync. This maintains a consistent user interface to perform the backup. i.e. “doing a photo backup? just click on that button in Photosync!”
   
   Synology NAS is used as it provides the Networked Time Machine service mentioned in point (2). It also provides the FTP service required in point (4). Without going into too much details, the Synology is used as it provides lots of easy to use file services and functions for advanced configuration of the drive. The drive configuration is the most important as it provides hard drive redundancy such that a disk failure will not compromise the availability of data. i.e. it wouldn’t be lost. The HDD configuration i use in the Synology is as follows:-
   • SHR-1 (NOT RAID5 as it is not expandable. SHR1 provides 1 drive redundancy as well as expansion without need to reformat.)
   • One Hotspare drive (so that there is auto recovery on drive failure)
   • One spare drive on the shelf
   • One drive bay kept empty
   • UPS for power protection and safe shutdowns
     
     The rationale for the above configuration is to optimize the balance of both cost effectiveness and protection. It offers resilience, offering protection of up to 1 drive failure, and does auto recovery without user intervention. Using SHR-1 also does not sacrifice potential expansion if more space is required. This gives me time to swap out the broken drive once data recovery is completed and replace it with a new drive (from shelf) to maintain 24×7 operation. i can then afford time to get the broken drive to RMA and get replaced. It also allows me to not panic if drive stocks are low and allows me to time to scout for a better deal online. The risk i take, however, is that if 2 drives is to fail at the same time, i’m screwed. To address this, i have a single 4TB External Drive that is connected via USB to the Synology. This uses Hyper Backup to create a copy of the most important folders daily. So even if the entire Synology fails, this external drive and the various edge devices can recover most media with minimal loss. Alternatively, you could go for SHR-2.
     
     Cost of Synology with drives is about SGD$2500 (?) in total with drives, but it is an up front cost. My Synology has been running for about 10 years. Electrical cost is about SGD$20/mth.
6. All media data can be retrieved (last stage of the life-cycle) with various traditional and modern methods. (a) File Services like a network drive in Windows, Mac or Linux, (b) Web browser with Synology (c) DS Photo iOS App or Web browser as offered by Synology (d) Photo Aggregator Software.
   
   These multiple access methods provides the flexibility for home users to choose how they might want to leverage the media. File Services offers great flexibility on how anyone would like to look at their own files and or folders. Media access might not be as advanced as iOS Photos, but it does ensure a basic level of access to all platforms. Synology DS Photo can also be used to provide a similar experience to iPhoto.
   
   The Photo Aggregator software is something that i wrote to provide a non-invasive way of accessing the raw media. It mirrors the exact workflow i require as i try to hunt for images related to a trip, date time, or location which i couldn’t find in other pieces of software available in the market. Perhaps i am picky. The software creates a copy of the images for cache without actually modifying the originals. It is therefore non-destructive, and you are always sure that the original of the image is intact. I can also index and provide EXIF search filters for a huge number of photos and videos (>100K in my library)and performing location related searches, date time filters etc. in an efficient manner. There are other advanced smaller features included but i wouldn’t mention them here.

Setup and Maintenance Cost

My backup strategy is a selection of choices considering convenience, maintenance time and cost.

• iCloud was used as i mainly use iOS. (~SGD$3.98/mth). Automatic backups.
• Synology was selected as it provides me the drive redundancy i require. Its an upfront cost, but as long as it runs smoothly for years (at least >10 years so far?). It has an easy to use UI and lots of add-on software for expansion and easy configuration. (~SGD$2500 one time. About $250 a year ?).
• Portable NAS drives for backup during transportation (~$300). I have also build a solution to replicate these NAS drives using a Raspberry Pi, which makes it even cheaper (~$100). Note: Cloud based solutions when travelling can incur very high roaming cost if wifi isn’t available.
• Replacement drives for the Synology is about SGD$300 every 5 years.
• Email alerts are configured on the Synology for hard drive health checks every month.
• Occasional Firmware updates as provided by Synology. i don’t have to worry about this much.
• Additional Electrical bill of hosting my Synology at about $20 a month

On the user end, iCloud, Time Machine and Photosync pretty much does all the data and system backups. It is almost automatic and only occasionally requires human intervention. TimeShift does the same as Time Machine, but my Linux system has the advantage of being able to use RAID1 for desktop drive redundancy. The Synology also allows me to extend the same backup strategy to my Proxmox cluster using its File Services features and provide enterprise like features running 24×7.

The Balance of Cost, Time and Convenience

I hope that by documenting my backup strategy, it will help someone looking to design a process and a backup strategy to plan how they might want to do the same.

This setup has given me a huge backup volume (~20TB total), speed (gigabit LAN and 1Gb WAN), convenience, automation as far as possible for my entire family. My photo library contains >100K images, and the system provides protection of more than 20 years of memories. And it is still growing everyday. While outages have occurred, i am yet to lose any critical data at this point. I hope this setup continues to serve me well into the future.

One risk i do acknowledge, is the lack off-site data redundancy other than iCloud. But this isn’t a big issue as i am sheltered from natural calamities where i live. Power is quite stable but i still have a UPS for protection. For about SGD$200 a year, the above isn’t so bad. (for SGD$48/yr a year on iCloud, and decreasing cost for the rest of the hardware and SGD$240/yr of electricity).

Final Notes: Legacy Planning

I recently added this as a final stage to my backup strategy. It is essentially a manual offload process that addresses the situation where my next of kin (NOK) will be able to retrieve and access the critical information backed up with minimal technical knowledge. In times of distress, it also shouldn’t burden them in trying to learn something just to get access to whatever data they want to get to. No doubt, the systems should technically keep running even if i were to pass, but there should be a way the NOK can extract the information and migrate them to any service that they might see fit in the future.

The solution to this is a Networked NAS drive (any one of the above mentioned) that can also double up as a USB drive when connected to a computer. When powered on without a computer connection, the Networked NAS drive is configured to automatically replicate, in neat files and folder hierarchy, the contents of the Synology NAS, and offer it for access like a regular NAS. It has a usable web interface. Once the replication is completed, the drive can be powered off and connected to a computer over USB. This will allow fast access to offload all the content into a computer for more familiar use. Just like a regular USB External drive.

My mechanism leverages rclone, pre-configured to replicate the Synology storage and a few other Cloud storages. rclone was selected as it provides a huge compatibility list of server connectivity and thus makes it easier to get it configured for this purpose. The open nature of this software also gives it hope that it has better longevity. However, as the software is configured within the drive, the compatible version is already in the device and should work regardless. There is no doubt that such a drive should be kept save in a safe and should occasionally be activated to help reduce sync time in times of need.

Through all of this, the designed backup strategy above should still be operational as it doesn’t require much hands on maintenance other than in event of hardware failure. So users can continue to use the infrastructure and offload the data only if required. This gives some time for the NOK to perhaps be familiar to handle the infrastructure… if they want to, but otherwise, the data will not be inaccessible to them for their own archival.