Problem / Outcome Summary
- This article will show you how to install Crashplan Headless on Ubuntu Linux
- (To Learn how to Install Crashplan on Ubuntu Linux Desktop (with a GUI) please click here)
- Please see the ‘Summary Overview’ tab below for a high level view of the objectives this ‘howto’ will achieve.
17 Jan 2016 – There are currently issues with the new CrashPlan client / server mix for version 4.5.2 in this configuration. In particular, the guide works, but some hours down the track it breaks. I am addressing this and will update this guide when complete.
23 Jan 2016 – Fixed. Please see comments section at bottom of article for details.
Why might I want to do this?
- To backup any data, centralised from your network or specific to your computer to the Crashplan Cloud (this saves money as you don’t need a family plan)
- To create a backup ‘target’ for other computers on your network to backup to
- Because a Crashplan headless (no GUI) style of computer performs better and is more stable
To be clear, what does Crashplan Headless do?
Simply put, Crashplan gives you good automatic options for backing up your data. These can be at a cost, or at no cost depending on which situation suits you best.
What are the current backup features of Crashplan Headless?
Crashplan works on Windows, Mac OS X and Linux. It also has unofficial Crashplan headless ports for a number of NAS devices including QNAP, Synology and Netgear’s ReadyNAS.
Other notable features of Crashplan include a truly unlimited backup for a very low cost and that it retains any and all deleted versions of your files forever.
Something particularly important in this day and age, is Crashplan’s ability to support client side encryption. Client side encryption means that Crashplan has an extra level of security than most online backup systems do today.
With client side encryption, Code42 (the Crashplan company) do not hold your decryption key themselves, you do. Client side encryption makes it nearly impossible for your data to end up being used for purposes other than it was intended because only you know the encryption key. I say ‘nearly impossible’, because there’s always a chance with technology that someone such as with the now famous NSA / FBI and Apple encryption battle. However with this feature the data is most definitely safer from e.g. identity theft than it will be on your own hard drive at home. Be warned however, if you use client side encryption and then lose your key, your backup is useless and you would be unable to retrieve your Crashplan data).
For me, client side encryption is a mandatory requirement – I do not put personal data on the internet such as e.g. Inland Revenue Department or Social Security numbers, banking numbers etc. Whether you realise it or not, these will likely be on your hard drive somewhere, which means you’re probably backing them up to the internet.
A full comparison list of features for Crashplan can be found here.
There are three tier’s of Plans available with Crashplan. Crashplan Home, Crashplan Business and Crashplan Enterprise.
Crashplan Home Options:
- Free: Backup to local assets such as hard drives and computers owned by you
- Free: Backup to remote assets such as friends computers, or computers you own that are connected to the internet
- Not Free: Individual / Family Plans listed here
Crashplan Business / Crashplan Enterprise
Some free options do exist in the Business / Enterprise plans too, specifically a peer to peer option in the enterprise product.
Remember, for a full comparison of all the options, see the Crashplan comparison here.
Please note: Storing your personal information on the internet is highly risky. Globally, the new generation of internet users have become complacent with persona data such as that submitted to social sharing sites so far with with little consequence. However the impacts of this are still being worked out through legal and moral systems. If you store any data that could be considered sensitive to you, or misinterpreted by others, it is strongly recommended to use a client side encryption key.
Do you REALLY want to pay someone else to backup your data?
As you have seen above, there ARE free options available, within the Crashplan application. These will work if you set up any kind of Crashplan software including Crashplan headless on Ubuntu. It is very rare to get such a full featured and reliable application as Crashplan is, especially one that is so simple to use and includes very usable free options.
Tip for OS X customers: Mac OS X customers in particular should note, that Crashplan removes the limitations of the Mac built in backup. The shipping version of OS X is limited to Apples Time Machine backup only. Strictly speaking, Time machine is designed only to do a full system backup, or a full user backup. Crashplan offers a much higher quality of backup (and a free alternative) that allows you to select a smaller subset of backup files (such as your documents folder) and enables you to back it up elsewhere on your network according to the free options above.
The three most likely scenarios requiring recovery of your data are:
- Accidental deletion of a file or folder
- Loss of data due to theft or fire
- Hard Drive Crash
All of these scenarios are covered by CrashPlan free, provided that that scenario 2 is hosted in a remote location that would not be affected by the same fire or theft. My suggestion is to make a mutually beneficial deal with a friend to use each others spare hard drive space (or buy each other a hard drive to use), then share that in Crashplan so that you have off site backups. This would actually get you 99% safety. The other 1% is in the unusual but possible case that your friends computer hard drive dies and e.g. your computer is stolen at the same time. Not common, but not unheard of. To mitigate this scenario, you could of course add a third friend to backup to.
So if I can backup for free, why would I pay to backup to the Crashplan cloud?
There are a few reasons for this:
CrashPlan only costs you $5 per month for what is essentially a guaranteed recovery. However there are some other items that are also worth noting.
- Restoration speed – Internet connections in countries like New Zealand and Australia, do not commonly have fast upload speeds. If you want to restore your data from a friends computer en-masse, it will likely be slower than using Crashplan due to their limited upload (typically about 1Mb/s in these countries). Make sure you check your friends upload speed. Of course this won’t matter for just a few files, only for many files.
- Enterprise data security – Crashplan utilise data centers that maintain your data in monitored conditions to protect against various threats such as overheating, security hacks and viruses such as Cryptolocker. If your data is stolen from your friends house, you would want to be double sure you have client side encryption before they start trolling through your personal photo’s and bank account numbers.
- Availability – This is potentially the worst one. Most people switch their computers off at night and this may happen at the time you turn yours on. This can result in backups not being completed (ever), especially considering how long it can take with the poor upload speed some customers have and the quantity/size of backup data many of us now have on our computers. Make sure you check the progress indicator to see how your backup is tracking over several days.
- Unlimited backups – Your friends hard drive space is eventually going to run out, (and so is yours if they are backing up to your computer too). Crashplan really does offer a truly unlimited backup.
What is the performance like?
The performance will entirely depend on your scenario and also on your client as I found out here. Certain types of files can be compressed which means that they will upload faster. If you’re uploading to a friends house, they may have a slow connection, or you may have a slow upload capability. ADSL customers in NZ and Australia will only have about a 1Mb/s upload speed which would take months to upload a fairly typical 1TB of disk space. VDSL or fibre customers on the other hand may take days or even weeks for the same data. One option in Crashplan that does make a difference is the de-duplication option in the advanced settings. Set this to minimum for more speed if you don’t have a powerful processor (especially if you run client side encryption), although the environmentalist in me says you should leave this alone for reasons I won’t explain here (it uses less hard disk by doing some clever stuff). That said, make sure you test it out yourself in both scenarios as de-duplication means you don’t have to upload similar data in different files and this could also be of benefit with a slow connection if you have a lot of duplication.
- Ubuntu Linux
- An internet modem / router capable of performing NAT
- SSH Access
- A recent web browser
- A working internet connection
High Level Summary Steps
The below lists the high level summary of steps we’re about to take during this howto.
- Download and extract the Crashplan Application
- Install and configure the Crashplan Application
- Connect to your Crashplan Headless on Ubuntu Service
Download and extract the Crashplan Application
- wget https://download2.code42.com/installs/linux/install/CrashPlan/CrashPlan_4.5.2_Linux.tgz
- sudo apt-get update
- tar zxvf
- tar zxvf CrashPlan_4.3.0_Linux.tgz
Install and configure the Crashplan application
- become root
- cd Crashplan-install/
(If you want to uninstall a previous installation, just type ./uninstall -i /usr/local/crashplan)
Welcome to the Crashplan Installer.
Press enter to continue with installation.
- Press Enter
detected root permissions
You must review and agree to the EULA before installation.
Press enter to read the EULA. (Space scrolls quicker)
- Press Enter
Do you accept and agree to be bound by the EULA? (yes/no)
- Type ‘Yes’ and press <Enter>
Press <Enter> on all the defaults below, taking particular note of number 4 as this is where ‘friends’ data is stored (or data from your other computers doing local backups. You may like to make it somewhere that has plenty of free space.
- What parent directory do you wish to install CrashPlan into? [/usr/local]
- /usr/local/crashplan does not exist. Create /usr/local/crashplan? (y/n) [y]
- What directory do you wish to link the CrashPlan executable to? [/usr/local/bin]
- What directory do you wish to store backups in? [/usr/local/var/crashplan] /mnt/backups
- /mnt/backups does not exist. Create /mnt/backups? (y/n) [y] y
- What directory contains your SYSV init scripts? [/etc/init.d]
- What directory contains your runlevel init links? [/etc/rc2.d]
The installer then confirms your selections in a list, press y if correct, n if not so that you can do them again.
Crashplan will install to: /usr/local/crashplan
And put links to binaries in: /usr/local/bin
And store datas in: /mnt/backups
Your init.d dir is: /etc/init.d
Your current runlevel directory is: /etc/rc2.d
Is this correct? (y/n) [y] y
The Java Runtime Environment is then downloaded from the internet and uses the locations you have just specified. Wait for this to complete.
downloading the JRE using wget
–2015-08-02 16:24:48– http://download.code42.com/installs/proserver/jre/jre-7u45-linux-x64.tgz
Resolving download.code42.com (download.code42.com)… 184.108.40.206
Connecting to download.code42.com (download.code42.com)|220.127.116.11|:80… connected.
HTTP request sent, awaiting response… 301 Moved Permanently
Location: https://download2.code42.com/installs/proserver/jre/jre-7u45-linux-x64.tgz [following]
–2015-08-02 16:24:48– https://download2.code42.com/installs/proserver/jre/jre-7u45-linux-x64.tgz
Resolving download2.code42.com (download2.code42.com)… 18.104.22.168
Connecting to download2.code42.com (download2.code42.com)|22.214.171.124|:443… connected.
HTTP request sent, awaiting response… 200 OK
Length: 46843538 (45M) [application/octet-stream]
Saving to: ‘jre-7u45-linux-x64.tgz’
100%[======================================>] 46,843,538 4.16MB/s in 12s
2015-08-02 16:25:01 (3.66 MB/s) – ‘jre-7u45-linux-x64.tgz’ saved [46843538/46843538]
Unpacking /home/username/CrashPlan-install/./CrashPlan_4.3.0.cpi …
Also note the number of files that linux will watch. You will likely have more files than this so you will need to adjust it.
Your Linux system is currently configured to watch 8192 files in real time.
We recommend using a larger value; see the CrashPlan support site for details
Starting CrashPlan Engine … Using standard startup
CrashPlan has been installed and the Service has been started automatically.
If you want to check that CrashPlan is running on your Ubuntu Headless computer, simply enter:
- cd /usr/local/crashplan/bin (lately changed to /usr/local/bin
- ./CrashPlanEngine –status
You should see a message saying the CrashPlan engine is running
Note as above if you have a lot of files, you’ll need to update the Java heap size – see CrashPlan’s article on this here.
Also, to update the number of files your linux machine can watch enter the below:
echo 1000000 > /proc/sys/fs/file-max (assuming you have 1,000,000 files to watch in your backup (I do).
Connect to your Crashplan Headless on Ubuntu service
When you install a Crashplan headless system, you’ll want to connect the Crashplan GUI to it, from another computer. This will allow you to configure your Crashplan backup folders (and other things) easily. When you’re finished, the changes are saved to the headless computer.
Make sure you have installed the Crashplan application to your local client computer from the Crashplan downloads page. A guide for doing this in an Ubuntu GUI can be found here.
There is varying information about what you must and must not do to set this up. Some information suggests you do not need to copy the ‘ui.info’ file, some says you do, some says you need an SSH port redirect, some doesn’t mention it at all. The truth is I’ve had mixed results. Often nothing works permanently. Until recently, I did not need to create an SSH tunnel, then I did and now that’s stopped working again. This is the way it’s likely to stay while Crashplan oddly insists it’s unsupported but details how to do it at the same time. Occasionally the ui_info file gets auto detected making people believe it is not needed, but in most cases you will need to do it. As far as SSH port redirects, try it without, and if you can’t get it to work, try it with the tunneling.
Backup and copy the .ui_info file from the Ubuntu headless computer to the appropriate directory on the local computer (if you’re using mac, you can use the following commands – note be careful with the spaces, if in doubt you can put ” around it). For windows, use PuTTY, for Linux just use same as Mac method.
On your Mac, backup your .ui_info file
- mv /Library/Application Support/CrashPlan/.ui_info /Library/Application Support/CrashPlan/.ui_info_original
Copy the Headless .ui_info file to your mac
- scp username@yourheadlessIP:/var/lib/crashplan/.ui_info /Users/yourmacusername
- sudo cp /Users/yourmacusername/.ui_info /Library/Application Support/CrashPlan/
(Current .ui_info file locations can be found on the Code42 support page here).
Point your mac Crashplan application to the headless server
- vi /Applications/CrashPlan.app/Contents/Resources/Java/conf/ui.properties
- Change the #serviceHost line to the ip address of your headless server, remove the ‘#’ and save the file
If you need ssh tunneling
On a Mac
- sudo vi /Library/Application Support/CrashPlan/.ui_info
- change the line that reads #servicePort from 4243 to 4200 and remove the # to uncomment the line
- ssh -L 4200:localhost:4243 user@yourheadlessipaddress
Start the local Crashplan app and this should now be working.
- To date, I have found no reason to change other settings such as port settings in the ui.properties file. I’m sure some people will have their reasons.
- You can always change the IP address back should you wish to manage the CrashPlan service on your local computer.
So there you have it, a fully functioning Crashplan backup that allows centrally hosted files to be backed up to Crashplan. This saves money because you don’t need to employ a more expensive licence provided you all save files to a single location on your home Ubuntu Server or NAS.
**Looking for creating a manual backup process? Why not read our article on how to create a manual backup process and why you need one, here.
So for any of you wondering, this is what’s happening. Another infamous CrashPlan upgrade causing yet more failures. Great product, terrible change control on their headless architecture albeit unsupported – yes CrashPlan, that’s certainly clear. Reinstalling the CrashPlan server does get this running again however several hours later, it stops. I have just had another look at it and found that the /var/lib/crashplan/.ui_info file (server side) is getting overwritten with an IP address that does not equal the server OR the client. For some reason it’s adding the IP address of the NAS that is attached to my /mnt/shares directory. This NAS previously had the CrashPlan app installed on it (it’s a QNAP), but the NAS was rebuilt, the CrashPlan app was never installed on it and the Ubuntu system was used instead. Furthermore this was working well for some time until 4.5.2 came out. Changing the IP address manually to 0.0.0.0 sticks only until I try to start the crashplan service: /etc/init.d/crashplan start and then the file is overwritten and the IP address comes back again. So the question I have is where is it getting this IP address from?
Editing the file /usr/local/crashplan/conf/my.service.xml and replacing the incorrect IP address in the service host line with 0.0.0.0 seems to have gotten past the continuing stopped service (it was rightly complaining that it couldn’t bind to it, obviously it can’t as it doesn’t own that address). However, it now continually restarts (which according to the logs may be an out of memory error). Unfortunately (and unbelievably) connecting to the service via the ssh method that I was using does not work because CrashPlan is changing the servicePort line in this file back to 4243 every time I changer it, again with the start of the crashplan service it is overwritten, even though the serviceHost field is not overwritten. Even adding the host address (the alternative method) to the mac .ui_info file instead of using the ssh method is not yet working. I need the GUI front end to address the Java memory issue.
Right, back on this today – so, the .ui_info file on my mac again has changed to something other than what is on the Headless box. OR the hedless box changed and I didn’t notice / copy it properly. Anyway, fixing that I can now connect again, albeit that it get kicked out quite regularly due to the service restarting. Now that I can connect though, in between web sites I have raised the Java allocated memory from 1GB to 2GB since I have a lot of files. A good option is to go to /usr/local/crashplan/log and run a tail -f on the service.log.0. So far so good. So I’m not sure why this broke so bad in my case except that if adding more memory fixes it, then essentially the new version uses more or somehow perhaps a lot more files were added which caused it. I can’t think of anything I added, but perhaps it’s that.