---

Download a huge file using split and cat
2011-04-14

I’ve got a decent internet connection at home sitting around 15 Mbps down. Not top of the line, but respectable nonetheless.

I have a personal server that I use to store various files that generally range in size from 150 MB to 14 GB. Now if we lived in a perfect world the download would take up as much bandwidth as it can which would have me downloading at around 1.8 MB/s. Unfortunately through forces that I understand to be outside my control I only get about 200 Kb/s.

Interestingly, I can download several files from the same sever concurrently at that speed. This is great when downloading several smaller files at the same time, but rather useless when you’ve got one big one. It doesn’t seem fair.

This is where split and cat come in! These are two commands you can use via command line (though there are many GUI alternatives, but that’s lame). Split, of course, will take in a file and split it into smaller pieces. Cat (short for concatenate) will take all those pieces and reassemble them on the other side. Don’t be afraid, it’s super easy.

First, on the server, split your large file into a bunch of lil’ pieces.

This will split ‘totally.legit.file.mkv’ into 500 Mb pieces. The last piece will be less than 500 Mb. Your split up files will be named ‘my_split_file_aa’, ‘my_split_file_ab’, ‘my_split_file_ac’ and so on.

Download these files to your local computer.

Now we’ll concatenate the files back together!

Depending on how big the file is this could take a little while. When it’s done the split files will still be there and you’ll have a shiny new ‘totally.legit.file.mkv’ ready to go!

Don’t forget to delete the split file pieces that are still laying around on your server and local computers.

Share
Tags: cli, command line, shell, ssh, terminal

trackback URL for this post: http://lab.joelgillman.com/archives/179_download-a-huge-file-using-split-and-ca/trackback