Spreading a Data Project over Multiple Discs in Linux

After spending hours trying to find software in Linux that can span a large directory over multiple discs without having to create an archive set, I finally came across this small Python script that will do the job.

How to Use

Put the script(s) in your “~/bin/”.

mkdir work/
cd work
mkdir in/
mkdir out/

Make a link copy of your data in the “in” dir:
cp -vrl originals/* in/
xsplit_dvd.py

The dir work/out/ should contain the output dirs dvd_001, dvd_002, etc…

Notes:

  • actually, the script creates out/ itself, if you don’t.
  • Real Soon Now[R], I’ll change the scripts, so they make the links
    themselves from the original data.
  • several xsplit_… scripts exist, with hardcoded volumes. You can change
    the volume yourself. In the future, the volume could be an argument.
  • if a file is found that is larger than the block volume (a file bigger
    than 4.2 GB in you case, or a file bigger than 700 MB with xsplit_cd),
    that file is left in “in/” dir.
  • The previous point means that “in/” should be empty if everything went
    well.

Reference

https://forums.opensuse.org/showthread.php/403148-K3b-Spreading-a-Data-Project-over-Multiple-Discs