Large File Support for Git
Toshi Dávila avatar
Written by Toshi Dávila
Updated over a week ago

Distributed version control systems like Git have enabled new and powerful workflows but they haven't always been practical for versioning large files. Git Large File Support (LFS) is an improved way to integrate large binary files such as audio samples, datasets, graphics, and videos into your Git workflow. 

We support Git LFS over SSH and HTTPS protocols. 

LFS replaces large files such as audio samples, videos, datasets, and graphics with text pointers inside Git, while storing the file contents on a remote server.

Git LFS uses reference pointers within small text files to point to large files stored on the Assembla servers, or on your own servers for our on-premise offering. Large files such as high resolution images and videos, audio files, and assets can be called from a remote server. This allows for Git users to bypass the size restrictions previously given for repositories. The general recommendation has been to not allow Git repositories larger than 1GB to preserve performance and decrease the chances of performance issues.

Git LFS is easy to download and configure, works on all major platforms, and is open sourced under the MIT license. Versioning video files, images and even microchip designs, teams need to be able to work collaboratively to version any type of large file efficiently and reliably.

Now game developers, graphic designers, mobile app developers, and anyone else building software requiring large files can use Assembla to version all of their assets.

How to use Git LFS

First things first, install a recent version of git such as 1.8.2+ or newer and then install git-lfs. Here is a great link to follow for various flavours of the install.

Creating a repo

git init .

Now let's create some more files

touch > foo.txt && touch bar.txt && 
git add foo.txt bar.txt

Now for this example are going to make sure  *.largefiles files are managed by the LFS side of git. We need to tell git how to track these. Tracking means that in subsequent commits, these files will now be LFS files.

We do this by setting a track pattern, using the git lfs track command.

git lfs track '*.largefiles'

This tells git-lfs to track all files matching the *.largefiles pattern. 

To see a list of all patterns currently being track by git-lfs, run git lfs track with no arguments:

Listing tracked paths
    *.largefiles (.gitattributes)

To see the list of files being track by git-lfs, run git lfs ls-files. The list is currently empty. This is because technically the file isn't an lfs object until after you add and commit it - so let's do that now.

Add .gitattributes to your git repository. git lfs track stores the tracked files patterns in .gitattributes. This way when the repo is cloned, the track files patterns are preserved. Let's add some *.largefilesfiles too!

git add .gitattributes "*.largefiles"
touch testing.largefiles && git add testing.largefiles
git commit -m "testing.largefiles"

Now git status should look like this.

On branch master
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

        new file:   .gitattributes
        new file:   bar.txt
        new file:   testing.largefiles
        new file:   foo.txt

Finally, commit the new files

git commit -m "Added files"

Now, when you run git lfs ls-files, you will see the list of tracked files

f05131d24d * testing.largefiles

Working with repos and cloning

Once git lfs is installed, to clone an lfs repo, just run a normal git clone command

git clone git@git.assembla.com:myrepo.git

You can use git push and git pull and various other commands which are optimized to get stuff faster. 

Understanding where LFS is located

When using a git server that supports lfs, the lfs url defaults upon clone/adding a remote. However, sometimes the LFS server and git server are two separate services.

To see the lfs url, run git lfs env

Credentials

Assembla supports git and https transparently so you don't have to change your workflow!

Adding git-lfs to a pre-existing repo

This is an advanced topic and involves rewriting git history. Although it is possible using git commands, it's safer to use a migrator!

Migrating existing repository data to LFS using git-lfs-migrate

  • Install Java 1.8 or later

  • Download the latest binaries from here

  • Do a mirror clone of the repository to rewrite: git clone --mirror git@github.com:bozaro/git-lfs-migrate.git 

  • Rewrite e.g. all *.mp4 video files in the repository:

    java -jar git-lfs-migrate.jar \
      -s git-lfs-migrate.git \
      -d git-lfs-migrate-converted.git \
      -g git@github.com:bozaro/git-lfs-migrate-converted.git \
      "*.mp4"
  • Push the converted repository as a new repository:

cd git-lfs-migrate-converted.git
git fsck
git push --mirror git@github.com:bozaro/git-lfs-migrate-converted.git

 


Have questions? Email us at support@assembla.com

Did this answer your question?