First, Happy July 4th! I was going to take today off from everything, but woke up early with a bug to do some more OS X tagging research. I’ll go back to bed after this…

Yesterday I offered a warning about GNU Coreutils and the stripping of extended attributes (like OS X tags). This is a collection of results from some more extensive testing. It’s a work in progress and will likely become its own resource after it’s fleshed out a bit.

I’m also writing a book on tagging right now, so this resource is a valuable part of that. It would be great if anyone who tests some additional scenarios (and offer possible solutions) contact me and share the results. You’ll make it into the credits!

*nix tools

First, the default system utilities in Darwin (such as mv and cp), by and large, have no problem preserving the extended attributes.

When transferring files to other systems (or even mirroring locally), though, the common utilities rsync and scp do not preserve extended attributes by default. Both utilities have a -E switch, though, which does a fine job keeping all attributes (along with probably-unnecessary resource forks).

I personally have aliased both of these tools to always include the -E switch1:

alias rsync="rsync -E"
alias scp="scp -E"

Update: rsync 3.0 (installable with homebrew) does an even better job of preserving tags by adding the -X option. Thanks to Emory L for pointing this out.

As mentioned yesterday, the GNU Coreutils versions of some commands (e.g. mv and cp) will strip your tags. I’d recommend just deleting the symlinks for mv and cp in /usr/local/opt/coreutils/libexec/gnubin (where brew installs them).

Git strips tags when transferring to a repository, and I haven’t found any way around that other than compressing tagged files into a zipball or tarball before transfer, which is only useful for files that don’t need individual version control. I haven’t been using tags in any of my coding projects anyway, so the times that this matters to me are minimal.

Zip and tar both preserve tags in archived files.

Ruby FileUtils

According to one tester (and my own confirmation), the default FileUtils class in Ruby will preserve tags with FileUtils.mv, but strip them with FileUtils.cp. If you have the GNU Coreutils mv and cp installed, FileUtils.mv strips tags as well.

Update: There is an option for FileUtils.cp that is supposed to preserve tags:

FileUtils.cp "from.txt", "to.txt", :preserve => true

Thanks to Juha Ranta for that one. I tested with Ruby 2.1 and it hasn’t worked yet, but I need to do some further research.

Other scripting languages

I haven’t tested with other languages that are commonly used in scripts that would affect my files yet. I’m especially curious about Python, so if anyone beats me to testing it out, I’d appreciate the update.

Backup

Good:

  • Dropbox does preserve tags during sync (as of an update quite some time ago)
  • Time Machine maintains them (obviously).
  • Arq preserves tags
  • Bittorrent Sync preserves tags
  • CrashPlan preserves tags. They can be lost if you download a restore file through the web interface, but are maintained during restores from the app. (Thanks to Tom for checking, and see his comment for more info and tricks)

Backblaze does not back up extended attributes, so tags are gone when you restore. (Thanks to JustinDaveyB for testing.) I’m unsure about other backup systems (DollyDrive, etc.). If you’re using something else, do a restore on a few files that you know were tagged at the time of backup, then check the restored files for the tags. If you could let me know either way, I’ll update this post with the results and include them in other resources.

I’ve been testing Transporter out, and it looks like Transporter Dekstop does not preserve tags. I’ll be in touch with the developers of the Transporter Desktop app after the holiday weekend to confirm this and request a fix. (The Transporter team confirms they have a fix slated for release this summer.) I’m loving the personal cloud that Transporter provides, but losing tags would be an issue.

Email and file sharing

iCloud preserves tags. We’ll be seeing tags on iOS soon enough, and it’s vital that Apple keeps this working. I’m not worried about it.

As mentioned above, Dropbox is good with tags, unless you share files through the website, as mentioned below.

You’ll lose tags on most files attached to emails. I think this is generally a good thing, as I don’t need tags emailed to me by other people polluting my carefully-curated taxonomy. If you want to preserve tags within a group tagging environment, just zip your files or share a Dropbox folder. Sharing a Dropbox link downloaded through the browser will remove tags.

Uploading a file to services like Droplr or Cloudapp — or sharing through the Dropbox web interface — will also strip your tags, but again this is of little concern to me on the receiving end. Anything downloaded through a browser, as far as I can tell, loses extended attributes in the process.

Summary

Tags are generally safe within the OS X ecosystem, from Darwin tools to iCloud sync, but third-party tools can wreak havoc on your carefully-crafted tagging system. As a proponent of tagging, losing tags inadvertently is a fear (and one shared by any tagger). I’d like to keep this resource growing and would appreciate your input.

  1. My rsync alias is actually rsync -avzE, but if you’re using rsync, I’ll assume you already have your own preferred defaults. ↩