@siracusa the compression speed and parallelism discussion on the most recent ATP reminded me of @saagar’s https://github.com/saagarjha/unxip. Though it only focuses on decompression, it shows how much faster than Archive Utilityyou can be.

@mihaip @siracusa Ok do disclaimer I am like a month behind on my ATP backlog so I just skimmed the transcript, and also I’m not at my computer so I can’t confirm this myself yet. However I think it would probably be interesting to look at using compression_tool(1) for this

@mihaip @siracusa If you just use Finder to make a zip it uses some single-threaded code by default. I don’t actually remember if you can make a zip in parallel but I’m pretty sure even if you can libarchive or whatever Finder uses doesn’t do that

@mihaip @siracusa Apple has a proprietary format (which is documented in that man page, and also used internally for Xcode archives) that essentially makes a big tar of your files and then splits it into chunks to compress those in parallel

@mihaip @siracusa So, like, if your usecase is “I have chonker file pls make smol” and you’re ok with using a special Apple format and also probably giving up a few percent in size you should give it a try. Also if you’re a developer this is API: https://developer.apple.com/documentation/applearchive

@mihaip @siracusa (Advertising/semi-related: just because you *can* decode something in parallel doesn’t mean you’ll do it efficiently. The reason unxip exists even though it does the same work is because it schedules the work better and thus runs much faster)

@mihaip @siracusa (Where “better” is, like, “please murder my machine with 1200% CPU for 20 seconds because I want my Xcode now” rather than doing 400% for a minute. But even 400% from compression_tool is better than 100% from single-threaded zip)

@mihaip @siracusa oh ALSO because I cannot actually shut up I would like to mention (even though you might know more about this than I do) but there’s a bunch of other stuff that influences GUI operations. For example your compression will go faster if it’s in the foreground

@mihaip @siracusa Because macOS will throttle apps that are not visible and this includes their XPC services unless they opt out and I don’t think Archive Utility does. Also I seem to remember it sets like an iopolicy or something to have less of an impact on your system