Disc image compression

Systems from the 5th and 6th console generation often use optical discs like CD or DVD to store game data. Disc image sizes range from 700 MB (CD), 1.4 GB (GC Mini-DVD), 4.7 GB (single-layered DVD) and even 25 GB (Blu-Ray) for newer systems. Often times only a fraction of the actual disc size is used for game data and the rest is filled with dummy data to be optimized for read access by the optical drive laser.

On the other side systems like the Sony PlayStation often use multi-track images (data and audio tracks) containing several *.bin and *.cue files which can become cumbersome in daily usage with Batocera or other emulation frontends.

Choosing the right disc image compression can help in both cases and should be the preferred format for most of the disc based systems. This page will help you to find the suitable compression method and toolset.

CHD is short for (C)ompressed (H)unks of (D)ata. It is a lossless compression format originally developed for MAME, for the hard-drive contents of certain arcade machines. It has since been used in several other emulators as a means of storing disc based game data. It compresses the contents of a disc image to a handy single .chd file. Here's a comparison between 7z file compression and CHD (originally an 700 MB image):

CHD is supported by many systems and the recommended format for the following systems:

• 3do
• dreamcast
• segacd
• neogeocd
• pcenginecd
• psx
• ps2 (available with Batocera v31)
• saturn

CHD files can be created using the chdman program, developed by the MAME project. Its installation process varies based on which operating system is being used to create the files.

The official MAME package contains a copy of the executable file. You can download the EXE file and open it up like an archive with 7-zip, then extract chdman.exe.

On most Linux based distributions, this is included with the mame-tools package.

On macOS use brew to install the rom-tools package.

chdman is a commmand line application. It can create a .chd file by targeting *.cue, *.iso or *.gdi files.

If there is a *.cue file available, use this for creating the .chd. Otherwise choose the *.iso (Sony PlayStation 2 DVDs) or *.gdi (Dreamcast GD-ROM) file. Note that GameCube/Wii have their own compressed format.

Open the command line in either Windows with cmd.exe or in Linux with your favorite terminal emulator. Put both chdman and the game file in the same folder, navigate to that folder in your terminal and run:

chdman createcd -i <game.cue> -o <game.chd>

For example, if the file structure was like so:

/totally-rad-folder-name/
 ├─ chdman
 ├─ Spiderman.cue
 └─ Spiderman.bin

Switch to the folder first with cd /totally-rad-folder-name then run:

chdman createcd -i Spiderman.cue -o Spiderman.chd

Then make yourself a cup of tea. This can take a while. After that you'll have <game.chd> and you can delete the *.cue/*.bin, *.iso or *.gdi/*.bin file(s).

The compression is fully lossless so it is possible to redo the compression and get the original files back:

chdman extractcd -i <game.chd> -o <game.cue>

You can batch convert all files in a directory like following (replace .cue with the correct file type you want to convert):

Linux/macOS:

Navigate to the folder that has all your game ROMs and run the following for loop to convert all the files in one go:

for i in *.cue; do chdman createcd -i "${i}" -o "${i%.*}.chd"; done

replacing *.cue as needed. If you're converting a lot of files, practice some yoga poses. This may take a while.

Here's a command to convert all the games in the current folder and recursive folders to CHD:

For *.cue:

find . -name "*cue" -exec chdman createcd -i {} -o {}.chd \;

For *.iso:

find . -name "*iso" -exec chdman createcd -i {} -o {}.chd \;

For *.gdi:

find . -name "*gdi" -exec chdman createcd -i {} -o {}.chd \;

Windows:

Just place the chdman.exe into the directory where you have your disc images.

Here's some handy BAT scripts you can just double-click to instantly convert all the games in the current folder to CHD:

For *.cue:

cue-to-chd.bat

for /r %%i in (*.cue) do chdman createcd -i "%%i" -o "%%~ni.chd"

For *.iso:

iso-to-chd.bat

for /r %%i in (*.iso) do chdman createcd -i "%%i" -o "%%~ni.chd"

For *.gdi:

gdi-to-chd.bat

for /r %%i in (*.gdi) do chdman createcd -i "%%i" -o "%%~ni.chd"

Or combine both: for /r i in (*.cue, *.gdi, *.iso) do chdman createcd -i "i“ -o ”%%~ni.chd“

If you're converting a lot of files, play some tunes on your guitar. This may take a while.

CSO is a compression method for the ISO image format and is the short form for Compressed ISO. It also called CISO.

It was originally used to compress PlayStation Portable UMD disc images.

CSO/CISO is supported by multiple systems and the recommended format for the following systems:

• ps2 (prior to Batocera v31)
• psp

There are different tools to create CSO files. maxcso is a free command line application that is available to Linux, macOS and Windows.

To compress an ISO file just use:

maxcso <game.iso>

This will create a <game.cso> file in the same folder.

The compression is fully lossless so it is possible to redo the compression and get the original files back:

maxcso --decompress <game.cso>

Batch converting can be done with a simple for loop.

Linux/macOS:

for i in *.iso; do maxcso "${i}"; done

Windows:

for %i in (*.iso) do maxcso.exe "%i"

ciso is a command line utility for converting PSP iso files to cso. It is notable for being more compatible with a real PSP than maxcso.

Compress an ISO to CSO:

ciso <level> game.iso game.cso

Where <level> is the compression level, from 1 (fast, poor compression) to 9 (slow, high compression).

To decompress a CSO to ISO:

ciso 0 game.cso game.iso

Here, the “level” is 0, that is “no compression”.

Batch conveting can be done with a simple for loop.

Linux/MacOS:

for i in *.iso; do ciso 9 "${i}" "${i}".ciso; done

Windows:

for %i in (*.iso) do ciso.exe 9 "%i" "%i".ciso

The Dolphin team developed a new compression format based on WIA called RVZ. Unlike all the previous formats, RVZ is lossless and can preserve the padding data on Wii discs as well as the necessary files needed by the Wii's IOS. Usage is very similar to GCZ in Dolphin itself, but it only works on newer Dolphin versions.

RVZ is only supported by the Dolphin emulator and therefore the recommended format for the following systems:

• gamecube
• wii

You need the Dolphin emulator (minimum version 5.0-12188+) to compress ISO files. At the moment the compression has to be done via dolphin-config in the Applications folder of the [F1] file manager.

Find the game in Dolphin's game list, right click and select Convert File…

The default settings should be fine.

This process is completely reversible.

Just multi-select with pressing the CTRL key to choose multiple games, right-click and select Convert File…

SquashFS allows you to losslessly compress entire folders and all their contained items. This image can be extracted to be identical to the files that were compressed. Unlike ZIP/7Z/GZ files, SquashFS allows Batocera to present the contents of the compressed image to the system as if though it weren't compressed at all by automatically mounting them upon launch. This has been a latent feature but full integration with the tools necessary to handle SquashFS images was introduced in Batocera v33.

There are some limitations to keep in mind:

• SquashFS is read-only. Obviously, this causes issues with games that attempt to write to the directory it has been installed to, which is typical for many PC games to do. Some are exceptions, such as basic DOS titles. For Windows, this can be worked around. ( is this only for Windows?)
• Only systems that specify in their _info.txt as supporting squashfs will be able to launch SquashFS images.

SquashFS is particularly good for PS3 and Wii U games, which are stored as folders which are not written to by the system.

Technical details

First, to be listed in EmulationStation, SquashFS images must be added technically by a developer to the list of eligible system. (aka, dos and ps3 are added for the moment, but it is very easy to add any system while it supports folder as rom). Squashfs images are mounted under /var/run/squashfs/<rom> and this is this directory that is given to the emulator.

This is currently only possible via SSH. Batocera utilizes mksquashfs to create SquashFS images. Syntax: mksquashfs <game> <game>.squashfs.

For example:

cd /userdata/roms/dos
mksquashfs skweek.pc skweek.pc.squashfs

Or to compress a single file :

cd /userdata/roms/xbox
mksquashfs crazytaxi3.iso crazytaxi3.iso.squashfs

Or to compress a single file with the xz compression option:

mksquashfs crazytaxi3.iso crazytaxi3.iso.squashfs -comp xz

Or to compress multiple files :

for i in *; do if [ -d "$i" ]; then mksquashfs "$i" "$i.squashfs"; fi; done
or
find . -maxdepth 1 -name "[^.]*" -type d -exec mksquashfs {} {}.squashfs \;

Or to compress multiple files with the xz compression option:

for i in *; do if [ -d "$i" ]; then mksquashfs "$i" "$i.squashfs" -comp xz; fi; done
or
find . -maxdepth 1 -name "[^.]*" -type d -exec mksquashfs {} {}.squashfs -comp xz \;

After confirming that the game still launches and runs correctly, the old folder can be safely removed.

If you use the btrfs filesystem, you can enable compression, and you'll get the same compression result without any action.

Connect via samba and decompress files using a compatilble decompress tool (use the smart uncompress to create a new folder): 7-zip, Picodrive

It can also be uncompressed using the unsquashfs command. Syntax: unsquashfs -f -d <destination> <folder/game>.squashfs

For example:

unsquashfs -f -d /userdata/roms/<system>/<gamedirectory> /userdata/roms/<system>/<game>.squashfs

A successful decompression would look something like this:

batocera$ cd /userdata/roms/wiiu/
batocera$ unsquashfs -f -d ExtractionFolder SuperMario.squashfs
Parallel unsquashfs: Using 4 processors
23 inodes (2649 blocks) to write

[=============================================================\] 2672/2672 100%

created 23 files
created 4 directories
created 0 symlinks
created 0 devices
created 0 fifos
created 0 sockets
created 0 hardlinks

It is possible to manually mount a squashFS. This could be used to simply access the data and copy it to somewhere else.

mount -t squashfs path/to/file.squashfs /mnt

WUA is a format used by the Wii U emulator Cemu. It is a single compressed file that contains the base game, its updates and installed DLC content.

Converting to WUA must be done via Cemu's interface. In Batocera, this can be found in the [F1] file manager in Applications.

This is automatically done if Cemu has been launched at least once from ES. However if this is the first time Cemu has been opened, ensure that the paths are pointing to the correct locations:

• MLC Path: /userdata/saves/wiiu
• Game Paths: /userdata/roms/wiiu

Cemu will automatically include any updates and DLCs that are installed when creating the WUA file. Ensure they are installed first before doing this.

Now, go to title manager, to convert them to .wua:

Navigate to Tools → Title Manager → ( wtf is this?) Left button in the BASE game item with folder format → Convert to Compressed Wii U archive (.wua), then set the destination folder as /userdata/roms/wiiu.

Here is the last chance to ensure it's detecting the update and DLC files. The base game, update and DLC must be the same region to be detected.

Hit OK.