Toggle menu
Toggle personal menu
Not logged in
Your IP address will be publicly visible if you make any edits.

GCM (File format): Difference between revisions

No edit summary
 
(24 intermediate revisions by 2 users not shown)
Line 1: Line 1:
[[Gotcha Force | ← Gotcha Force]]
[[Gotcha Force | ← Gotcha Force]]


The GameCube mini DVD use GCM file format (Gamecube Disc Image with extension gcm or iso) and can be [https://discourse.world/h/2017/07/24/Gamecube-file-system-device in 6 blocks] :
''This article is about the GCM file format.''
* boot.bin (Base Informations - Disk Header)
* bi2.bin (Additionnal informations - debug, locales - Disk Header Information)
* apploader.img (Loader of the main executable)
* fst.bin (FileStringTable)
* boot.dol / elf (Main executable)
* Files and Folders tree


The file system is handled by GameCube itself, and we can see that files are loaded by names no matter in which order the have been stored (Dolphin Emulator).
{{Research | 1| The structure of this file is well known. }}


The dol file and FST are stored after de apploader.img dol after FST or FST after dol. Most of the time dol is before FST. All GCM ROMs have a mini DVD fixed length : 1 459 978 240 bytes. Because of that there is often empty spaces between files in the game released containing junk datas or old values partially overide by new ones during devs. This empty spaces contains useless information of the game. It could be used for mini DVD cyphering hardening . The question could be "does this junks data are the sames in all released mini DVD of the game?" An intersting thing is that burning data at the out border of the DVD increases performances when hardware DVD read head access to it.
{| class="wikitable"
|+ The GameCube mini DVD uses GCM file format (Gamecube Disc Image with extension gcm or iso) and can be divided into 6 blocks:
! Block Name !! Description
|-
| boot.bin || Base Information - Disc Header
|-
| bi2.bin || Additional Information - Debug, locales - Disc Header Information
|-
| apploader.img || Main Executable Loader
|-
| fst.bin || File String Table
|-
| boot.dol / elf || Main Executable
|-
| Files and Folders Tree || User Space Files and Folders Tree
|}
 
The file system is handled by the GameCube and we can see that files are loaded by names no matter in which order they have been stored (Dolphin Emulator).
 
The dol file and FST are stored after de apploader.img. The dol can be after or before FST. Most of the time dol is before FST. All GCM ROMs have a fixed length (mini DVD length): 1 459 978 240 bytes. Because of that there is often empty spaces between files in the game released containing junk datas or old values partially overridden by new ones during devs. This empty spaces contains unused information. It could be used for mini DVD cyphering hardening. The question could be "does this junks data are the sames in all released mini DVD of the game?" An interesting thing is that burning data at the out border of the DVD increases performances when hardware DVD read head access to it.


== GCM/ISO format Architecture ==
== GCM/ISO format Architecture ==
=== boot.bin ===
=== boot.bin ===
boot.bin file is at beginning of the mini DVD and its length is 0x440 bytes. We found on it:
{| class="wikitable"
* 0 - 4 bytes ascii - GameCode - A-Za-z0-9
|+ boot.bin file is at beginning of the mini DVD and its length is 0x440 bytes. We found on it:
* 4 - 2 bytes ascii - MakerCode - A-Za-z0-9
! Offset (hex) !! Size (bytes) !! Description
* 6 - 1 bytes - DiskNumber - beginning at 0 - 0-98
|-
* 7 - 1 bytes - Version - 0-99
| 0x00 || 4 || GameCode (ASCII) - A-Za-z0-9
* 8 - 1 bytes - AudioStreaming - flag (0 ou 1)
|-
* 9 - 1 bytes - StreamBufferSize - streaming buffers count - 0-15
| 0x04 || 2 || MakerCode (ASCII) - A-Za-z0-9
* 0x1c - 4 bytes - DVDMagic - 0xc2339f3d
|-
* 0x20 - 0x40 bytes - GameName can containc kanjis
| 0x06 || 1 || DiscNumber (0-98)
* 0x420 - 4 bytes - DolOffset
|-
* 0x424 - 4 bytes - FstOffset
| 0x07 || 1 || Version (0-99)
* 0x428 - 4 bytes - FstLen
|-
* 0x42c - 4 bytes - FstMaxLen - correspond often to the FST len but it's the space reserved for others mini DVDs FST length
| 0x08 || 1 || AudioStreaming Flag (0 or 1)
* 0x430 - 4 bytes - FST memory address?
|-
* 0x434 - 4 bytes - UserPosition - correspond to the offset on mini DVD where begin first game files
| 0x09 || 1 || StreamBufferSize (0-15)
* 0x438 - 4 bytes - UserLength - correspond to the total length where we found all files (could contains empty junk spaces)
|-
| 0x1C || 4 || DVDMagic (0xc2339f3d)
|-
| 0x20 || 64 || GameName (ASCII and possibly Kanji)
|-
| 0x420 || 4 || DolOffset
|-
| 0x424 || 4 || FstOffset
|-
| 0x428 || 4 || FstLen
|-
| 0x42C || 4 || FstMaxLen (often corresponds to FST length but reserved for other mini DVDs)
|-
| 0x430 || 4 || FST Memory Address
|-
| 0x434 || 4 || UserPosition (Offset on mini DVD where the first game files begin)
|-
| 0x438 || 4 || UserLength (Total length where all files are located, may contain empty spaces)
|}


=== bi2.bin ===
=== bi2.bin ===
bi2.bin files has a length of 0x2000 bytes and reside just after boot.bin (offset 0x440 of the GCM). It contains locales informations and debug configurations.
{| class="wikitable"
* 0 - 4 bytes - DebugMonitorSize
|+ bi2.bin file has a length of 0x2000 bytes and reside just after boot.bin (offset 0x440 of the GCM). It contains locales informations and debug configurations.
* 4 - 4 bytes - SimulatedMemorySize
! Offset (hex) !! Size (bytes) !! Description
* 8 - 4 bytes - ArgumentOffset
|-
* 0xc - 4 bytes - DebugFlag - 0 pour tout debugueur sur GDEV; 3 pour CodeWarrior sur EV
| 0x00 || 4 || DebugMonitorSize
* 0x10 - 4 bytes - TrackLocation - Il faut vérifier s'il ne sagit pas de MetroTRK (dans certaines dokus)
|-
* 0x14 - 4 bytes - TrackSize - Il faut vérifier s'il ne sagit pas de MetroTRK (dans certaines dokus)
| 0x04 || 4 || SimulatedMemorySize
* 0x18 - 4 bytes - CountryCode - 0 pour Japon, 1 pour USA, 2 pour PAL, 4 pour Corée du Sud
|-
* 0x1c - 4 bytes - TotalDisk - nombre total de disk <= 99
| 0x08 || 4 || ArgumentOffset
* 0x20 - 4 bytes - LongFileNameSupport - 1 = support, 0 = contraindre au format 8.3
|-
* 0x24 - 4 bytes - PadSpec ? - controller version for PADSetSpec ?
| 0x0C || 4 || [[#Debug Flags|DebugFlag]]
* 0x28 - 4 bytes - DolLimit - (limite max de la somme des tailles des sections Text et Data)
|-
| 0x10 || 4 || TrackLocation (or MetroTRK for some documentations)
|-
| 0x14 || 4 || TrackSize (or MetroTRK for some documentations)
|-
| 0x18 || 4 || [[#Country Codes|CountryCode]]
|-
| 0x1C || 4 || TotalDisc (Total number of discs, <= 99)
|-
| 0x20 || 4 || LongFileNameSupport (1 = supported, 0 = constrained to 8.3 format)
|-
| 0x24 || 4 || PadSpec (Controller version for PADSetSpec)
|-
| 0x28 || 4 || DolLimit (Max length of the sum of data and text sections)
|}
 
{| class="wikitable"
|+ Debug Flags
! Value!! Description
|-
| 0 || Normal Boot
|-
| 1 || Devkit Set to 0; MetroTRK Set to AMC DDH hardware
|-
| 2 || Devkit Set to 1; MetroTRK Set to GDEV hardware
|-
| 3 || Devkit Set to 2; MetroTRK Set to BBA
|}
 
{| class="wikitable"
|+ Country Codes
! Value!! Pays
|-
| 0 || Japan
|-
| 1 || USA
|-
| 2 || PAL
|-
| 4 || Korea
|}


=== apploader.img ===
=== apploader.img ===
L'[https://www.gc-forever.com/wiki/index.php?title=apploader apploader] est un fichier, lancé par le BIOS de la GameCube (aussi appelé l'[https://wiki.dolphin-emu.org/index.php?title=GameCube_Main_Menu Initial Program Loader / IPL]), qui permet de créer les différents espaces mémoires pour lancer l'exécutable du jeu (généralement au format dol). Il se situe à l'offset 0x2440, à la suite du fichier bi2.bin. Il portera un nom différent selon les programmes que l'on utilisera pour ouvrir l'image DVD du jeu : AppLoader.ldr / apploader.img (dolphin Emulator) etc.
The [https://www.gc-forever.com/wiki/index.php?title=apploader apploader] is a file, loaded by the GameCube BIOS (also called the [https://wiki.dolphin-emu.org/index.php?title=GameCube_Main_Menu Initial Program Loader / IPL]), that creates virtual memory spaces and rebase the dol datas in those areas before starting it. Apploader is at offset 0x2440 just after the bi2.bin. The GCM/iso doesn't store any name of it so it's name depend to the software used to extract it: AppLoader.ldr / apploader.img (dolphin Emulator) and so on.


La taille de l'apploader se calcul à partir de son header :
The apploader length is calculated with its header information:
* Le header fait 0x20 octets :
* Header has a length of 0x20 bytes:
** 0 - 10 octets - version
** 0 - 10 bytes - Version
** 0x10 - 4 octets - entry_point
** 0x10 - 4 bytes - entry_point
** 0x14 - 4 octets - size
** 0x14 - 4 bytes - size
** 0x18 - 4 octets - trailer_size
** 0x18 - 4 bytes - trailer_size
* apploader_size à l'offset 0x14, donc à l'offset 0x2454 de l'iso GCM.
* apploader_size is at offset 0x14 of the apploader so at offset 0x2454 of the GCM/iso.
* trailer_size à l'offset 0x18, donc à l'offset 0x2458 de l'iso GCM. La trailer_size est un espace réservé à d'éventuelles évolutions de l'apploader.
* trailer_size is at offset 0x18 of the apploader so at offset 0x2458 of the GCM/iso. The trailer_size is an empty space reserved for further apploader updates.


=== boot.dol ===
=== boot.dol ===
Le fichier au format [[DOL (Format de fichier)|dol]] est un exécutable compilé pour l'architecture Gekko (powerpc). Son [http://wiki.tockdom.com/wiki/DOL_(File_Format) header] fait 0x100 octets, et on trouvera la taille totale du [[DOL (Format de fichier)|dol]] en additionnant la taille du header et la taille de l'ensemble des 18 sections de celui-ci. A partir de l'offset 0x90, on trouvera 18 uint de 4 octets (big endian) contenant les tailles des 18 sections.
The [[DOL (File format)|dol]] file format is an executable compiled for Gekko powerpc RISC architecture. It's [http://wiki.tockdom.com/wiki/DOL_(File_Format) header] has a length of 0x100 bytes and we calculate its total length by summing the header length with all 18 sections in it. At offset 0x90 we found 18 uint of 4 bytes (big endian) containing the length of the 18 sections.


=== File String Table ===
=== File String Table ===
La FST permet de packer l'arborescence de fichiers et de dossiers du jeu dans la suite de l'iso. Elle se divise en deux blocks :
The FST allow to pack the files and folders tree of the game and is divided into two blocks:
* Le premier contient les entrées de la FST (dossiers et fichiers). (Big endian)
* The first one contains FST entries (folders and files) in Big Endian.
* Le second, le name_block, contient tous les noms des dossiers et des fichiers chacun terminés par un NULL byte (0x00). (Little endian)
* The second one is the name_block which contains names of folders and files terminated by a NULL byte (0x00) in Little Endian.


Dans la première partie, on indexera tous les dossiers et les fichiers à partir du numéro 1. Chaque dossier ou fichier est traduit en 12 octets, et la toute première entrée est le nœud racine, portant l'index 0.
In the first block we index each folder and file beginning with the index 1. Each folder or file is translated into a 12 bytes entry. The first entry is the root with index 0.


Les 12 octets d'une entrée pour un dossier :
The 12 bytes for a folder entry:
* 1 octet pour le type, dossier=0x01.
* 1 byte for type, folder=0x01.
* 3 octets uint pour l'offset du nom dans le name_block, commençant à l'offset 0 au début du name_block.
* 3 bytes uint - name_block name offset, relative to the start of the name_block.
* 4 octets uint pour l'index du dossier parent.
* 4 bytes uint - index of the parent folder.
* 4 octets uint pour l'index du dossier ou fichier suivant.
* 4 bytes uint - index of the next file or folder outside of it.


Les 12 octets d'une entrée pour un fichier :
The 12 bytes for a file entry:
* 1 octet pour le type, fichier=0x00.
* 1 byte for type, file=0x00.
* 3 octets uint pour l'offset du nom dans le name_block, commençant à l'offset 0 au début du name_block.
* 3 bytes uint - name_block name offset relative to the start of the name_block.
* 4 octets uint pour l'offset du fichier par rapport au début de l'iso GCM.
* 4 bytes uint - file offset relative to the start of the GCM/iso.
* 4 octets uint pour la taille du fichier.
* 4 bytes uint - file length.


Le nœud racine se situe au tout début de la FST, c'est un dossier avec les 3 octets de l'offset du nom et les 4 octets du dossier parent à 0x00. L'index du fichier suivant est égal au nombre total de dossiers et de fichiers du DVD. On trouvera alors l'offset du name_block en multipliant cet index par 12 et en y ajoutant 12 (taille du nœud racine).
The root node is at the beginning of the FST and has the 3 bytes name_block name offset equal to 0x00 and the 4 bytes of the parent folder id also equal to 0x00. The index of the next file is equal to the total number of files and folders packed in the GCM/iso user space. We can find the name_block offset by multiplying this total by 12 and adding 12 to it (the root node length).


L'alignement de l'ensemble des fichiers est important lors de la construction du boot.bin et de la FST. En effet, on alignera par exemple sur 4 octets, l'iso GCM ne fonctionnant pas au cas contraire.
The alignment is important when building boot.bin and FST and is generally aligned to 4. If adx/sfd or audio files are packed then it's often aligned to 0x800 bytes. When unaligned the game could crash or songs are not loaded correctly.


Les DVD GCM JAP contiennent des caractères non utf-8 dans le name_block. L'application [https://github.com/Virtual-World-RE/NeoGF/tree/main/gcmtool gcmtool.py], permettant de parser les iso GCM et de les patcher, ne prends pas en charge ces jeux là. Dolphin Emulator extraira les fichiers en traduisant le charset original en utf-8.
JAP mini DVD contains non utf-8 chars like JIS charsets in the name_block. If using the Virtual World RE tool [https://github.com/Virtual-World-RE/NeoGF/tree/main/gcmtool gcmtool.py] then you have to install this charsets in your OS. When Dolphin Emulator extracts this JAP files it converts the original charset to utf-8.


Le tris par défaut des dossiers et fichiers est non sensible à la casse (tous les noms en majuscules), et fait apparaître les caractères spéciaux avant les lettres. Néanmoins le format GCM permet de doubler des fichiers aux mêmes offsets ou encore de les positionner dans un ordre arbitraire.
The sorting of files in the FST is often case insensitive and special chars are sorted before letters. However some FST are arbitrary sorted and we found sometimes duplicated files like the dol or even the apploader also packed in the user space.


== Formats d'extraction ==
== Extraction formats ==


Le format GCReEx comporte les informations suivantes :
GCReEx extracted files are as following:
  sys:
  sys:
  |- apploader.img
  |- apploader.img
  |- boot.bin (Information de base)
  |- boot.bin (Disc Header)
  |- bi2.bin  (Information supplémentaires)
  |- bi2.bin  (Disc Header Information)
  |- fst.bin  (FileStringTable)
  |- fst.bin  (FileStringTable)
  \- main.dol ("Executable" DOL)
  \- main.dol ("Executable" DOL)
  root:
  root:
  |- banner.bnr
  |- banner.bnr
  \+ ALL GAME DATA (AFS, etc)
  \+ ALL GAME DATA (AFS, and so on.)


Le format DiscEx comporte moins d'informations :
DiscEx extracted files are as following:
  sys:
  sys:
  |- apploader.img
  |- apploader.img
  |- boot.bin (Information de base du jeu)
  |- boot.bin (Information de base du jeu)
  \- bi2.bin  (Information supplémentaires)
  \- bi2.bin  (Information supplémentaires)
  game.iso : Copie repack de l'iso d'origine.
  game.iso: Repack copy of original iso.


On retrouvera d'autres formats.
We find many others formats.


Ce wiki sur le [https://wiki.gbatemp.net/wiki/NKit/Discs#Disc_Header format nkit] offre une très bonne documentation sur le format GCM.
This wiki about the [https://wiki.gbatemp.net/wiki/NKit/Discs#Disc_Header nkit format] offer a very good documentation about the GCM file format.


[[Category:File format]]
[[Category:File format]]
[[Category:Gotcha Force]]
[[Category:Gotcha Force]]

Latest revision as of 12:29, 7 October 2023

← Gotcha Force

This article is about the GCM file format.


This file format is almost completely documented.
The structure of this file is well known.


The GameCube mini DVD uses GCM file format (Gamecube Disc Image with extension gcm or iso) and can be divided into 6 blocks:
Block Name Description
boot.bin Base Information - Disc Header
bi2.bin Additional Information - Debug, locales - Disc Header Information
apploader.img Main Executable Loader
fst.bin File String Table
boot.dol / elf Main Executable
Files and Folders Tree User Space Files and Folders Tree

The file system is handled by the GameCube and we can see that files are loaded by names no matter in which order they have been stored (Dolphin Emulator).

The dol file and FST are stored after de apploader.img. The dol can be after or before FST. Most of the time dol is before FST. All GCM ROMs have a fixed length (mini DVD length): 1 459 978 240 bytes. Because of that there is often empty spaces between files in the game released containing junk datas or old values partially overridden by new ones during devs. This empty spaces contains unused information. It could be used for mini DVD cyphering hardening. The question could be "does this junks data are the sames in all released mini DVD of the game?" An interesting thing is that burning data at the out border of the DVD increases performances when hardware DVD read head access to it.

GCM/ISO format Architecture

boot.bin

boot.bin file is at beginning of the mini DVD and its length is 0x440 bytes. We found on it:
Offset (hex) Size (bytes) Description
0x00 4 GameCode (ASCII) - A-Za-z0-9
0x04 2 MakerCode (ASCII) - A-Za-z0-9
0x06 1 DiscNumber (0-98)
0x07 1 Version (0-99)
0x08 1 AudioStreaming Flag (0 or 1)
0x09 1 StreamBufferSize (0-15)
0x1C 4 DVDMagic (0xc2339f3d)
0x20 64 GameName (ASCII and possibly Kanji)
0x420 4 DolOffset
0x424 4 FstOffset
0x428 4 FstLen
0x42C 4 FstMaxLen (often corresponds to FST length but reserved for other mini DVDs)
0x430 4 FST Memory Address
0x434 4 UserPosition (Offset on mini DVD where the first game files begin)
0x438 4 UserLength (Total length where all files are located, may contain empty spaces)

bi2.bin

bi2.bin file has a length of 0x2000 bytes and reside just after boot.bin (offset 0x440 of the GCM). It contains locales informations and debug configurations.
Offset (hex) Size (bytes) Description
0x00 4 DebugMonitorSize
0x04 4 SimulatedMemorySize
0x08 4 ArgumentOffset
0x0C 4 DebugFlag
0x10 4 TrackLocation (or MetroTRK for some documentations)
0x14 4 TrackSize (or MetroTRK for some documentations)
0x18 4 CountryCode
0x1C 4 TotalDisc (Total number of discs, <= 99)
0x20 4 LongFileNameSupport (1 = supported, 0 = constrained to 8.3 format)
0x24 4 PadSpec (Controller version for PADSetSpec)
0x28 4 DolLimit (Max length of the sum of data and text sections)
Debug Flags
Value Description
0 Normal Boot
1 Devkit Set to 0; MetroTRK Set to AMC DDH hardware
2 Devkit Set to 1; MetroTRK Set to GDEV hardware
3 Devkit Set to 2; MetroTRK Set to BBA
Country Codes
Value Pays
0 Japan
1 USA
2 PAL
4 Korea

apploader.img

The apploader is a file, loaded by the GameCube BIOS (also called the Initial Program Loader / IPL), that creates virtual memory spaces and rebase the dol datas in those areas before starting it. Apploader is at offset 0x2440 just after the bi2.bin. The GCM/iso doesn't store any name of it so it's name depend to the software used to extract it: AppLoader.ldr / apploader.img (dolphin Emulator) and so on.

The apploader length is calculated with its header information:

  • Header has a length of 0x20 bytes:
    • 0 - 10 bytes - Version
    • 0x10 - 4 bytes - entry_point
    • 0x14 - 4 bytes - size
    • 0x18 - 4 bytes - trailer_size
  • apploader_size is at offset 0x14 of the apploader so at offset 0x2454 of the GCM/iso.
  • trailer_size is at offset 0x18 of the apploader so at offset 0x2458 of the GCM/iso. The trailer_size is an empty space reserved for further apploader updates.

boot.dol

The dol file format is an executable compiled for Gekko powerpc RISC architecture. It's header has a length of 0x100 bytes and we calculate its total length by summing the header length with all 18 sections in it. At offset 0x90 we found 18 uint of 4 bytes (big endian) containing the length of the 18 sections.

File String Table

The FST allow to pack the files and folders tree of the game and is divided into two blocks:

  • The first one contains FST entries (folders and files) in Big Endian.
  • The second one is the name_block which contains names of folders and files terminated by a NULL byte (0x00) in Little Endian.

In the first block we index each folder and file beginning with the index 1. Each folder or file is translated into a 12 bytes entry. The first entry is the root with index 0.

The 12 bytes for a folder entry:

  • 1 byte for type, folder=0x01.
  • 3 bytes uint - name_block name offset, relative to the start of the name_block.
  • 4 bytes uint - index of the parent folder.
  • 4 bytes uint - index of the next file or folder outside of it.

The 12 bytes for a file entry:

  • 1 byte for type, file=0x00.
  • 3 bytes uint - name_block name offset relative to the start of the name_block.
  • 4 bytes uint - file offset relative to the start of the GCM/iso.
  • 4 bytes uint - file length.

The root node is at the beginning of the FST and has the 3 bytes name_block name offset equal to 0x00 and the 4 bytes of the parent folder id also equal to 0x00. The index of the next file is equal to the total number of files and folders packed in the GCM/iso user space. We can find the name_block offset by multiplying this total by 12 and adding 12 to it (the root node length).

The alignment is important when building boot.bin and FST and is generally aligned to 4. If adx/sfd or audio files are packed then it's often aligned to 0x800 bytes. When unaligned the game could crash or songs are not loaded correctly.

JAP mini DVD contains non utf-8 chars like JIS charsets in the name_block. If using the Virtual World RE tool gcmtool.py then you have to install this charsets in your OS. When Dolphin Emulator extracts this JAP files it converts the original charset to utf-8.

The sorting of files in the FST is often case insensitive and special chars are sorted before letters. However some FST are arbitrary sorted and we found sometimes duplicated files like the dol or even the apploader also packed in the user space.

Extraction formats

GCReEx extracted files are as following:

sys:
|- apploader.img
|- boot.bin (Disc Header)
|- bi2.bin  (Disc Header Information)
|- fst.bin  (FileStringTable)
\- main.dol ("Executable" DOL)
root:
|- banner.bnr
\+ ALL GAME DATA (AFS, and so on.)

DiscEx extracted files are as following:

sys:
|- apploader.img
|- boot.bin (Information de base du jeu)
\- bi2.bin  (Information supplémentaires)
game.iso: Repack copy of original iso.

We find many others formats.

This wiki about the nkit format offer a very good documentation about the GCM file format.