Thread:Thaswordster/@comment-26817273-20151013083841

Thanks for your contributions. I haven't been able to add much new content lately as I've been busy with school, so any bit of help is greatly appreciated.

I do have a list of remaining pages/topics I plan on covering when I find the time, currently structured so that each topic corresponds to a game routine/function (updated memory map for game routines here ). I think next I'll start discussing the paging system and the various paging routines-a few of which are already somewhat covered in lines 811 - 1608 of https://github.com/wurlyfox/crashutils/blob/master/doc/crash/disasm_guide.txt. Here's an overview:

"When the game needs the data of an entry, it asks the paging system to page its data from the NSF file on disc into main memory. This is accomplished by calling the pageEntry function and specifying the EID of the entry to be paged as an argument. PageEntry uses the entry hash table (from the NSD file which is already loaded into memory) to find the CID of the chunk that contains that entry, and a new/blank page is created and assigned the state of "pending transfer"; pages are then "updated", in which the blank page pending a transfer is detected, resulting in a transfer of the chunk data from disc to that destination page. This transfer is possible because the sector position on disc of the chunk with that CID is known, by a mapping of CIDs to sector positions located in the NSD; thus, by simply seeking to that sector position on disc and reading 64kb of data, the chunk is transferred successfully into main memory. Then, with the chunk data in that page of memory, PageEntry locates the requested entry with specified EID in the chunk and returns a pointer.

Of course, with only 22 physical pages of data and typically more than 22 chunks in the NSF file, creating a page becomes trivial when all available pages are eventually used up. Furthermore, since it is possible, if the game requests the data of multiple entries all at once, located in possibly several different chunks, it becomes costly to have to repeat "seek to sector, read 64kb" for each chunk. Also, if the game requests the data of an entry from a chunk that is already in a page of memory, it shouldn't have to transfer that chunk again; however, the request for that entry data should be kept track of.

If a new page is created when all pages are used up, by employing the appropriate page replacement algorithms, i.e. least frequently and recently used (page is 'used' when a request is made for the data of an entry its contained chunk), a page can be chosen for replacement.

Since, in a short span of time, the game will typically request only entry data located in chunks that are contiguous/in a local frame (i.e. the entry for the first zone in the level is typically located in one of the first chunks in its NSF, and its neighboring zones are located in the same or one of the nearest following chunks-there is never a huge gap between the container chunks for entries of neighboring zones, so by moving from zone to zone one should expect data requests to remain relatively local), requests for the data of multiple entries that happen to be located in a group of contiguous chunks can be satisfied in a single read of 64 x n kb of data from the sector position of the first chunk in the group. Thus, the slow n-1 individual seeks and transfers are traded for a much faster single seek and transfer.

There are 38 'virtual pages' as well; these are how the game specifies the transfer of data for multiple entries in one request as explained above. When entering the territory of a new zone, the game will preemptively request paging into memory the data for the entries that will be used by the zone [if necessary]; thus, several potential requests for the paging of multiple chunks that do not already exist in a page of memory will "queue up", or rather, a virtual page will be created for each request. Since the requests are only preemptive and the entry data is not required immediately, only during the next "page update" (done for each frame of execution) will actual physical pages be created for paging the chunks with requested entry data. This includes establishing groups of physical pages for any corresponding contiguous chunks to get the performance boost explained above.

Then you have compressed chunks, and the transfer process for them is different since now you've got to do decompression, and etc. And then theres texture chunks and wavebank chunks which have completely separate groups of associated pages and yet another type of transfer process, since now the data paged from disc is going to VRAM or sound hardware. Also when normal chunks are initially paged into memory, many of their contained entries have data with relative pointers, so then the appropriate subsystem routines have to be used to convert them to absolute pointers. Again, these are all topics that will eventually be covered in detail."

Also, feel free to ask if you have any more technical questions. There's a few other discompiled/disassembled GOOL objects here that you might find interesting, namely willc-updated.gooc which comments much of the crash object, turtc.gooctemp which comments the entire crab object, and starting at line 821 of wingc.gooc where I have comments showing the actual GOOL code equivalent for that section [of the ending sequence], as specified in Andy's blog. 