morgan/Grinch-AP
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Ocarina of Time (#64)

Browse Source 
* first commit (not including OoT data files yet)

* added some basic options

* rule parser works now at least

* make sure to commit everything this time

* temporary change to BaseClasses for oot

* overworld location graph builds mostly correctly

* adding oot data files

* commenting out world options until later since they only existed to make the RuleParser work

* conversion functions between AP ids and OOT ids

* world graph outputs

* set scrub prices

* itempool generates, entrances connected, way too many options added

* fixed set_rules and set_shop_rules

* temp baseclasses changes

* Reaches the fill step now, old event-based system retained in case the new way breaks

* Song placements and misc fixes everywhere

* temporary changes to make oot work

* changed root exits for AP fill framework

* prevent infinite recursion due to OoT sharing usage of the address field

* age reachability works hopefully, songs are broken again

* working spoiler log generation on beatable-only

* Logic tricks implemented

* need this for logic tricks

* fixed map/compass being placed on Serenade location

* kill unreachable events before filling the world

* add a bunch of utility functions to prepare for rom patching

* move OptionList into generic options

* fixed some silly bugs with OptionList

* properly seed all random behavior (so far)

* ROM generation working

* fix hints trying to get alttp dungeon hint texts

* continue fixing hints

* add oot to network data package

* change item and location IDs to 66000 and 67000 range respectively

* push removed items to precollected items

* fixed various issues with cross-contamination with multiple world generation

* reenable glitched logic (hopefully)

* glitched world files age-check fix

* cleaned up some get_locations calls

* added token shuffle and scrub shuffle, modified some options slightly to make the parsing work

* reenable MQ dungeons

* fix forest mq exception

* made targeting style an option for now, will be cosmetic later

* reminder to move targeting to cosmetics

* some oot option maintenance

* enabled starting time of day

* fixed issue breaking shop slots in multiworld generation

* added "off" option for text shuffle and hints

* shopsanity functionality restored

* change patch file extension

* remove unnecessary utility functions + imports

* update MIT license

* change option to "patch_uncompressed_rom" instead of "compress_rom"

* compliance with new AutoWorld systems

* Kill only internal events, remove non-internal big poe event in code

* re-add the big poe event and handle it correctly

* remove extra method in Range option

* fix typo

* Starting items, starting with consumables option

* do not remove nonexistent item

* move set_shop_rules to after shop items are placed

* some cleanup

* add retries for song placement

* flagged Skull Mask and Mask of Truth as advancement items

* update OoT to use LogicMixin

* Fixed trying to assign starting items from the wrong players

* fixed song retry step

* improved option handling, comments, and starting item replacements

* DefaultOnToggle writes Yes or No to spoiler

* enable compression of output if Compress executable is present

* clean up compression

* check whether (de)compressor exists before running the process

* allow specification of rom path in host.yaml

* check if decompressed file already exists before decompressing again

* fix triforce hunt generation

* rename all the oot state functions with prefix

* OoT: mark triforce pieces as completion goal for triforce hunt

* added overworld and any-dungeon shuffle for dungeon items

* Hide most unshuffled locations and events from the list of locations in spoiler

* build oot option ranges with a generic function instead of defining each separately

* move oot output-type control to host.yaml instead of individual yamls

* implement dungeon song shuffle

* minor improvements to overworld dungeon item shuffle

* remove random ice trap names in shops, mostly to avoid maintaining a massive censor list

* always output patch file to folder, remove option to generate ROM in preparation for removal

* re-add the fix for infinite recursion due to not being light or dark world

* change AP-sendable to Ocarina of Time model, since the triforce piece has some extra code apparently

* oot: remove item_names and location_names

* oot: minor fixes

* oot: comment out ROM patching

* oot: only add CollectionState objects on creation if actually needed

* main entrance shuffle method and entrances-based rules

* fix entrances based rules

* disable master quest and big poe count options for client compatibility

* use get_player_name instead of get_player_names

* fix OptionList

* fix oot options for new option system

* new coop section in oot rom: expand player names to 16 bytes, write AP_PLAYER_NAME at end of PLAYER_NAMES

* fill AP player name in oot rom with 0 instead of 0xDF

* encode player name with ASCII for fixed-width

* revert oot player name array to 8 bytes per name

* remove Pierre location if fast scarecrow is on

* check player name length

* "free_scarecrow" not "fast_scarecrow"

* OoT locations now properly store the AP ID instead of the oot internal ID

* oot __version__ updates in lockstep with AP version

* pull in unmodified oot cosmetic files

* also grab JSONDump since it's needed apparently

* gather extra needed methods, modify imports

* delete cosmetics log, replace all instances of SettingsList with OOTWorld

* cosmetic options working, except for sound effects (due to ear-safe issues)

* SFX, Music, and Fanfare randomization reenabled

* move OoT data files into the worlds folder

* move Compress and Decompress into oot data folder

* Replace get_all_state with custom method to avoid the cache

* OoT ROM: increment item counter before setting incoming item/player values to 0, preventing desync issues

* set data_version to 0

* make Kokiri Sword shuffle off by default

* reenable "Random Choice" for various cosmetic options

* kill Ruto's Letter turnin if open fountain
also fix for shopsanity

* place Buy Goron/Zora Tunic first in shop shuffle

* make ice traps appear as other items instead of breaking generation

* managed to break ice traps on non-major-only

* only handle ice traps if they are on

* fix shopsanity for non-oot games, and write player name instead of player number

* light arrows hint uses player name instead of player number

* Reenable "skip child zelda" option

* fix entrances_based_rules

* fix ganondorf hint if starting with light arrows

* fix dungeonitem shuffle and shopsanity interaction

* remove has_all_of, has_any_of, count_of in BaseClasses, replace usage with has_all, has_any, has_group

* force local giveable item on ZL if skip_child_zelda and shuffle_song_items is any

* keep bosses and bombchu bowling chus out of data package

* revert workaround for infinite recursion and fix it properly

* fix shared shop id caches during patching process

* fix shop text box overflows, as much as possible

* add default oot host.yaml option

* add .apz5, .n64, .z64 to gitignore

* Properly document and name all (functioning) OOT options

* clean up some imports

* remove unnecessary files from oot's data

* fix typo in gitignore

* readd the Compress and Decompress utilities, since they are needed for generation

* cleanup of imports and some minor optimizations

* increase shop offset for item IDs to 0xCB

* remove shop item AP ids entirely

* prevent triforce pieces for other players from being received by yourself

* add "excluded" property to Location

* Hint system adapted and reenabled; hints still unseeded

* make hints deterministic with lists instead of sets

* do not allow hints to point to Light Arrows on non-vanilla bridge

* foreign locations hint as their full name in OoT rather than their region

* checkedLocations now stores hint names by player ID, so that the same location in different worlds can have hints associated

* consolidate versioning in Utils

* ice traps appear as major items rather than any progression item

* set prescription and claim check as defaults for adult trade item settings

* add oot options to playerSettings

* allow case-insensitive logic tricks in yaml

* fix oot shopsanity option formatting

* Write OoT override info even if local item, enabling local checks to show up immediately in the client

* implement CollectionState.can_live_dmg for oot glitched logic

* filter item names for invalid characters when patching shops

* make ice traps appear according to the settings of the world they are shuffled into, rather than the original world

* set hidden-spoiler items and locations with Shop items to events

* make GF carpenters, Gerudo Card, Malon, ZL, and Impa events if the relevant settings are enabled, preventing them from appearing in the client on game start

* Fix oot Glitched and No Logic generation

* fix indenting

* Greatly reduce displayed cosmetic options

* Change oot data version to 1

* add apz5 distribution to webhost

* print player name if an ALttP dungeon contains a good item for OoT world

* delete unneeded commented code

* remove OcarinaSongs import to satisfy lint
This commit is contained in:
espeon65536
2021-09-02 08:35:05 -04:00
committed by GitHub
parent 74c30ce09a
commit 51c38fc628
134 changed files with 80588 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
worlds/oot/data/Compress/src/bSwap.h Normal file
View File

@@ -0,0 +1,20 @@
#ifndef BSWAP_H
#define BSWAP_H
#include <stdint.h>
uint32_t bSwap32(uint32_t a)
{
return( (a & 0x000000FF) << 24 |
(a & 0x0000FF00) << 8 |
(a & 0x00FF0000) >> 8 |
(a & 0xFF000000) >> 24 );
}
uint16_t bSwap16(uint16_t a)
{
return( (a & 0x00FF) << 8 |
(a & 0xFF00) >> 8 );
}
#endif

601
worlds/oot/data/Compress/src/compressor.c Normal file
View File

@@ -0,0 +1,601 @@
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <unistd.h>
#include <string.h>
#include <pthread.h>
#include "bSwap.h"
#include "yaz0.c"
#include "crc.c"
/* Needed to compile on Windows */
#ifdef _WIN32
#include <Windows.h>
#endif
/* Different ROM sizes */
#define UINTSIZE 0x1000000
#define COMPSIZE 0x2000000
/* Number of extra bytes to add to compression buffer */
#define COMPBUFF 0x250
//Structs {{{1
/* DMA table entry */
typedef struct
{
uint32_t startV;
uint32_t endV;
uint32_t startP;
uint32_t endP;
}
table_t;
/* Temporary storage for output data */
typedef struct
{
table_t table;
uint8_t* data;
uint8_t comp;
uint32_t size;
}
output_t;
/* Archive struct */
typedef struct
{
uint32_t fileCount;
uint32_t* refSize;
uint32_t* srcSize;
uint8_t** ref;
uint8_t** src;
}
archive_t;
/* 1}}} */
/* Functions {{{1 */
uint32_t findTable(uint8_t*);
void getTableEnt(table_t*, uint32_t*, uint32_t);
void* threadFunc(void*);
void errorCheck(int, char**);
void makeArchive();
int32_t getNumCores();
int32_t getNext();
/* 1}}} */
/* Globals {{{1 */
char* inName;
char* outName;
uint8_t* inROM;
uint8_t* outROM;
uint8_t* refTab;
pthread_mutex_t filelock;
pthread_mutex_t countlock;
int32_t numFiles, nextFile;
int32_t arcCount, outSize;
uint32_t* fileTab;
archive_t* archive;
output_t* out;
/* 1}}} */
/* int main(int, char**) {{{1 */
int main(int argc, char** argv)
{
FILE* file;
int32_t tabStart, tabSize, tabCount, junk;
volatile int32_t prev;
int32_t i, j, size, numCores, tempSize;
pthread_t* threads;
table_t tab;
errorCheck(argc, argv);
printf("Zelda64 Compressor, Version 2\n");
fflush(stdout);
/* Open input, read into inROM */
file = fopen(argv[1], "rb");
fseek(file, 0, SEEK_END);
tempSize = ftell(file);
fseek(file, 0, SEEK_SET);
inROM = calloc(tempSize, sizeof(uint8_t));
junk = fread(inROM, tempSize, 1, file);
fclose(file);
/* Read archive if it exists*/
file = fopen("ARCHIVE.bin", "rb");
if(file != NULL)
{
/* Get number of files */
printf("Loading Archive.\n");
fflush(stdout);
archive = malloc(sizeof(archive_t));
junk = fread(&(archive->fileCount), sizeof(uint32_t), 1, file);
/* Allocate space for files and sizes */
archive->refSize = malloc(sizeof(uint32_t) * archive->fileCount);
archive->srcSize = malloc(sizeof(uint32_t) * archive->fileCount);
archive->ref = malloc(sizeof(uint8_t*) * archive->fileCount);
archive->src = malloc(sizeof(uint8_t*) * archive->fileCount);
/* Read in file size and then file data */
for(i = 0; i < archive->fileCount; i++)
{
/* Decompressed "Reference" file */
junk = fread(&tempSize, sizeof(uint32_t), 1, file);
archive->ref[i] = malloc(tempSize);
archive->refSize[i] = tempSize;
junk = fread(archive->ref[i], 1, tempSize, file);
/* Compressed "Source" file */
junk = fread(&tempSize, sizeof(uint32_t), 1, file);
archive->src[i] = malloc(tempSize);
archive->srcSize[i] = tempSize;
junk = fread(archive->src[i], 1, tempSize, file);
}
fclose(file);
}
else
{
printf("No archive found, this could take a while.\n");
fflush(stdout);
archive = NULL;
}
/* Find the file table and relevant info */
tabStart = findTable(inROM);
fileTab = (uint32_t*)(inROM + tabStart);
getTableEnt(&tab, fileTab, 2);
tabSize = tab.endV - tab.startV;
tabCount = tabSize / 16;
/* Allocate space for the exclusion list */
/* Default to 1 (compress), set exclusions to 0 */
file = fopen("dmaTable.dat", "r");
size = tabCount - 1;
refTab = malloc(sizeof(uint8_t) * size);
memset(refTab, 1, size);
/* The first 3 files are never compressed */
/* They should never be given to the compression function anyway though */
refTab[0] = refTab[1] = refTab[2] = 0;
/* Read in the rest of the exclusion list */
for(i = 0; fscanf(file, "%d", &j) == 1; i++)
{
/* Make sure the number is within the dmaTable */
if(j > size || j < -size)
{
fprintf(stderr, "Error: Entry %d in dmaTable.dat is out of bounds\n", i);
exit(1);
}
/* If j was negative, the file shouldn't exist */
/* Otherwise, set file to not compress */
if(j < 0)
refTab[(~j + 1)] = 2;
else
refTab[j] = 0;
}
fclose(file);
/* Initialise some stuff */
out = malloc(sizeof(output_t) * tabCount);
pthread_mutex_init(&filelock, NULL);
pthread_mutex_init(&countlock, NULL);
numFiles = tabCount;
outSize = COMPSIZE;
nextFile = 3;
arcCount = 0;
/* Get CPU core count */
numCores = getNumCores();
threads = malloc(sizeof(pthread_t) * numCores);
printf("Detected %d cores.\n", (numCores));
printf("Starting compression.\n");
fflush(stdout);
/* Create all the threads */
for(i = 0; i < numCores; i++)
pthread_create(&threads[i], NULL, threadFunc, NULL);
/* Wait for all of the threads to finish */
for(i = 0; i < numCores; i++)
pthread_join(threads[i], NULL);
printf("\n");
/* Get size of new ROM */
/* Start with size of first 3 files */
tempSize = tabStart + tabSize;
for(i = 3; i < tabCount; i++)
tempSize += out[i].size;
/* If ROM is too big, update size */
if(tempSize > outSize)
outSize = tempSize;
/* Setup for copying to outROM */
printf("Files compressed, writing new ROM.\n");
outROM = calloc(outSize, sizeof(uint8_t));
memcpy(outROM, inROM, tabStart + tabSize);
prev = tabStart + tabSize;
tabStart += 0x20;
/* Free some stuff */
pthread_mutex_destroy(&filelock);
pthread_mutex_destroy(&countlock);
if(archive != NULL)
{
free(archive->ref);
free(archive->src);
free(archive->refSize);
free(archive->srcSize);
free(archive);
}
free(threads);
free(refTab);
/* Write data to outROM */
for(i = 3; i < tabCount; i++)
{
tab = out[i].table;
size = out[i].size;
tabStart += 0x10;
/* Finish table and copy to outROM */
if(tab.startV != tab.endV)
{
/* Set up physical addresses */
tab.startP = prev;
if(out[i].comp == 1)
tab.endP = tab.startP + size;
else if(out[i].comp == 2)
tab.startP = tab.endP = 0xFFFFFFFF;
/* If the file existed, write it */
if(tab.startP != 0xFFFFFFFF)
memcpy(outROM + tab.startP, out[i].data, size);
/* Write the table entry */
tab.startV = bSwap32(tab.startV);
tab.endV = bSwap32(tab.endV);
tab.startP = bSwap32(tab.startP);
tab.endP = bSwap32(tab.endP);
memcpy(outROM + tabStart, &tab, sizeof(table_t));
}
prev += size;
if(out[i].data != NULL)
free(out[i].data);
}
free(out);
/* Fix the CRC before writing the ROM */
fix_crc(outROM);
/* Make and fill the output ROM */
file = fopen(outName, "wb");
fwrite(outROM, outSize, 1, file);
fclose(file);
/* Make the archive if needed */
if(archive == NULL)
{
printf("Creating archive.\n");
makeArchive();
}
/* Free up the last bit of memory */
if(argc != 3)
free(outName);
free(inROM);
free(outROM);
printf("Compression complete.\n");
return(0);
}
/* 1}}} */
/* uint32_t findTAble(uint8_t*) {{{1 */
uint32_t findTable(uint8_t* argROM)
{
uint32_t i;
uint32_t* tempROM;
tempROM = (uint32_t*)argROM;
/* Start at the end of the makerom (0x10600000) */
/* Look for dma entry for the makeom */
/* Should work for all Zelda64 titles */
for(i = 1048; i+4 < UINTSIZE; i += 4)
{
if(tempROM[i] == 0x00000000)
if(tempROM[i+1] == 0x60100000)
return(i * 4);
}
fprintf(stderr, "Error: Couldn't find dma table in ROM!\n");
exit(1);
}
/* 1}}} */
/* void getTableEnt(table_t*, uint32_t*, uint32_t) {{{1 */
void getTableEnt(table_t* tab, uint32_t* files, uint32_t i)
{
tab->startV = bSwap32(files[i*4]);
tab->endV = bSwap32(files[(i*4)+1]);
tab->startP = bSwap32(files[(i*4)+2]);
tab->endP = bSwap32(files[(i*4)+3]);
}
/* 1}}} */
/* void* threadFunc(void*) {{{1 */
void* threadFunc(void* null)
{
uint8_t* src;
uint8_t* dst;
table_t t;
int32_t i, nextArchive, size, srcSize;
while((i = getNext()) != -1)
{
/* Setup the src */
getTableEnt(&(t), fileTab, i);
srcSize = t.endV - t.startV;
src = inROM + t.startV;
/* If refTab is 1, compress */
/* If refTab is 2, file shouldn't exist */
/* Otherwise, just copy src into out */
if(refTab[i] == 1)
{
pthread_mutex_lock(&countlock);
nextArchive = arcCount++;
pthread_mutex_unlock(&countlock);
/* If uncompressed is the same as archive, just copy/paste the compressed */
/* Otherwise, compress it manually */
if((archive != NULL) && (memcmp(src, archive->ref[nextArchive], archive->refSize[nextArchive]) == 0))
{
out[i].comp = 1;
size = archive->srcSize[nextArchive];
out[i].data = malloc(size);
memcpy(out[i].data, archive->src[nextArchive], size);
}
else
{
size = srcSize + COMPBUFF;
dst = calloc(size, sizeof(uint8_t));
yaz0_encode(src, srcSize, dst, &(size));
out[i].comp = 1;
out[i].data = malloc(size);
memcpy(out[i].data, dst, size);
free(dst);
}
if(archive != NULL)
{
free(archive->ref[nextArchive]);
free(archive->src[nextArchive]);
}
}
else if(refTab[i] == 2)
{
out[i].comp = 2;
size = 0;
out[i].data = NULL;
}
else
{
out[i].comp = 0;
size = srcSize;
out[i].data = malloc(size);
memcpy(out[i].data, src, size);
}
/* Set up the table entry and size */
out[i].table = t;
out[i].size = size;
}
return(NULL);
}
/* 1}}} */
/* void makeArchive() {{{1 */
void makeArchive()
{
table_t tab;
uint32_t tabSize, tabCount, tabStart;
uint32_t fileSize, fileCount, i;
FILE* file;
/* Find DMAtable info */
tabStart = findTable(outROM);
fileTab = (uint32_t*)(outROM + tabStart);
getTableEnt(&tab, fileTab, 2);
tabSize = tab.endV - tab.startV;
tabCount = tabSize / 16;
fileCount = 0;
/* Find the number of compressed files in the ROM */
/* Ignore first 3 files, as they're never compressed */
for(i = 3; i < tabCount; i++)
{
getTableEnt(&tab, fileTab, i);
if(tab.endP != 0 && tab.endP != 0xFFFFFFFF)
fileCount++;
}
/* Open output file */
file = fopen("ARCHIVE.bin", "wb");
if(file == NULL)
{
perror("ARCHIVE.bin");
fprintf(stderr, "Error: Could not create archive\n");
return;
}
/* Write the archive data */
fwrite(&fileCount, sizeof(uint32_t), 1, file);
/* Write the fileSize and data for each ref & src */
for(i = 3; i < tabCount; i++)
{
getTableEnt(&tab, fileTab, i);
if(tab.endP != 0 && tab.endP != 0xFFFFFFFF)
{
/* Write the size and data for the decompressed portion */
fileSize = tab.endV - tab.startV;
fwrite(&fileSize, sizeof(uint32_t), 1, file);
fwrite(inROM + tab.startV, 1, fileSize, file);
/* Write the size and data for the compressed portion */
fileSize = tab.endP - tab.startP;
fwrite(&fileSize, sizeof(uint32_t), 1, file);
fwrite((outROM + tab.startP), 1, fileSize, file);
}
}
fclose(file);
}
/* 1}}} */
/* int32_t getNumCores() {{{1 */
int32_t getNumCores()
{
/* Windows */
#ifdef _WIN32
SYSTEM_INFO info;
GetSystemInfo(&info);
return(info.dwNumberOfProcessors);
/* Mac */
#elif MACOS
int nm[2];
size_t len;
uint32_t count;
len = 4;
nm[0] = CTL_HW;
nm[1] = HW_AVAILCPU;
sysctl(nm, 2, &count, &len, NULL, 0);
if (count < 1)
{
nm[1] = HW_NCPU;
sysctl(nm, 2, &count, &len, NULL, 0);
if (count < 1)
count = 1;
}
return(count);
/* Linux */
#else
return(sysconf(_SC_NPROCESSORS_ONLN));
#endif
}
/* 1}}} */
/* int32_t getNext() {{{1 */
int32_t getNext()
{
int32_t file, temp;
pthread_mutex_lock(&filelock);
file = nextFile++;
/* Progress tracker */
if (file < numFiles)
{
temp = numFiles - (file + 1);
printf("%d files remaining\n", temp);
fflush(stdout);
}
else
{
file = -1;
}
pthread_mutex_unlock(&filelock);
return(file);
}
/* 1}}} */
/* void errorCheck(int, char**) {{{1 */
void errorCheck(int argc, char** argv)
{
int i, j;
FILE* file;
/* Check for arguments */
if(argc < 2)
{
fprintf(stderr, "Usage: %s [Input ROM] <Output ROM>\n", argv[0]);
exit(1);
}
/* Check that input ROM exists & has permissions */
inName = argv[1];
file = fopen(inName, "rb");
if(file == NULL)
{
perror(inName);
exit(1);
}
/* Check that dmaTable.dat exists & has permissions */
file = fopen("dmaTable.dat", "r");
if(file == NULL)
{
perror("dmaTable.dat");
fprintf(stderr, "Please make a dmaTable.dat file first\n");
exit(1);
}
/* Check that output ROM is writeable */
/* Create output filename if needed */
if(argc < 3)
{
i = strlen(inName) + 6;
outName = malloc(i);
strcpy(outName, inName);
for(; i >= 0; i--)
{
if(outName[i] == '.')
{
outName[i] = '\0';
break;
}
}
strcat(outName, "-comp.z64");
file = fopen(outName, "wb");
if(file == NULL)
{
perror(outName);
free(outName);
exit(1);
}
fclose(file);
}
else
{
outName = argv[2];
file = fopen(outName, "wb");
if(file == NULL)
{
perror(outName);
exit(1);
}
fclose(file);
}
}
/* 1}}} */

175
worlds/oot/data/Compress/src/crc.c Normal file
View File

@@ -0,0 +1,175 @@
/* snesrc - SNES Recompiler
*
* Copyright notice for this file:
* Copyright (C) 2005 Parasyte
*
* Based on uCON64's N64 checksum algorithm by Andreas Sterbenz
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <stdio.h>
#include <stdlib.h>
#define ROL(i, b) (((i) << (b)) | ((i) >> (32 - (b))))
#define BYTES2LONG(b) ( (b)[0] << 24 | \
(b)[1] << 16 | \
(b)[2] << 8 | \
(b)[3] )
#define N64_HEADER_SIZE 0x40
#define N64_BC_SIZE (0x1000 - N64_HEADER_SIZE)
#define N64_CRC1 0x10
#define N64_CRC2 0x14
#define CHECKSUM_START 0x00001000
#define CHECKSUM_LENGTH 0x00100000
#define CHECKSUM_CIC6102 0xF8CA4DDC
#define CHECKSUM_CIC6103 0xA3886759
#define CHECKSUM_CIC6105 0xDF26F436
#define CHECKSUM_CIC6106 0x1FEA617A
unsigned int crc_table[256];
void gen_table()
{
uint32_t crc, poly;
int32_t i, j;
poly = 0xEDB88320;
for (i = 0; i < 256; i++) {
crc = i;
for (j = 8; j > 0; j--) {
if (crc & 1) crc = (crc >> 1) ^ poly;
else crc >>= 1;
}
crc_table[i] = crc;
}
}
unsigned int crc32(unsigned char *data, int len)
{
uint32_t crc, i;
crc = ~0;
for (i = 0; i < len; i++)
crc = (crc >> 8) ^ crc_table[(crc ^ data[i]) & 0xFF];
return ~crc;
}
int N64GetCIC(unsigned char *data)
{
switch (crc32(&data[N64_HEADER_SIZE], N64_BC_SIZE)) {
case 0x6170A4A1: return 6101;
case 0x90BB6CB5: return 6102;
case 0x0B050EE0: return 6103;
case 0x98BC2C86: return 6105;
case 0xACC8580A: return 6106;
}
return 0;
}
int N64CalcCRC(unsigned int *crc, unsigned char *data)
{
int32_t bootcode, i;
uint32_t seed, r, d;
uint32_t t1, t2, t3;
uint32_t t4, t5, t6;
switch ((bootcode = N64GetCIC(data))) {
case 6101:
case 6102:
seed = CHECKSUM_CIC6102;
break;
case 6103:
seed = CHECKSUM_CIC6103;
break;
case 6105:
seed = CHECKSUM_CIC6105;
break;
case 6106:
seed = CHECKSUM_CIC6106;
break;
default:
return 0;
}
t1 = t2 = t3 = t4 = t5 = t6 = seed;
i = CHECKSUM_START;
while (i < (CHECKSUM_START + CHECKSUM_LENGTH)) {
d = BYTES2LONG(&data[i]);
if ((t6 + d) < t6) t4++;
t6 += d;
t3 ^= d;
r = ROL(d, (d & 0x1F));
t5 += r;
if (t2 > d) t2 ^= r;
else t2 ^= t6 ^ d;
if (bootcode == 6105) t1 += BYTES2LONG(&data[N64_HEADER_SIZE + 0x0710 + (i & 0xFF)]) ^ d;
else t1 += t5 ^ d;
i += 4;
}
if (bootcode == 6103) {
crc[0] = (t6 ^ t4) + t3;
crc[1] = (t5 ^ t2) + t1;
}
else if (bootcode == 6106) {
crc[0] = (t6 * t4) + t3;
crc[1] = (t5 * t2) + t1;
}
else {
crc[0] = t6 ^ t4 ^ t3;
crc[1] = t5 ^ t2 ^ t1;
}
return 1;
}
void fix_crc (uint8_t* buffer)
{
uint8_t CRC1[4];
uint8_t CRC2[4];
uint32_t crc[2];
uint32_t i;
gen_table();
/* If the CRC calc was successful, do stuff */
if(N64CalcCRC(crc, buffer))
{
for(i = 0; i < 4; i++)
{
CRC1[i] = (crc[0] >> (24-8*i))&0xFF;
CRC2[i] = (crc[1] >> (24-8*i))&0xFF;
}
/* If the CRC1 changed, update it */
if(crc[0] != BYTES2LONG(&buffer[N64_CRC1]))
memcpy(buffer + N64_CRC1, CRC1, 4);
/* If the CRC2 changed, update it */
if (crc[1] != BYTES2LONG(&buffer[N64_CRC2]))
memcpy(buffer + N64_CRC2, CRC2, 4);
}
}

186
worlds/oot/data/Compress/src/yaz0.c Normal file
View File

@@ -0,0 +1,186 @@
#include <stdlib.h>
#include <stdint.h>
#include <string.h>
uint32_t RabinKarp(uint8_t*, int, int, uint32_t*);
uint32_t findBest(uint8_t*, int, int, uint32_t*, uint32_t*, uint32_t*, uint8_t*);
int yaz0_internal(uint8_t*, int, uint8_t*);
void yaz0_encode(uint8_t*, int, uint8_t*, int*);
uint32_t RabinKarp(uint8_t* src, int srcSize, int srcPos, uint32_t* matchPos)
{
int startPos, smp, i;
uint32_t hash, curHash, curSize;
uint32_t bestSize, bestPos;
smp = srcSize - srcPos;
startPos = srcPos - 0x1000;
bestPos = bestSize = 0;
/* If available size is too small, return */
if(smp < 3)
return(0);
/* If available size is too big, reduce it */
if(smp > 0x111)
smp = 0x111;
/* If start position is negative, make it 0 */
if(startPos < 0)
startPos = 0;
/* Generate "hash" by converting to an int */
hash = bSwap32(*(int*)(src + srcPos));
hash = hash >> 8;
curHash = bSwap32(*(int*)(src + startPos));
curHash = curHash >> 8;
/* Search through data */
for(i = startPos; i < srcPos; i++)
{
/* If 3 bytes match, check for more */
if(curHash == hash)
{
for(curSize = 3; curSize < smp; curSize++)
if(src[i + curSize] != src[srcPos + curSize])
break;
/* Uodate best if needed */
if(curSize > bestSize)
{
bestSize = curSize;
bestPos = i;
if(bestSize == 0x111)
break;
}
}
/* Scoot over 1 byte */
curHash = (curHash << 8 | src[i + 3]) & 0x00FFFFFF;
}
/* Set match position, return the size of the match */
*matchPos = bestPos;
return(bestSize);
}
uint32_t findBest(uint8_t* src, int srcSize, int srcPos, uint32_t* matchPos, uint32_t* pMatch, uint32_t* pSize, uint8_t* pFlag)
{
int rv;
/* Check to see if this location was found by a look-ahead */
if(*pFlag == 1)
{
*pFlag = 0;
return(*pSize);
}
/* Find best match */
*pFlag = 0;
rv = RabinKarp(src, srcSize, srcPos, matchPos);
/* Look-ahead */
if(rv >= 3)
{
/* Find best match if current one were to be a 1 byte copy */
*pSize = RabinKarp(src, srcSize, srcPos+1, pMatch);
if(*pSize >= rv+2)
{
rv = *pFlag = 1;
*matchPos = *pMatch;
}
}
return(rv);
}
int yaz0_internal(uint8_t* src, int srcSize, uint8_t* dst)
{
int dstPos, srcPos, codeBytePos;
uint32_t numBytes, matchPos, dist, pMatch, pSize;
uint8_t codeByte, bitmask, pFlag;
srcPos = codeBytePos = 0;
dstPos = codeBytePos + 1;
bitmask = 0x80;
codeByte = pFlag = 0;
/* Go through all of src */
while(srcPos < srcSize)
{
/* Try to find matching bytes for compressing */
numBytes = findBest(src, srcSize, srcPos, &matchPos, &pMatch, &pSize, &pFlag);
/* Single byte copy */
if(numBytes < 3)
{
dst[dstPos++] = src[srcPos++];
codeByte |= bitmask;
}
/* Three byte encoding */
else if (numBytes > 0x11)
{
dist = srcPos - matchPos - 1;
/* Copy over 0R RR */
dst[dstPos++] = dist >> 8;
dst[dstPos++] = dist & 0xFF;
/* Reduce N if needed, copy over NN */
if(numBytes > 0x111)
numBytes = 0x111;
dst[dstPos++] = (numBytes - 0x12) & 0xFF;
srcPos += numBytes;
}
/* Two byte encoding */
else
{
dist = srcPos - matchPos - 1;
/* Copy over NR RR */
dst[dstPos++] = ((numBytes - 2) << 4) | (dist >> 8);
dst[dstPos++] = dist & 0xFF;
srcPos += numBytes;
}
/* Move bitmask to next byte */
bitmask = bitmask >> 1;
/* If all 8 bytes were used, write and move to the next one */
if(bitmask == 0)
{
dst[codeBytePos] = codeByte;
codeBytePos = dstPos;
if(srcPos < srcSize)
dstPos++;
codeByte = 0;
bitmask = 0x80;
}
}
/* Copy over last byte if it hasn't already */
if(bitmask != 0)
dst[codeBytePos] = codeByte;
/* Return size of dst */
return(dstPos);
}
void yaz0_encode(uint8_t* src, int srcSize, uint8_t* dst, int* dstSize)
{
int temp;
/* Write Yaz0 header */
temp = bSwap32(srcSize);
memcpy(dst, "Yaz0", 4);
memcpy(dst + 4, &temp, 4);
/* Encode, adjust dstSize */
temp = yaz0_internal(src, srcSize, dst + 16);
*dstSize = (temp + 31) & -16;
return;
}