Pokemon Emerald: v2 Update (#2918)

worlds/pokemon_emerald/docs/en_Pokemon Emerald.md

@@ -38,8 +38,8 @@ Except for badges, your starting inventory will be in the PC.
 
 ## What does another world's item look like in Pokémon Emerald?
 
-When you find an item that is not your own, you will instead receive an "ARCHIPELAGO ITEM" which will *not* be added to
-your inventory.
+When you find an item that is not your own, you will see the item's name and its owner while the item received jingle
+plays.
 
 ## When the player receives an item, what happens?
 
@@ -55,9 +55,9 @@ you're playing a multiworld game, the client will sync your game with the server
 
 ## Will battle mechanics be updated?
 
-This is something we'd love to see, but it's unlikely. We don't want to force new mechanics on players who would prefer
-to play with the classic mechanics, but trying to switch between old and new mechanics based on an option would be a
-monumental task, and is probably best solved some other way.
+Unfortunately, no. We don't want to force new mechanics on players who would prefer to play with the classic mechanics,
+but updating would require such drastic changes to the underlying code that it would be unreasonable to toggle between
+them.
 
 ## Is this randomizer compatible with other mods?
 
@@ -68,11 +68,21 @@ suggestion or contribute.
 
 ## Can I use tools like the Universal Pokémon Randomizer?
 
-No, those tools expect data to be in certain locations and in a certain format, but this randomizer has to shift it
+No, tools like UPR expect data to be in certain locations and in a certain format, but this randomizer has to shift it
 around. Using tools to try to modify the game would only corrupt the ROM.
 
 We realize this means breaking from established habits when it comes to randomizing Pokémon games, but this randomizer
 would be many times more complex to develop if it were constrained by something like UPR.
 
-The one exception might be PKHeX. You may be able to extract pokémon from your save using PKHeX, but this isn't a
-guarantee, and we make no effort to keep our saves compatible with PKHeX.
+### There are two possible exceptions
+
+#### PKHex
+
+You may be able to extract pokémon from your save using PKHeX, but this isn't a guarantee, and we make no effort to keep
+our saves compatible with PKHeX. Box and party pokémon are the only aspects of your save file likely to work.
+
+#### PokéFinder/RNG Reporter
+
+In the spirit of randomization, Emerald's broken RNG is fixed in Archipelago. More specifically, it's reverted to work
+as it did in Ruby/Sapphire. So while you can't make the assumption that the RNG is seeded at 0, you can set the battery
+to dry, which will seed it in the same way that Ruby/Sapphire are seeded when the battery is dry.

worlds/pokemon_emerald/docs/region data.md

@@ -0,0 +1,79 @@
+## Region Data
+
+Regions, connections, and associated locations are defined in `data/regions`. If you know what you're doing, it should
+be pretty clear how the data works by taking a quick look through the files. But the quick tl;dr is:
+
+- Every map, even trivial ones, gets a region definition, and they cannot be coalesced (because of warp rando)
+- Stick to the naming convention for regions and events (look at Route 103 and Petalburg City for guidance)
+- Locations and warps can only be claimed by one region
+- Events are declared here
+
+A `Map`, which you will see referenced in `parent_map` attribute in the region JSON, is an id from the source code.
+`Map`s are sets of tiles, encounters, warps, events, and so on. Route 103, Littleroot Town, the Oldale Town Mart, the
+second floor of Devon Corp, and each level of Victory Road are all examples of `Map`s. You transition between `Map`s by
+stepping on a warp (warp pads, doorways, etc.) or walking over a border between `Map`s in the overworld. Some warps
+don't go to a different `Map`.
+
+Regions usually describe physical areas which are subsets of a `Map`. Every `Map` must have one or more defined regions.
+A region should not contain area from more than one `Map`. We'll need to draw those lines now even when there is no
+logical boundary (like between two the first and second floors of your rival's house), for warp rando.
+
+Most `Map`s have been split into multiple regions. In the example below, `MAP_ROUTE103` was split into
+`REGION_ROUTE_103/WEST`, `REGION_ROUTE_103/WATER`, and `REGION_ROUTE_103/EAST` (this document may be out of date; the
+example is demonstrative). Keeping the name consistent with the `Map` name and adding a label suffix for the subarea
+makes it clearer where we are in the world and where within a `Map` we're describing.
+
+Every region (except `Menu`) is configured here. All files in this directory are combined with each other at runtime,
+and are only split and ordered for organization. Regions defined in `data/regions/unused` are remnants from
+automatically generated regions and represent places that exist but aren't reachable or aren't currently relevant to the
+randomizer. Any locations or warps in there should be ignored. Data for a single region looks like this:
+
+```json
+"REGION_ROUTE103/EAST": {
+  "parent_map": "MAP_ROUTE103",
+  "locations": [
+    "ITEM_ROUTE_103_GUARD_SPEC",
+    "ITEM_ROUTE_103_PP_UP"
+  ],
+  "events": [],
+  "exits": [
+    "REGION_ROUTE103/WATER",
+    "REGION_ROUTE110/MAIN"
+  ],
+  "warps": [
+    "MAP_ROUTE103:0/MAP_ALTERING_CAVE:0"
+  ]
+}
+```
+
+- `[key]`: The name of the object, in this case `REGION_ROUTE103/EAST`, should be the value of `parent_map` where the
+`MAP` prefix is replaced with `REGION`. Then there should be a following `/` and a label describing this specific region
+within the `Map`. This is not enforced or required by the code, but it makes things much more clear.
+- `parent_map`: The name of the `Map` this region exists under. It can relate this region to information like encounter
+tables.
+- `locations`: Locations contained within this region. This can be anything from an item on the ground to a badge to a
+gift from an NPC. Locations themselves are defined in `data/extracted_data.json`, and the names used here should come
+directly from it.
+- `events`: Events that can be completed in this region. Defeating a gym leader or Aqua/Magma team leader, for example,
+can trigger story progression and unblock roads and buildings. Events are defined here and nowhere else, and access
+rules are set in `rules.py`.
+- `exits`: Names of regions that can be directly accessed from this one. Most often regions within the same `Map`,
+neighboring maps in the overworld, or transitions from using HM08 Dive. Most connections between maps/regions come from
+warps. Any region in this list should be defined somewhere in `data/regions/`.
+- `warps`: Warp events contained within this region. Warps are defined in `data/extracted_data.json`, and must exist
+there to be referenced here. More on warps in [../docs/warps.md](../docs/warps.md).
+
+Think of this data as defining which regions are "claiming" a given location, event, or warp. No more than one region
+may claim ownership of a location. Even if some "thing" may happen in two different regions and set the same flag, they
+should be defined as two different events and anything conditional on said "thing" happening can check whether either of
+the two events is accessible. (e.g. Interacting with the Poke Ball in your rival's room and going back downstairs will
+both trigger a conversation with them which enables you to rescue Professor Birch. It's the same "thing" on two
+different `Map`s.)
+
+Conceptually, you shouldn't have to "add" any new regions. You should only have to "split" existing regions. When you
+split a region, make sure to correctly reassign `locations`, `events`, `exits`, and `warps` according to which new
+region they now exist in. Make sure to define new `exits` to link the new regions to each other if applicable. And
+especially remember to rename incoming `exits` defined in other regions which are still pointing to the pre-split
+region. `sanity_check.py` should catch you if there are other regions that point to a region that no longer exists, but
+if one of your newly-split regions still has the same name as the original, it won't be detected and you may find that
+things aren't connected correctly.

worlds/pokemon_emerald/docs/rom changes.md

@@ -1,7 +1,20 @@
+## New Behaviors
+
+- The union room receptionist on the second floor of Pokemon Centers was reworked for wonder trading via Archipelago
+- Norman will give you all event ticket items when he gives you the S.S. Ticket
+- Use of event tickets is streamlined and the scripts are refactored to skip "first time use" stuff
+- The roaming pokemon is forced to Latios
+- The pokemon at Southern Island is forced to Latias
+- There is new code for changing your party's levels during trainer battles which also modifies exp gain
+
 ## QoL
 
+- The menu has a GO HOME option instead of EXIT, which will immediately teleport you to Birch's Lab
+- It is possible to teach over HM moves
 - The catch tutorial and cutscenes during your first visit to Petalburg are skipped
 - The match call tutorial after you leave Devon Corp is skipped
+- Random match calls in general are skipped, and trainers no longer ask to register you after a battle
+- Searching by type in the pokedex includes species you have seen but not yet caught
 - Cycling and running is allowed in every map (some exceptions like Fortree and Pacifidlog)
 - When you run out of Repel steps, you'll be prompted to use another one if you have more in your bag
 - Text is always rendered in its entirety on the first frame (instant text)
@@ -12,6 +25,9 @@ you can still read the species when deciding whether to change pokemon
 - When receiving TMs and HMs, the move that it teaches is consistently displayed in the "received item" message (by
 default, certain ways of receiving items would only display the TM/HM number)
 - The Pokedex starts in national mode
+- The fishing minigame is always successful at finding a catch, only requires one round, and will always show four dots
+- With an option in Archipelago, spinning trainers become predictable
+- Removed a ledge on Route 123 which allows you to collect every item without backtracking
 - The Oldale Pokemart sells Poke Balls at the start of the game
 - Pauses during battles (e.g. the ~1 second pause at the start of a turn before an opponent uses a potion) are shorter
 by 62.5%
@@ -29,6 +45,10 @@ predetermined priority
 - Shoal cave changes state every time you reload the map and is no longer tied to the RTC
 - Increased safari zone steps from 500 to 50000
 - Trainers will not approach the player if the blind trainers option is set
+- Defeating the elite 4 respawns all legendary encounters where the encounter ended by fainting the pokemon
+- The cutscene revealing the existence of Latios also gives you dex info for having seen Latios
+- The braille wall hinting at the solution to the Wailord/Relicanth puzzle gives you dex info for having seen Wailord
+and Relicanth
 - Changed trade evolutions to be possible without trading:
   - Politoed: Use King's Rock in bag menu
   - Alakazam: Level 37
@@ -47,11 +67,23 @@ predetermined priority
 ## Game State Changes/Softlock Prevention
 
 - Mr. Briney never disappears or stops letting you use his ferry
+- Upon releasing Kyogre, Sootopolis and Sky Pillar will be advanced to after Rayquaza has been awakened, skipping the
+Wallace and Rayquaza fetch quest
 - Prevent the player from flying or surfing until they have received the Pokedex
-- The S.S. Tidal will be available at all times if you have the option enabled
+- The S.S. Tidal will be available at all times
+- All time-based berry gifts are locked to a one-time gift of a specific berry
+- Terra and Marine Cave are given fixed locations, and the weather events revealing them are permanent until the
+legendary encounter is resolved
+- Mirage Island is always present
+- During dexsanity, certain trainers don't disappear/deactivate
+- During berry randomization, it is impossible to plant berries or for berry trees to change state
 - Some NPCs or tiles are removed on the creation of a new save file based on player options
 - Ensured that every species has some damaging move by level 5
-- Route 115 may have strength boulders between the beach and cave entrance based on player options
+- Route 115 has an alternate layout (must be enabled through Archipelago) which includes a bumpy slope that can cross
+the ledge normally blocking you from entering Meteor Falls from Rustboro City
+- Route 115 may have strength boulders (must be enabled through Archipelago) between the beach and cave entrance
+- Route 118 has an alternate layout (must be enabled through Archipelago) that blocks you from surfing between shores
+and adds a rail so that it can be crossed using the Acro Bike
 - The Petalburg Gym is set up based on your player options rather than after the first 4 gyms
 - The E4 guards will actually check all your badges (or gyms beaten based on your options) instead of just the Feather
 Badge

worlds/pokemon_emerald/docs/warps.md

@@ -0,0 +1,50 @@
+## Warps
+
+Quick note to start, you should not be defining or modifying encoded warps from this repository. They're encoded in the
+source code repository for the mod, and then assigned to regions in `data/regions/`. All warps in the game already exist
+within `extracted_data.json`, and all relevant warps are already placed in `data/regions/` (unless they were deleted
+accidentally).
+
+Many warps are actually two or three events acting as one logical warp. Doorways, for example, are often 2 tiles wide
+indoors but only 1 tile wide outdoors. Both indoor warps point to the outdoor warp, and the outdoor warp points to only
+one of the indoor warps. We want to describe warps logically in a way that retains information about individual warp
+events. That way a 2-tile-wide doorway doesnt look like a one-way warp next to an unrelated two-way warp, but if we want
+to randomize the destinations of those warps, we can still get back each individual id of the multi-tile warp.
+
+This is how warps are encoded:
+
+`{source_map}:{source_warp_ids}/{dest_map}:{dest_warp_ids}[!]`
+
+- `source_map`: The map the warp events are located in
+- `source_warp_ids`: The ids of all adjacent warp events in source_map which lead to the same destination (these must be
+in ascending order)
+- `dest_map`: The map of the warp event to which this one is connected
+- `dest_warp_ids`: The ids of the warp events in dest_map
+- `[!]`: If the warp expects to lead to a destination which does not lead back to it, add a ! to the end
+
+Example: `MAP_LAVARIDGE_TOWN_HOUSE:0,1/MAP_LAVARIDGE_TOWN:4`
+
+Example 2: `MAP_AQUA_HIDEOUT_B1F:14/MAP_AQUA_HIDEOUT_B1F:12!`
+
+Note: A warp must have its destination set to another warp event. However, that does not guarantee that the destination
+warp event will warp back to the source.
+
+Note 2: Some warps _only_ act as destinations and cannot actually be interacted with by the player as sources. These are
+usually places you fall from a hole above. At the time of writing, these are actually not accounted for, but there are
+no instances where it changes logical access.
+
+Note 3: Some warp destinations go to the map `MAP_DYNAMIC` and have a special warp id. These edge cases are:
+
+- The Moving Truck
+- Terra Cave
+- Marine Cave
+- The Department Store Elevator
+- Secret Bases
+- The Trade Center
+- The Union Room
+- The Record Corner
+- 2P/4P Battle Colosseum
+
+Note 4: The trick house on Route 110 changes the warp destinations of its entrance and ending room as you progress
+through the puzzles, but the source code only sets the trick house up for the first puzzle, and I assume the destination
+gets overwritten at run time when certain flags are set.

worlds/pokemon_emerald/docs/wonder trades.md

@@ -0,0 +1,103 @@
+# Wonder Trades
+
+Pokemon Emerald uses Archipelago's data storage to reproduce what the Pokemon series calls wonder trading. Wonder
+trading is meant as a sort of gacha game surprise trade where you give up one of your pokemon and at some point in the
+future you'll receive one in return from another player who decided to participate. In practice, small groups will be
+able to use it as a means of simple trading as well by coordinating when they participate.
+
+The goal of the implementation used by Pokemon Emerald is to allow players to interact with an NPC in-game to deposit
+and withdraw pokemon without having to touch their client. The client will automatically detect their state, look for
+available trades, and notify the player when they've received something.
+
+It's also intended to work for Pokemon games other than Emerald, should any other games decide to opt in and implement
+the feature into their clients.
+
+## Data Storage Format
+
+There is one wonder trade entry per team at `pokemon_wonder_trades_{team number}`.
+
+It should be a dict that looks something like this:
+
+```json
+{
+  "_lock": 0,
+  "0": [3, "{some json data}"],
+  "3": [2, "{some json data}"]
+}
+```
+
+### Lock
+
+`_lock` tells you whether you're allowed to try to modify the key. Its value should be either `0` to represent an
+unlocked state, or a timestamp represented by time since Epoch in ms (`int(time.time_ns() / 1000000)`).
+[See below](#preventing-race-conditions) for more info.
+
+### Non-lock Keys
+
+All other keys are just non-negative integers as strings. You can think of them as wonder trade slots. Pidgeon holes
+with a label. For consistency and ease of use, keep the keys between 0 and 255, and prefer the lowest number you can
+use. They ONLY act as names that can be easily written to and removed from.
+- You SHOULD NOT rely on those numbers being contiguous or starting at 0.
+- You SHOULD NOT rely on a "trade" residing at a single slot until it is removed.
+- You SHOULD NOT assume that the number has any significance to a player's slot, or trade order, or anything really.
+
+### Values
+
+The first entry in the tuple represents which slot put the pokemon up for trade. You could use this to display in your
+game or client who the trade came from, but its primary purpose is to discriminate entries you can take from those you
+can't. You don't want to send something to the server, see that the server has something to take, and then take your own
+pokemon right back.
+
+The JSON data should match the schema currently located at `data/trade_pokemon_schema.json`. It should be universally
+understandable by anything trying to interact with wonder trades. Of course, some Pokemon games include more data than
+others for a given pokemon, some games don't have species introduced in later generations, and some data is of a
+different format, has different values, or is even spelled differently. The hope is that translating to and from JSON is
+reasonable for any game (or at least any game likely to be integrated into AP), and you can easily tell from the JSON
+whether your game is capable of giving the pokemon to the player in-game.
+
+## Preventing Race Conditions
+
+This caused by far the most headache of implementing wonder trades. You should be very thorough in trying to prevent
+issues here.
+
+If you prefer more technical explanations, the Pokemon Emerald client has documented wonder trade functions. The rest of
+this section explains what problems are being solved and why the solutions work.
+
+The problem that needs solving is that your client needs to know what the value of the trade data is before it commits
+some sort of action. By design, multiple clients are writing to and removing from the same key in data storage, so if
+two clients try to interact and there's ambiguity in what the data looks like, it will cause issues of duplication and
+loss of data.
+
+For example, client 1 and client 2 both see a pokemon that they can take, so they copy the pokemon to their respective
+games, and both send a command to remove that pokemon from the data store. The first command works and removes the
+entry, which sends an update to both clients that there no longer exists a pokemon at that slot. And then the second
+command, which was already sent, tries to remove the same entry. At best, the data was duplicated, and at worst the
+server raises an exception or crashes.
+
+Thankfully, when you receive an update from the server that a storage value changed, it will tell you both the previous
+and current value. That's where the lock comes in. At a basic level, your client attempts to claim ownership of the key
+temporarily while it makes its modifications, and all other clients respect that claim by not interacting until the lock
+is released. You know you locked the key because the `SetReply` you receive for modifying the lock is the one that set
+it from an unlocked state to a locked state. When two clients try to lock at the same time, one will see an unlocked
+state move to a locked state, and the other will see an already locked state move to a locked state. You can identify
+whether a `SetReply` was triggered by your client's `Set` by attaching a uuid to the `Set` command, which will also be
+attached to the `SetReply`. See the Emerald client for an example.
+
+Which brings us to problem 2, which is the scenario where a client crashes or closes before unlocking the key. One rogue
+client might prevent all other clients from ever interacting with wonder trading again.
+
+So for this reason, the lock is a timestamp, and the key is considered "locked" if that timestamp is less than 5 seconds
+in the past. If a client dies after locking, its lock will expire, and other clients will be able to make modifications.
+Setting the lock to 0 is the canonical way of marking it as unlocked, but it's not a special case really. It's
+equivalent to marking the key as last locked in 1970.
+
+Which brings us to problem 3. Multiple clients which want to obtain the lock can only check whether the lock is
+obtainable by refreshing the current lock's timestamp. So two clients trying to secure a lock made by a dead client may
+trade back and forth, updating the lock to see if it is expired yet, seeing that it is not, and then waiting 5 seconds
+while the other client does the same thing, which causes the lock to again be less than 5 seconds old.
+
+Using a cooldown period longer than the time to expire only increases the minimum number of clients that can trigger
+this cycle. Instead, the solution is to double your cooldown every time you bounce off an expired lock (and reset it
+once you acquire it). Eventually the amount of time every client is waiting will be enough to create a gap large enough
+for one client to consider the lock expired, and it will acquire the lock, make its changes, and set the lock state to
+definitively unlocked, which will let the next client claim it, and so on.