Core, WebHost: lazy-load worlds in unpickler, WebHost and WebHostLib (#2156)

* Core: lazy-load worlds in unpickler

this should hopefully fix customserver's memory consumption

* WebHost: move imports around to save memory in MP

* MultiServer: prefer loading _speedups without pyximport

This saves ~15MB per MP and speeds up module import if it was built in-place.

* Tests: fix tests for changed WebHost imports

* CustomServer: run GC after setup

* CustomServer: cleanup exception handling
This commit is contained in:
black-sliver
2023-09-20 16:05:56 +02:00
committed by GitHub
parent 4a27fae1ab
commit d471dcc067
8 changed files with 93 additions and 75 deletions

View File

@@ -407,14 +407,22 @@ class _LocationStore(dict, typing.MutableMapping[int, typing.Dict[int, typing.Tu
if typing.TYPE_CHECKING: # type-check with pure python implementation until we have a typing stub
LocationStore = _LocationStore
else:
try:
import pyximport
pyximport.install()
except ImportError:
pyximport = None
try:
from _speedups import LocationStore
import _speedups
import os.path
if os.path.getctime(_speedups.__file__) < os.path.getctime("_speedups.pyx"):
warnings.warn(f"{_speedups.__file__} outdated! "
f"Please rebuild with `cythonize -b -i _speedups.pyx` or delete it!")
except ImportError:
warnings.warn("_speedups not available. Falling back to pure python LocationStore. "
"Install a matching C++ compiler for your platform to compile _speedups.")
LocationStore = _LocationStore
try:
import pyximport
pyximport.install()
except ImportError:
pyximport = None
try:
from _speedups import LocationStore
except ImportError:
warnings.warn("_speedups not available. Falling back to pure python LocationStore. "
"Install a matching C++ compiler for your platform to compile _speedups.")
LocationStore = _LocationStore