Record phase 0 forks, phase 1 completion and phase 2 server fork in the plan
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
+69
-8
@@ -138,8 +138,9 @@ These checkouts are in `~/src`. Run `git pull` before relying on them.
|
|||||||
unreachable**: `beatleader-server-private` and `bs-simulator`. Upstream
|
unreachable**: `beatleader-server-private` and `bs-simulator`. Upstream
|
||||||
already copes with that. `Startup.cs` falls back to `ReplayRecalculatorStub`,
|
already copes with that. `Startup.cs` falls back to `ReplayRecalculatorStub`,
|
||||||
and nothing in the public code imports a namespace that only the private
|
and nothing in the public code imports a namespace that only the private
|
||||||
repos define. **I never compiled it**: an agent build of third-party code was
|
repos define. **Verified 2026-11-08:** it builds on Linux with nixpkgs
|
||||||
blocked in that session. Building on Linux is the first task in phase 2.
|
`dotnet-sdk_10` (10.0.403) with both private folders empty: 0 errors,
|
||||||
|
about 2,300 warnings, about 1 minute.
|
||||||
- Database: `UseSqlServer` for two contexts (`Startup.cs:380-386`):
|
- Database: `UseSqlServer` for two contexts (`Startup.cs:380-386`):
|
||||||
`AppContext` (`DefaultConnection`) and `StorageContext`
|
`AppContext` (`DefaultConnection`) and `StorageContext`
|
||||||
(`StorageConnection`, which holds `PlayerLeaderboardStats`). There are 570
|
(`StorageConnection`, which holds `PlayerLeaderboardStats`). There are 570
|
||||||
@@ -225,8 +226,11 @@ plebsaber.stream/ (this repo)
|
|||||||
src/lib/server/beatleader.ts extend: bearer auth from the sync's token store, paging, time_from, raw passthrough
|
src/lib/server/beatleader.ts extend: bearer auth from the sync's token store, paging, time_from, raw passthrough
|
||||||
src/lib/server/scoresaber.ts extend: player scores, replay download
|
src/lib/server/scoresaber.ts extend: player scores, replay download
|
||||||
|
|
||||||
gitea pleb/beatleader-server fork, branch `pleb/main`
|
gitea pleb/beatleader-server fork, branch `pleb/main` (based on upstream `rebalanced-pp`)
|
||||||
|
MIRROR.md what MirrorMode changes and how to run it
|
||||||
|
appsettings.Mirror.json incineroar values (ASPNETCORE_ENVIRONMENT=Mirror)
|
||||||
Tools/ArchiveImporter/ C# console app sharing AppContext/StorageContext
|
Tools/ArchiveImporter/ C# console app sharing AppContext/StorageContext
|
||||||
|
gitea pleb/beatleader-server-models fork of the models submodule, branch `pleb/main`
|
||||||
gitea pleb/beatleader-website fork, branch `pleb/main`
|
gitea pleb/beatleader-website fork, branch `pleb/main`
|
||||||
|
|
||||||
/var/lib/plebleader/ on incineroar, user `plebleader`
|
/var/lib/plebleader/ on incineroar, user `plebleader`
|
||||||
@@ -264,14 +268,24 @@ the fork, so do it first.
|
|||||||
|
|
||||||
### Phase 0: Prep
|
### Phase 0: Prep
|
||||||
|
|
||||||
1. Update the checkouts:
|
1. **Done 2026-11-08** (all were already current; upstream server `master`
|
||||||
|
`1474312`, `rebalanced-pp` `52a50ed`, website `2b3ef891`, ArcViewer `deploy`
|
||||||
|
`c776256`, ScoreSaber website `836d789`). Update the checkouts:
|
||||||
`git -C ~/src/BeatLeader/beatleader-server pull --recurse-submodules`, and
|
`git -C ~/src/BeatLeader/beatleader-server pull --recurse-submodules`, and
|
||||||
the same for `beatleader-website`, `~/src/AllPoland/ArcViewer` (also
|
the same for `beatleader-website`, `~/src/AllPoland/ArcViewer` (also
|
||||||
`git fetch origin deploy`), and `~/src/ScoreSaber/website`. The ScoreSaber
|
`git fetch origin deploy`), and `~/src/ScoreSaber/website`. The ScoreSaber
|
||||||
website is public now; use it to check the score and replay API shapes.
|
website is public now; use it to check the score and replay API shapes.
|
||||||
2. Create the Gitea forks `pleb/beatleader-server` and `pleb/beatleader-website`
|
2. **Done 2026-11-08.** Create the Gitea forks `pleb/beatleader-server` and `pleb/beatleader-website`
|
||||||
with the `gitea-api` skill (mirror or push). Keep `upstream` as a remote so
|
with the `gitea-api` skill (mirror or push). Keep `upstream` as a remote so
|
||||||
you can rebase later.
|
you can rebase later.
|
||||||
|
Created as private, non-mirror imports through `POST /repos/migrate` (the
|
||||||
|
Gitea token lacks `write:user`, which `POST /user/repos` needs), with
|
||||||
|
default branch `pleb/main`. The `~/src` checkouts keep `origin` = GitHub and
|
||||||
|
gained a `gitea` remote; `pleb/main` is checked out there. The server's
|
||||||
|
`pleb/main` starts from `rebalanced-pp`, which is `master` plus 4 commits
|
||||||
|
and what the live API most likely runs, so the entities match the archived
|
||||||
|
responses. `pleb/beatleader-server-models` was forked too (see phase 2,
|
||||||
|
step 7); the server's `.gitmodules` points at it over SSH.
|
||||||
3. Bring this repo's flake up to date (it's still on `nixos-25.05`) only if the
|
3. Bring this repo's flake up to date (it's still on `nixos-25.05`) only if the
|
||||||
exporter gets packaged here.
|
exporter gets packaged here.
|
||||||
4. Ask the user to add a sops secret `plebleader.env` for incineroar, holding
|
4. Ask the user to add a sops secret `plebleader.env` for incineroar, holding
|
||||||
@@ -386,21 +400,26 @@ leaderboard snapshot, plus a BeatSaver zip or recovered game folder, unless
|
|||||||
it's listed in `missing.json`. Write a `report` subcommand that prints counts
|
it's listed in `missing.json`. Write a `report` subcommand that prints counts
|
||||||
and what's missing, and paste its output into this file.
|
and what's missing, and paste its output into this file.
|
||||||
|
|
||||||
**Report, 2026-11-08** (ScoreSaber main replays still downloading):
|
**Report, 2026-11-08 (phase 1 complete).** The ScoreSaber replays finished
|
||||||
|
with zero 429s.
|
||||||
|
|
||||||
```
|
```
|
||||||
Window: scores since all time, BeatSaver maps since 2024-11-08
|
Window: scores since all time, BeatSaver maps since 2024-11-08
|
||||||
BeatLeader scores 3568 in window (3568 archived total, both accounts)
|
BeatLeader scores 3568 in window (3568 archived total, both accounts)
|
||||||
BeatLeader replays 3568/3568
|
BeatLeader replays 3568/3568
|
||||||
BeatLeader score stats 3568/3568
|
BeatLeader score stats 3568/3568
|
||||||
ScoreSaber 76561199407393962: 2107 scores, replays 435/1034
|
ScoreSaber 76561199407393962: 2107 scores, replays 1034/1034
|
||||||
ScoreSaber 76561197962107242: 199 scores, replays 58/58
|
ScoreSaber 76561197962107242: 199 scores, replays 58/58
|
||||||
Played hashes 2560
|
Played hashes 2560
|
||||||
BL leaderboard snapshot 2552/2560
|
BL leaderboard snapshot 2552/2560
|
||||||
BeatSaver zip 1897/1926 in maps window (missing.json lists 29)
|
BeatSaver zip 1897/1926 in maps window (missing.json lists 29)
|
||||||
Archive size 21.15 GB
|
Archive size 21.78 GB
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Steps 8 (game-folder recovery), 9 (unpack into `maps/`) and 13 (local mod
|
||||||
|
replays) are still open. Step 9 is needed before phase 4; it can happen when
|
||||||
|
the archive moves to `/var/lib/plebleader`.
|
||||||
|
|
||||||
The 29 missing hashes are all "hash unknown to BeatSaver", and none belongs to
|
The 29 missing hashes are all "hash unknown to BeatSaver", and none belongs to
|
||||||
a legacy score with a replay (all 53 of those maps are archived). The 8 hashes
|
a legacy score with a replay (all 53 of those maps are archived). The 8 hashes
|
||||||
without a BL leaderboard are ScoreSaber-only maps BeatLeader never saw. Zips
|
without a BL leaderboard are ScoreSaber-only maps BeatLeader never saw. Zips
|
||||||
@@ -408,6 +427,48 @@ average about 8 MB (15 GB for 1,897), more than the estimate below.
|
|||||||
|
|
||||||
### Phase 2: Server fork on SQLite
|
### Phase 2: Server fork on SQLite
|
||||||
|
|
||||||
|
**Done 2026-11-08** (`pleb/main` at `1741ce9`, models fork at `8f233e8`).
|
||||||
|
What differs from the steps below:
|
||||||
|
|
||||||
|
- Step 2: `Microsoft.Data.SqlClient` stays as a direct package, because one
|
||||||
|
controller catches `SqlException`. One controller had an unused
|
||||||
|
`using BeatLeader_Server.Migrations;` that broke the build once the old
|
||||||
|
migrations were gone. The fresh migrations create 155 tables and 217
|
||||||
|
indexes. WAL and migrations are applied at startup by `Utils/MirrorSetup.cs`;
|
||||||
|
`busy_timeout` comes from `Default Timeout=30` in the connection string.
|
||||||
|
- Step 3: `Utils/BulkCompat.cs` sits in the global namespace, like Z's own
|
||||||
|
extensions, because some callers have no EF `using`. 49 of the `BulkUpdate`
|
||||||
|
calls pass `ColumnInputExpression`, often on detached stubs that carry only
|
||||||
|
the key and those columns, so the shim marks only the listed columns
|
||||||
|
modified and detaches the stubs after saving. A plain `UpdateRange` would
|
||||||
|
have wiped every other column. **Not exercised at runtime yet**; the
|
||||||
|
importer will be the first real user.
|
||||||
|
- Step 4: the two index-defragmentation scripts became `PRAGMA optimize` and
|
||||||
|
`REINDEX; ANALYZE;`.
|
||||||
|
- Step 5: `GetS3Client` returns a `LocalStorageClient` (an `AmazonS3Client`
|
||||||
|
subclass) when `StoragePath` is set; the six S3 primitives branch on it.
|
||||||
|
Upstream behaviour is unchanged without `StoragePath`.
|
||||||
|
- Step 6: `MirrorMode` also skips `RefreshTaskService` and `MapRefresh`, and
|
||||||
|
the external OAuth providers. Two surprises showed up in testing:
|
||||||
|
`GetLazy` fetched unknown players from Steam (it now returns null, so 404),
|
||||||
|
and the BeatSaver login is registered unconditionally and validated on
|
||||||
|
every request (it gets placeholder credentials, otherwise every request
|
||||||
|
was a 500).
|
||||||
|
- Step 7: instead of patching about 35 `showRatings` decisions, the four
|
||||||
|
`HideRatings()` leaves return early when `MirrorSettings.ShowAllRatings`
|
||||||
|
(in the models fork) is set; `MirrorSetup` sets it. Sorting and filtering by
|
||||||
|
stars on unranked maps still follow the requesting user's settings; patch
|
||||||
|
`Utils/{Map,Leaderboard}ListUtils/Sort.cs` if that matters.
|
||||||
|
- Step 8: `appsettings.Mirror.json` loads with `ASPNETCORE_ENVIRONMENT=Mirror`;
|
||||||
|
`MIRROR.md` in the fork documents it. OpenIddict uses ephemeral keys.
|
||||||
|
Run the DLL with `--contentRoot <bin>` from a working directory in the state
|
||||||
|
dir, because data protection keys go to `../keys/`. `dotnet run` ignores
|
||||||
|
`ASPNETCORE_URLS` in favour of `launchSettings.json`.
|
||||||
|
|
||||||
|
Check result: on empty files, `/swagger/index.html` 200,
|
||||||
|
`/player/76561199407393962` 404, `/players` and `/leaderboards` 200 with empty
|
||||||
|
pages, no `fail:` lines in the log.
|
||||||
|
|
||||||
Work on the `pleb/main` branch of `pleb/beatleader-server`. Keep each change
|
Work on the `pleb/main` branch of `pleb/beatleader-server`. Keep each change
|
||||||
small and in its own commit, so upstream rebases stay manageable.
|
small and in its own commit, so upstream rebases stay manageable.
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user