Record phase 3 importer and fetch full player history
The importer needs every day of /history for the triangle's monthly points; the API returns only 50 by default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
+49
-1
@@ -543,6 +543,54 @@ loads, and `GET /player/76561199407393962` returns 404 rather than 500.
|
||||
|
||||
### Phase 3: Importer
|
||||
|
||||
**Importer done 2026-11-09** (`pleb/main` at `737575b`, not pushed yet). Usage
|
||||
and the full file → table mapping are in the fork's `MIRROR.md`. A full import
|
||||
takes about 11 s, and an unchanged rerun writes 0 rows. What differs from the
|
||||
steps below:
|
||||
|
||||
- The archived responses are deserialized straight into the entity types,
|
||||
because their property names match. Only scalar columns are copied onto the
|
||||
tracked rows; keys, foreign keys and navigations are set explicitly. Leaderboard
|
||||
snapshots are the main source for `Song`/`DifficultyDescription`. A score's
|
||||
embedded leaderboard is a fallback only, and no score needed it.
|
||||
- One-to-one children (`ScoreImprovement`, `ReplayOffsets`, `ModifiersRating`,
|
||||
`DifficultyStatistics`, `ModifiersMap`, stats, settings) get local ids, so
|
||||
they can't collide with upstream ids. Scores, difficulties, songs,
|
||||
leaderboards, mappers and history rows keep upstream ids.
|
||||
- `PlayerScoreStatsHistory` lives in **`StorageContext`**, so that database
|
||||
isn't empty. The exporter had saved only 50 days of `/history` (the API
|
||||
default); it now asks for `count=100000` (1,138 days for pleb). The triangle
|
||||
needs those monthly points.
|
||||
- No legacy `Player` row. An `AccountLink` (`PCOculusID` = legacy id) plus a
|
||||
`// pleb:` patch in `PlayerIdToMain` resolves `/player/76561197962107242`
|
||||
to pleb. Upstream only checks Steam-range ids against Oculus links. The
|
||||
legacy account's 3 BL scores are skipped until phase 5.
|
||||
- `StoragePath/replays` is a symlink to the archive's `beatleader/replays`.
|
||||
Score stats are copied to `StoragePath/scorestats/<scoreId>.json`.
|
||||
- Clans aren't mirrored (`ClanOrder` is blank).
|
||||
|
||||
Imported: 2,552 songs, 10,104 difficulties, 10,098 leaderboards, 628 mappers,
|
||||
3,565 scores, 3,568 score stats, 1,138 history rows. API check against the
|
||||
archived upstream responses, on a scratch DB with `ASPNETCORE_ENVIRONMENT=Mirror`:
|
||||
|
||||
- `/player/{id}?stats=true` matches. The only differences are row ids,
|
||||
clans, and `linkedIds`, which now lists the alt.
|
||||
- `/player/{id}/scores?includeIO=true`: 3,565 total. Scores, improvements and
|
||||
offsets match, and unranked difficulties now carry stars and the skill
|
||||
triangle ratings.
|
||||
- `/accgraph` and `/history/triangle` match. `/score/statistic/{id}` redirects
|
||||
to the local file.
|
||||
- **Open: `/leaderboard/{id}` and `/leaderboard/scores/{id}` return 500.**
|
||||
`LeaderboardController.GeneralScores` projects
|
||||
`Player.Clans.OrderBy(…ClanOrder…).Take(1)`, which needs SQL `APPLY`, and
|
||||
SQLite has none. The same pattern appears about 8 times, in
|
||||
`ScoreController`, `ReplayController`, `LeaderboardController` and
|
||||
`ClanUtils`. Fix it in a separate server commit before phase 4. For example,
|
||||
skip the clan projection in MirrorMode (there are no clans), or move the
|
||||
ordering client-side.
|
||||
|
||||
The check below (stock website) is still to do in phase 4.
|
||||
|
||||
`Tools/ArchiveImporter` lives in the fork, references the main project, and
|
||||
uses its `AppContext`/`StorageContext`. Run it as a one-shot that is idempotent
|
||||
and upserts by upstream id.
|
||||
@@ -563,7 +611,7 @@ and upserts by upstream id.
|
||||
an `AccountLink` or alt pointing to the main player, so `PlayerIdToMain`
|
||||
resolves `/u/76561197962107242` to `pleb`. Its 3 BL scores go through the
|
||||
phase 5 merge like any other candidate.
|
||||
- `StorageContext` (attempts) stays empty, because attempts aren't exported.
|
||||
- `StorageContext` holds `PlayerScoreStatsHistory`; its attempts tables stay empty, because attempts aren't exported.
|
||||
- `scorestats/*.json` → the `scorestats` storage container, using the filename
|
||||
the server expects (check `ScoreController.GetStatistic`).
|
||||
|
||||
|
||||
@@ -26,7 +26,9 @@ export async function exportProfile(playerId: string): Promise<void> {
|
||||
const dir = (kind: string) => bl('players', playerId, kind);
|
||||
const endpoints: [string, string][] = [
|
||||
['profile', `/player/${playerId}?stats=true`],
|
||||
['history', `/player/${playerId}/history`],
|
||||
// count: days of history; the default is 50, which loses the triangle's
|
||||
// monthly points. Upstream keeps one row per day since the first score.
|
||||
['history', `/player/${playerId}/history?count=100000`],
|
||||
['triangle', `/player/${playerId}/history/triangle`],
|
||||
['accgraph', `/player/${playerId}/accgraph`]
|
||||
];
|
||||
|
||||
Reference in New Issue
Block a user