Record phase 3 importer and fetch full player history
The importer needs every day of /history for the triangle's monthly points; the API returns only 50 by default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
+49
-1
@@ -543,6 +543,54 @@ loads, and `GET /player/76561199407393962` returns 404 rather than 500.
|
|||||||
|
|
||||||
### Phase 3: Importer
|
### Phase 3: Importer
|
||||||
|
|
||||||
|
**Importer done 2026-11-09** (`pleb/main` at `737575b`, not pushed yet). Usage
|
||||||
|
and the full file → table mapping are in the fork's `MIRROR.md`. A full import
|
||||||
|
takes about 11 s, and an unchanged rerun writes 0 rows. What differs from the
|
||||||
|
steps below:
|
||||||
|
|
||||||
|
- The archived responses are deserialized straight into the entity types,
|
||||||
|
because their property names match. Only scalar columns are copied onto the
|
||||||
|
tracked rows; keys, foreign keys and navigations are set explicitly. Leaderboard
|
||||||
|
snapshots are the main source for `Song`/`DifficultyDescription`. A score's
|
||||||
|
embedded leaderboard is a fallback only, and no score needed it.
|
||||||
|
- One-to-one children (`ScoreImprovement`, `ReplayOffsets`, `ModifiersRating`,
|
||||||
|
`DifficultyStatistics`, `ModifiersMap`, stats, settings) get local ids, so
|
||||||
|
they can't collide with upstream ids. Scores, difficulties, songs,
|
||||||
|
leaderboards, mappers and history rows keep upstream ids.
|
||||||
|
- `PlayerScoreStatsHistory` lives in **`StorageContext`**, so that database
|
||||||
|
isn't empty. The exporter had saved only 50 days of `/history` (the API
|
||||||
|
default); it now asks for `count=100000` (1,138 days for pleb). The triangle
|
||||||
|
needs those monthly points.
|
||||||
|
- No legacy `Player` row. An `AccountLink` (`PCOculusID` = legacy id) plus a
|
||||||
|
`// pleb:` patch in `PlayerIdToMain` resolves `/player/76561197962107242`
|
||||||
|
to pleb. Upstream only checks Steam-range ids against Oculus links. The
|
||||||
|
legacy account's 3 BL scores are skipped until phase 5.
|
||||||
|
- `StoragePath/replays` is a symlink to the archive's `beatleader/replays`.
|
||||||
|
Score stats are copied to `StoragePath/scorestats/<scoreId>.json`.
|
||||||
|
- Clans aren't mirrored (`ClanOrder` is blank).
|
||||||
|
|
||||||
|
Imported: 2,552 songs, 10,104 difficulties, 10,098 leaderboards, 628 mappers,
|
||||||
|
3,565 scores, 3,568 score stats, 1,138 history rows. API check against the
|
||||||
|
archived upstream responses, on a scratch DB with `ASPNETCORE_ENVIRONMENT=Mirror`:
|
||||||
|
|
||||||
|
- `/player/{id}?stats=true` matches. The only differences are row ids,
|
||||||
|
clans, and `linkedIds`, which now lists the alt.
|
||||||
|
- `/player/{id}/scores?includeIO=true`: 3,565 total. Scores, improvements and
|
||||||
|
offsets match, and unranked difficulties now carry stars and the skill
|
||||||
|
triangle ratings.
|
||||||
|
- `/accgraph` and `/history/triangle` match. `/score/statistic/{id}` redirects
|
||||||
|
to the local file.
|
||||||
|
- **Open: `/leaderboard/{id}` and `/leaderboard/scores/{id}` return 500.**
|
||||||
|
`LeaderboardController.GeneralScores` projects
|
||||||
|
`Player.Clans.OrderBy(…ClanOrder…).Take(1)`, which needs SQL `APPLY`, and
|
||||||
|
SQLite has none. The same pattern appears about 8 times, in
|
||||||
|
`ScoreController`, `ReplayController`, `LeaderboardController` and
|
||||||
|
`ClanUtils`. Fix it in a separate server commit before phase 4. For example,
|
||||||
|
skip the clan projection in MirrorMode (there are no clans), or move the
|
||||||
|
ordering client-side.
|
||||||
|
|
||||||
|
The check below (stock website) is still to do in phase 4.
|
||||||
|
|
||||||
`Tools/ArchiveImporter` lives in the fork, references the main project, and
|
`Tools/ArchiveImporter` lives in the fork, references the main project, and
|
||||||
uses its `AppContext`/`StorageContext`. Run it as a one-shot that is idempotent
|
uses its `AppContext`/`StorageContext`. Run it as a one-shot that is idempotent
|
||||||
and upserts by upstream id.
|
and upserts by upstream id.
|
||||||
@@ -563,7 +611,7 @@ and upserts by upstream id.
|
|||||||
an `AccountLink` or alt pointing to the main player, so `PlayerIdToMain`
|
an `AccountLink` or alt pointing to the main player, so `PlayerIdToMain`
|
||||||
resolves `/u/76561197962107242` to `pleb`. Its 3 BL scores go through the
|
resolves `/u/76561197962107242` to `pleb`. Its 3 BL scores go through the
|
||||||
phase 5 merge like any other candidate.
|
phase 5 merge like any other candidate.
|
||||||
- `StorageContext` (attempts) stays empty, because attempts aren't exported.
|
- `StorageContext` holds `PlayerScoreStatsHistory`; its attempts tables stay empty, because attempts aren't exported.
|
||||||
- `scorestats/*.json` → the `scorestats` storage container, using the filename
|
- `scorestats/*.json` → the `scorestats` storage container, using the filename
|
||||||
the server expects (check `ScoreController.GetStatistic`).
|
the server expects (check `ScoreController.GetStatistic`).
|
||||||
|
|
||||||
|
|||||||
@@ -26,7 +26,9 @@ export async function exportProfile(playerId: string): Promise<void> {
|
|||||||
const dir = (kind: string) => bl('players', playerId, kind);
|
const dir = (kind: string) => bl('players', playerId, kind);
|
||||||
const endpoints: [string, string][] = [
|
const endpoints: [string, string][] = [
|
||||||
['profile', `/player/${playerId}?stats=true`],
|
['profile', `/player/${playerId}?stats=true`],
|
||||||
['history', `/player/${playerId}/history`],
|
// count: days of history; the default is 50, which loses the triangle's
|
||||||
|
// monthly points. Upstream keeps one row per day since the first score.
|
||||||
|
['history', `/player/${playerId}/history?count=100000`],
|
||||||
['triangle', `/player/${playerId}/history/triangle`],
|
['triangle', `/player/${playerId}/history/triangle`],
|
||||||
['accgraph', `/player/${playerId}/accgraph`]
|
['accgraph', `/player/${playerId}/accgraph`]
|
||||||
];
|
];
|
||||||
|
|||||||
Reference in New Issue
Block a user