mirror of
https://github.com/civitai/civitai.git
synced 2026-09-20 22:08:18 +08:00
fix(training): drop the version name from epoch filenames, keep the id
The review left this open as a decision: the ask was `esadribicstyle_krea2`, what shipped was `esadribicstyle_krea2_v1-1284593_epoch_10.safetensors`. Settled as keep-the-id, drop-the-name: esadribicstyle_krea2_1284593_epoch_10.safetensors Dropping the scope entirely was rejected. Cumulative epoch numbering only separates a CONTINUATION from its parent — a fresh retrain of one model on the same base gets no offset, restarts at epoch 1, and reproduces the exact collision this feature exists to fix. That case is not exotic; it is what iterating on settings looks like. The version NAME went because it carried nothing the epoch number didn't. For continuations it is auto-generated as `V2 (from epoch 5)` and sanitizes to `V2__from_epoch_5`, which was the bulk of the ugliness and none of the uniqueness — versionId is globally unique on its own. Nothing persists these names: a `link.download` attribute, a Content-Disposition header, and a transient zip manifest, all generated per request. No migration, no backfill, and already-downloaded files keep the names they were given. `versionName` leaves TrainingRunNameParts entirely, so the two Prisma selects that widened to `name: true` only to feed it are narrowed back, along with the two fixtures that were still supplying it. Three tests went with it rather than being rewritten — they existed to prove the version-name component was present, truncation-safe, and collision-free, and all three are vacuous once the component is gone. Suite drops 21526 -> 21523 for exactly that reason. A fourth survives under a new name and is what now pins the retrain case. The tracker records this under "Closed", not "Answered from the repo" — the repo could not settle it, and a heading saying otherwise is how a decision loses its owner. The feedback doc points at the tracker for the reasoning rather than restating it, so the two cannot drift apart. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
briant
@@ -112,8 +112,9 @@ All three came out of the **Creator Studio article (33297) comments, not the Dis
|
||||
- [ ] **Option to hide Blue Buzz from the top-right balance** Blue buzz noise obscures sales performance. Default stays combined; opt-in yellow-only display, both still shown in the dropdown.
|
||||
- [ ] **Mute/unfollow a notification thread** a maintainer, 8/1 3:36 PM, alongside the dedup work.
|
||||
- [x] **Architecture-name suffix on trained model filenames** Multi-architecture training produces identical filenames; wants `esadribicstyle_krea2` / `esadribicstyle_pdxl`. Prefers a shared standard, default-on, and **sticky** so it isn't re-enabled every run.
|
||||
- **Shipped** — every epoch download, sample and run zip carries the architecture: `<model>_<architecture>_<version>-<versionId>_epoch_<n>.safetensors`, from the one helper in `src/shared/utils/training-file-names.ts`. Runs predating the field omit the segment.
|
||||
- **No toggle, so "sticky" does not apply** — it is unconditional, which is the shared standard he asked for one rung further. The segment sits between model and version rather than trailing, because the version scope has to stay adjacent to the epoch number it disambiguates.
|
||||
- **Shipped** — every epoch download, sample and run zip carries the architecture: `<model>_<architecture>_<versionId>_epoch_<n>.safetensors`, from the one helper in `src/shared/utils/training-file-names.ts`. Runs predating the field omit the segment.
|
||||
- **No toggle, so "sticky" does not apply** — it is unconditional, which is the shared standard he asked for one rung further.
|
||||
- The `<versionId>` segment is the one part he did not ask for; the version *name* that sat beside it was dropped 2026-08-24. Why the id stays and the name went: [docs/training-filenames-review-followups.md](training-filenames-review-followups.md), "The version scope cost the requested shape".
|
||||
- The segment is the specific base-model key (`pdxl`, `krea2`), not the family: `baseModelType` reports `sdxl` for pony, illustrious and sdxl alike, which cannot tell a multi-training batch apart.
|
||||
- Scope is the pre-publish artifacts. The **published** `ModelFile` still takes the blob id as its stored `name` (`moveAssetFromBlob` → `<blobId>.safetensors`) — unique, so it never collided, but opaque. What downloaders see is already overridable: `overrideName` shipped in `e5573c88f6` (#2737), editable on the model-wizard version step and in the file list, and `getDownloadFilename()` prefers it over `name`. No architecture segment on that path, which is the only piece of this ask still missing there.
|
||||
- [ ] **Customizable precision list** (hide unused formats) He flagged this himself as possibly over-scoped.
|
||||
|
||||
@@ -7,8 +7,9 @@ Perf came back clean and is not listed: the two widened `trainingDetails` select
|
||||
4 buffers, the training webhook path gained zero I/O, and the new client modules added no bundle
|
||||
weight (`~/utils/training` was already a value import in that chunk via `training.store.ts`).
|
||||
|
||||
**Status:** everything answerable from the repo is done. What remains needs a decision from a
|
||||
person — the three under Open questions, plus the dead-v1-auto-label call at the bottom.
|
||||
**Status:** everything answerable from the repo is done, and the filename shape was settled on
|
||||
2026-08-24. What remains needs a decision from a person — the two under Open questions, plus the
|
||||
dead-v1-auto-label call at the bottom.
|
||||
|
||||
## Defects
|
||||
|
||||
@@ -71,7 +72,7 @@ person — the three under Open questions, plus the dead-v1-auto-label call at t
|
||||
of MyLoRA v2", matching the existing render, and falls back to the bare epoch when the run
|
||||
recorded no name.
|
||||
|
||||
## Answered from the repo — no longer open
|
||||
## Closed — no longer open
|
||||
|
||||
- [x] **"The settings used" is 10 of ~24 fields; LoRA Type is the notable omission.** LoRA Type
|
||||
cannot discriminate anything: `loraTypes` in `src/utils/training.ts` has exactly one member,
|
||||
@@ -87,14 +88,21 @@ person — the three under Open questions, plus the dead-v1-auto-label call at t
|
||||
prefers it. It carries no architecture segment; that is the only remaining gap on that path.
|
||||
Record: [docs/features/training-file-rename.md](features/training-file-rename.md).
|
||||
|
||||
- [x] **The version scope cost the requested shape.** Settled 2026-08-24: keep the id, drop the
|
||||
version name. `esadribicstyle_krea2_v1-1284593_epoch_10.safetensors` →
|
||||
`esadribicstyle_krea2_1284593_epoch_10.safetensors`. Dropping the scope entirely was rejected:
|
||||
cumulative numbering only separates a *continuation* from its parent, so a fresh retrain of one
|
||||
model on the same base still restarts at epoch 1 and reproduces the collision the feature
|
||||
exists to fix. The version name went because it carried nothing — for continuations it is
|
||||
auto-generated as `V2 (from epoch 5)` and sanitizes to `V2__from_epoch_5`. Nothing persists
|
||||
these names (a `link.download`, a `Content-Disposition`, a transient zip manifest), so the
|
||||
change needed no migration and stays reversible. Three tests that existed only to protect the
|
||||
version-name component were deleted rather than rewritten. A fourth, "distinguishes the same
|
||||
epoch number across two runs of one model", survives under a new name — it is the one that now
|
||||
pins the retrain case.
|
||||
|
||||
## Open questions — need a person, not the repo
|
||||
|
||||
- [ ] **The version scope serves neither ticket.** Cumulative numbering alone removes the
|
||||
continuation collision; the architecture segment alone separates a multi-training batch. The
|
||||
`v1-1284593` component addresses a third, unreported collision — two same-architecture runs on
|
||||
one model — and costs the requested shape: the ask was `esadribicstyle_krea2`, what ships is
|
||||
`esadribicstyle_krea2_v1-1284593_epoch_10.safetensors`. Decide whether to keep it, keep only
|
||||
the id, or drop it.
|
||||
- [ ] **The Mage-Flow ticket's symptom is unchanged.** The diagnostics are the correct app-side
|
||||
response to an orchestrator-side cause, but nothing user-visible moved, and there is no record
|
||||
in this repo that the orchestrator-side work was filed. The ticket now reads as addressed.
|
||||
|
||||
@@ -87,7 +87,6 @@ const EpochRow = ({
|
||||
canGenerate,
|
||||
isVideo,
|
||||
modelName,
|
||||
versionName,
|
||||
architecture,
|
||||
}: {
|
||||
epoch: TrainingResultsV2['epochs'][number];
|
||||
@@ -108,7 +107,6 @@ const EpochRow = ({
|
||||
canGenerate?: boolean;
|
||||
isVideo: boolean;
|
||||
modelName: string;
|
||||
versionName: string;
|
||||
architecture?: string | null;
|
||||
}) => {
|
||||
const currentUser = useCurrentUser();
|
||||
@@ -123,7 +121,6 @@ const EpochRow = ({
|
||||
link.href = `/api/download/training/${modelVersionId}?epochNumber=${epoch.epochNumber}`;
|
||||
link.download = trainingEpochModelFileName({
|
||||
modelName,
|
||||
versionName,
|
||||
versionId: modelVersionId,
|
||||
architecture,
|
||||
epochNumber: epoch.epochNumber,
|
||||
@@ -986,7 +983,6 @@ export default function TrainingSelectFile({
|
||||
canGenerate={features.privateModels && !!modelVersion.id && canGenerateWithEpochBool}
|
||||
isVideo={isVideo}
|
||||
modelName={model.name}
|
||||
versionName={modelVersion.name}
|
||||
architecture={architecture}
|
||||
/>
|
||||
{epochs.length > 1 && (
|
||||
@@ -1018,7 +1014,6 @@ export default function TrainingSelectFile({
|
||||
}
|
||||
isVideo={isVideo}
|
||||
modelName={model.name}
|
||||
versionName={modelVersion.name}
|
||||
architecture={architecture}
|
||||
/>
|
||||
))}
|
||||
|
||||
@@ -41,7 +41,6 @@ export default AuthedEndpoint(
|
||||
where: { id: modelVersionId },
|
||||
select: {
|
||||
id: true,
|
||||
name: true,
|
||||
trainingDetails: true,
|
||||
model: { select: { id: true, userId: true, name: true } },
|
||||
files: {
|
||||
@@ -138,7 +137,6 @@ export default AuthedEndpoint(
|
||||
|
||||
const fileName = trainingEpochModelFileName({
|
||||
modelName: modelVersion.model.name,
|
||||
versionName: modelVersion.name,
|
||||
versionId: modelVersion.id,
|
||||
architecture: trainingArchitectureKey(
|
||||
modelVersion.trainingDetails as TrainingDetailsObj | null
|
||||
|
||||
@@ -19,7 +19,6 @@ const EPOCH_URL = 'https://orchestration.civitai.com/v2/consumer/blobs/MODEL3.sa
|
||||
const givenModelVersion = (trainingDetails: unknown) =>
|
||||
findUnique.mockResolvedValue({
|
||||
id: 1284593,
|
||||
name: 'v1',
|
||||
trainingDetails,
|
||||
model: { id: 7, userId: OWNER.id, name: 'esadribicstyle' },
|
||||
files: [
|
||||
@@ -78,13 +77,13 @@ describe('training epoch download filename', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('carries the architecture and the version scope', async () => {
|
||||
it('carries the architecture and the version id', async () => {
|
||||
givenModelVersion({ baseModel: 'krea2' });
|
||||
|
||||
const res = await call();
|
||||
|
||||
expect(res.headers['Content-Disposition']).toBe(
|
||||
'attachment; filename="esadribicstyle_krea2_v1-1284593_epoch_3.safetensors"'
|
||||
'attachment; filename="esadribicstyle_krea2_1284593_epoch_3.safetensors"'
|
||||
);
|
||||
});
|
||||
|
||||
@@ -102,14 +101,13 @@ describe('training epoch download filename', () => {
|
||||
const res = await call();
|
||||
|
||||
expect(res.headers['Content-Disposition']).toBe(
|
||||
'attachment; filename="esadribicstyle_v1-1284593_epoch_3.safetensors"'
|
||||
'attachment; filename="esadribicstyle_1284593_epoch_3.safetensors"'
|
||||
);
|
||||
});
|
||||
|
||||
it('refuses an epoch URL outside the orchestrator hosts', async () => {
|
||||
findUnique.mockResolvedValue({
|
||||
id: 1284593,
|
||||
name: 'v1',
|
||||
trainingDetails: { baseModel: 'krea2' },
|
||||
model: { id: 7, userId: OWNER.id, name: 'esadribicstyle' },
|
||||
files: [
|
||||
|
||||
@@ -60,11 +60,11 @@ describe('buildEpochArchiveEntries', () => {
|
||||
expect(unresolvedCount).toBe(0);
|
||||
expect(cappedCount).toBe(0);
|
||||
expect(entries).toEqual([
|
||||
{ blobId: 'MODEL1.safetensors', fileName: 'My_Cool_Model__V1-77_epoch_1.safetensors' },
|
||||
{ blobId: 'MODEL2.safetensors', fileName: 'My_Cool_Model__V1-77_epoch_2.safetensors' },
|
||||
{ blobId: 'E1S1.jpeg', fileName: 'My_Cool_Model__V1-77_epoch_1_sample_1.jpeg' },
|
||||
{ blobId: 'E2S1.jpeg', fileName: 'My_Cool_Model__V1-77_epoch_2_sample_1.jpeg' },
|
||||
{ blobId: 'E2S2.mp4', fileName: 'My_Cool_Model__V1-77_epoch_2_sample_2.mp4' },
|
||||
{ blobId: 'MODEL1.safetensors', fileName: 'My_Cool_Model__77_epoch_1.safetensors' },
|
||||
{ blobId: 'MODEL2.safetensors', fileName: 'My_Cool_Model__77_epoch_2.safetensors' },
|
||||
{ blobId: 'E1S1.jpeg', fileName: 'My_Cool_Model__77_epoch_1_sample_1.jpeg' },
|
||||
{ blobId: 'E2S1.jpeg', fileName: 'My_Cool_Model__77_epoch_2_sample_1.jpeg' },
|
||||
{ blobId: 'E2S2.mp4', fileName: 'My_Cool_Model__77_epoch_2_sample_2.mp4' },
|
||||
]);
|
||||
});
|
||||
|
||||
@@ -91,8 +91,8 @@ describe('buildEpochArchiveEntries', () => {
|
||||
});
|
||||
|
||||
expect(entries).toEqual([
|
||||
{ blobId: 'LEGACY.safetensors', fileName: 'legacy_V1-77_epoch_1.safetensors' },
|
||||
{ blobId: 'LEGACYS1.jpeg', fileName: 'legacy_V1-77_epoch_1_sample_1.jpeg' },
|
||||
{ blobId: 'LEGACY.safetensors', fileName: 'legacy_77_epoch_1.safetensors' },
|
||||
{ blobId: 'LEGACYS1.jpeg', fileName: 'legacy_77_epoch_1_sample_1.jpeg' },
|
||||
]);
|
||||
});
|
||||
|
||||
@@ -114,9 +114,7 @@ describe('buildEpochArchiveEntries', () => {
|
||||
versionId: 77,
|
||||
});
|
||||
|
||||
expect(entries).toEqual([
|
||||
{ blobId: 'OK.jpeg', fileName: 'legacy_V1-77_epoch_1_sample_1.jpeg' },
|
||||
]);
|
||||
expect(entries).toEqual([{ blobId: 'OK.jpeg', fileName: 'legacy_77_epoch_1_sample_1.jpeg' }]);
|
||||
expect(unresolvedCount).toBe(2);
|
||||
expect(cappedCount).toBe(0);
|
||||
});
|
||||
@@ -153,7 +151,6 @@ describe('getTrainingEpochArchive', () => {
|
||||
|
||||
const modelVersion = {
|
||||
id: 1,
|
||||
name: 'V2 (from epoch 5)',
|
||||
trainingDetails: { baseModel: 'pony' },
|
||||
model: { userId: 10, name: 'My Cool Model!' },
|
||||
files: [{ metadata: { trainingResults: v2Results } }],
|
||||
@@ -170,14 +167,14 @@ describe('getTrainingEpochArchive', () => {
|
||||
entries: expect.arrayContaining([
|
||||
{
|
||||
blobId: 'MODEL1.safetensors',
|
||||
fileName: 'My_Cool_Model__pony_V2__from_epoch_5-1_epoch_1.safetensors',
|
||||
fileName: 'My_Cool_Model__pony_1_epoch_1.safetensors',
|
||||
},
|
||||
{
|
||||
blobId: 'E2S2.mp4',
|
||||
fileName: 'My_Cool_Model__pony_V2__from_epoch_5-1_epoch_2_sample_2.mp4',
|
||||
fileName: 'My_Cool_Model__pony_1_epoch_2_sample_2.mp4',
|
||||
},
|
||||
]),
|
||||
archiveName: 'My_Cool_Model__pony_V2__from_epoch_5-1_training.zip',
|
||||
archiveName: 'My_Cool_Model__pony_1_training.zip',
|
||||
});
|
||||
expect(createBlobArchive.mock.calls[0][0].entries).toHaveLength(5);
|
||||
expect(result.url).toContain('/archive/token');
|
||||
|
||||
@@ -57,7 +57,6 @@ export type TrainingEpochArchiveEntries = {
|
||||
export function buildEpochArchiveEntries({
|
||||
trainingResults,
|
||||
modelName,
|
||||
versionName,
|
||||
versionId,
|
||||
architecture,
|
||||
maxEntries = MAX_BLOB_ARCHIVE_ENTRIES,
|
||||
@@ -68,7 +67,7 @@ export function buildEpochArchiveEntries({
|
||||
const epochs = [...normalizeEpochs(trainingResults)].sort(
|
||||
(a, b) => a.epochNumber - b.epochNumber
|
||||
);
|
||||
const run = { modelName, versionName, versionId, architecture };
|
||||
const run = { modelName, versionId, architecture };
|
||||
const candidates: Array<{ url: string; fileName: (blobId: string) => string }> = [];
|
||||
for (const epoch of epochs) {
|
||||
candidates.push({
|
||||
@@ -128,7 +127,6 @@ export async function getTrainingEpochArchive({
|
||||
where: { id: modelVersionId },
|
||||
select: {
|
||||
id: true,
|
||||
name: true,
|
||||
trainingDetails: true,
|
||||
model: { select: { userId: true, name: true } },
|
||||
files: { select: { metadata: true }, where: { type: 'Training Data' } },
|
||||
@@ -145,7 +143,6 @@ export async function getTrainingEpochArchive({
|
||||
|
||||
const run = {
|
||||
modelName: modelVersion.model.name,
|
||||
versionName: modelVersion.name,
|
||||
versionId: modelVersion.id,
|
||||
architecture: trainingArchitectureKey(
|
||||
modelVersion.trainingDetails as TrainingDetailsObj | null
|
||||
|
||||
@@ -12,44 +12,22 @@ describe('training file names', () => {
|
||||
expect(
|
||||
trainingEpochModelFileName({
|
||||
modelName: MODEL,
|
||||
versionName: 'V1',
|
||||
versionId: 401,
|
||||
architecture: 'sdxl',
|
||||
epochNumber: 6,
|
||||
})
|
||||
).toBe('My_Cool_Model__sdxl_V1-401_epoch_6.safetensors');
|
||||
).toBe('My_Cool_Model__sdxl_401_epoch_6.safetensors');
|
||||
});
|
||||
|
||||
it('omits the architecture segment for runs that predate it', () => {
|
||||
expect(
|
||||
trainingEpochModelFileName({
|
||||
modelName: MODEL,
|
||||
versionName: 'V1',
|
||||
versionId: 401,
|
||||
epochNumber: 6,
|
||||
})
|
||||
).toBe('My_Cool_Model__V1-401_epoch_6.safetensors');
|
||||
expect(trainingEpochModelFileName({ modelName: MODEL, versionId: 401, epochNumber: 6 })).toBe(
|
||||
'My_Cool_Model__401_epoch_6.safetensors'
|
||||
);
|
||||
});
|
||||
|
||||
it('distinguishes the same epoch number across two runs of one model', () => {
|
||||
const first = trainingEpochModelFileName({
|
||||
modelName: MODEL,
|
||||
versionName: 'V1',
|
||||
versionId: 401,
|
||||
epochNumber: 6,
|
||||
});
|
||||
const second = trainingEpochModelFileName({
|
||||
modelName: MODEL,
|
||||
versionName: 'V2 (from epoch 5)',
|
||||
versionId: 402,
|
||||
epochNumber: 6,
|
||||
});
|
||||
|
||||
expect(first).not.toBe(second);
|
||||
});
|
||||
|
||||
it('distinguishes two versions that share a name', () => {
|
||||
const args = { modelName: MODEL, versionName: 'V1', epochNumber: 6 };
|
||||
// Cumulative numbering only covers continuations; a fresh retrain restarts at epoch 1.
|
||||
it('keeps two runs of one model distinct at the same epoch and architecture', () => {
|
||||
const args = { modelName: MODEL, architecture: 'sdxl', epochNumber: 6 };
|
||||
|
||||
expect(trainingEpochModelFileName({ ...args, versionId: 401 })).not.toBe(
|
||||
trainingEpochModelFileName({ ...args, versionId: 402 })
|
||||
@@ -61,7 +39,6 @@ describe('training file names', () => {
|
||||
const name = trainingEpochSampleFileName(
|
||||
{
|
||||
modelName: 'M'.repeat(664),
|
||||
versionName: 'V'.repeat(296),
|
||||
versionId: 4029135,
|
||||
architecture: 'A'.repeat(64),
|
||||
epochNumber: 100,
|
||||
@@ -71,19 +48,7 @@ describe('training file names', () => {
|
||||
);
|
||||
|
||||
expect(name.length).toBeLessThanOrEqual(255);
|
||||
expect(name).toContain('-4029135_');
|
||||
});
|
||||
|
||||
it('keeps long-named versions of the same model distinct', () => {
|
||||
const args = {
|
||||
modelName: 'M'.repeat(664),
|
||||
versionName: 'V'.repeat(296),
|
||||
epochNumber: 6,
|
||||
};
|
||||
|
||||
expect(trainingEpochModelFileName({ ...args, versionId: 401 })).not.toBe(
|
||||
trainingEpochModelFileName({ ...args, versionId: 402 })
|
||||
);
|
||||
expect(name).toContain('_4029135_');
|
||||
});
|
||||
|
||||
// These names land in a Content-Disposition header, and `architecture` comes from an unvalidated
|
||||
@@ -91,7 +56,6 @@ describe('training file names', () => {
|
||||
it('strips header-breaking characters from every user-controlled part', () => {
|
||||
const name = trainingEpochModelFileName({
|
||||
modelName: 'evil"\r\nX-Injected: 1',
|
||||
versionName: 'v\n1',
|
||||
versionId: 401,
|
||||
architecture: 'sd\r\nxl"',
|
||||
epochNumber: 6,
|
||||
@@ -100,26 +64,15 @@ describe('training file names', () => {
|
||||
expect(name).not.toMatch(/[\r\n"]/);
|
||||
});
|
||||
|
||||
it('still produces a usable name when the version name sanitizes away', () => {
|
||||
expect(
|
||||
trainingEpochModelFileName({
|
||||
modelName: MODEL,
|
||||
versionName: '!!!',
|
||||
versionId: 402,
|
||||
epochNumber: 6,
|
||||
})
|
||||
).toBe('My_Cool_Model__402_epoch_6.safetensors');
|
||||
});
|
||||
|
||||
it('scopes samples and the run archive the same way as epoch models', () => {
|
||||
const run = { modelName: MODEL, versionName: 'V1', versionId: 401 };
|
||||
const run = { modelName: MODEL, versionId: 401 };
|
||||
|
||||
expect(trainingEpochModelFileName({ ...run, epochNumber: 6 })).toBe(
|
||||
'My_Cool_Model__V1-401_epoch_6.safetensors'
|
||||
'My_Cool_Model__401_epoch_6.safetensors'
|
||||
);
|
||||
expect(trainingEpochSampleFileName({ ...run, epochNumber: 6, sampleNumber: 2 }, '.jpeg')).toBe(
|
||||
'My_Cool_Model__V1-401_epoch_6_sample_2.jpeg'
|
||||
'My_Cool_Model__401_epoch_6_sample_2.jpeg'
|
||||
);
|
||||
expect(trainingRunArchiveName(run)).toBe('My_Cool_Model__V1-401_training.zip');
|
||||
expect(trainingRunArchiveName(run)).toBe('My_Cool_Model__401_training.zip');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,33 +1,22 @@
|
||||
/**
|
||||
* Names are scoped by model VERSION, not model: each training run gets its own version and numbers
|
||||
* its epochs from its own start, so `<model>_epoch_3` is ambiguous within a model. The version id
|
||||
* is what makes that a guarantee — version names are not unique within a model — and the name is
|
||||
* kept beside it only because it is the half a user recognises ("from epoch 10").
|
||||
* its epochs from its own start, so `<model>_epoch_3` is ambiguous within a model. The id carries
|
||||
* that, not the version name — version names are not unique within a model.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Names are user-supplied and unbounded; most filesystems reject a path component over 255 bytes.
|
||||
* The version id is never truncated — it is what makes the name unique, and two versions of one
|
||||
* model truncate their shared model name identically.
|
||||
* Model names reach 664 chars in prod; most filesystems reject a path component over 255 bytes.
|
||||
* The version id is never truncated — it is what makes the name unique.
|
||||
*/
|
||||
const MODEL_NAME_MAX = 80;
|
||||
const VERSION_NAME_MAX = 40;
|
||||
const ARCHITECTURE_MAX = 20;
|
||||
|
||||
function sanitize(value: string, maxLength: number) {
|
||||
return value.replace(/[^a-zA-Z0-9_-]/g, '_').slice(0, maxLength);
|
||||
}
|
||||
|
||||
/** `-` survives sanitising, so it reads as a separator between the name and the id rather than
|
||||
* running into the underscores the name itself decays into. */
|
||||
function versionScope({ versionName, versionId }: { versionName: string; versionId: number }) {
|
||||
const name = sanitize(versionName, VERSION_NAME_MAX).replace(/^_+|_+$/g, '');
|
||||
return name ? `${name}-${versionId}` : `${versionId}`;
|
||||
}
|
||||
|
||||
export type TrainingRunNameParts = {
|
||||
modelName: string;
|
||||
versionName: string;
|
||||
versionId: number;
|
||||
/** Short base-model key from `trainingArchitectureKey`, e.g. `pony`, `krea2`. Absent on older runs. */
|
||||
architecture?: string | null;
|
||||
@@ -35,14 +24,11 @@ export type TrainingRunNameParts = {
|
||||
|
||||
export function trainingRunFilePrefix({
|
||||
modelName,
|
||||
versionName,
|
||||
versionId,
|
||||
architecture,
|
||||
}: TrainingRunNameParts) {
|
||||
const arch = architecture ? sanitize(architecture, ARCHITECTURE_MAX).replace(/^_+|_+$/g, '') : '';
|
||||
return [sanitize(modelName, MODEL_NAME_MAX), arch, versionScope({ versionName, versionId })]
|
||||
.filter(Boolean)
|
||||
.join('_');
|
||||
return [sanitize(modelName, MODEL_NAME_MAX), arch, String(versionId)].filter(Boolean).join('_');
|
||||
}
|
||||
|
||||
export function trainingEpochModelFileName(args: TrainingRunNameParts & { epochNumber: number }) {
|
||||
|
||||
Reference in New Issue
Block a user