merge: integrate Learning download size removal
This commit is contained in:
@@ -0,0 +1,84 @@
|
|||||||
|
# Task: Remove Learning archive size validation
|
||||||
|
|
||||||
|
## Identity
|
||||||
|
|
||||||
|
- Task ID: 20260820-remove-download-size-check-4f8a2c1d
|
||||||
|
- Mode: Feature
|
||||||
|
- Branch: codex/20260820-remove-download-size-check-4f8a2c1d-remove-download-size-check
|
||||||
|
- Worktree: D:\Datas\OthersProjects\makelore-remove-download-size-check-4f8a2c1d
|
||||||
|
- Base commit: 2168e291b2bf8ac8690c482f56400554a8d77531
|
||||||
|
- Owner: codex
|
||||||
|
- Status: Ready for Integration
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
- Remove Learning project ZIP download checks that reject a response because of
|
||||||
|
an archive byte limit, declared archive size, or `Content-Length` mismatch.
|
||||||
|
- Keep the existing authenticated Main-owned download route, controlled
|
||||||
|
same-origin redirects, SHA-256 verification, ZIP signature validation,
|
||||||
|
temporary-file cleanup, and atomic final rename.
|
||||||
|
- Add focused regression coverage proving a valid archive downloads even when
|
||||||
|
its reported sizes are absent, inconsistent, or above the former limit.
|
||||||
|
|
||||||
|
## Intent And Constraints
|
||||||
|
|
||||||
|
- Follow the user's explicit direction to download without any size validation.
|
||||||
|
- Do not expose Works credentials, upstream archive URLs, temporary paths, or
|
||||||
|
final local paths to Renderer.
|
||||||
|
- Do not weaken digest or ZIP-format integrity checks; those are independent of
|
||||||
|
archive size validation.
|
||||||
|
- Limit production changes to the existing Electron Main Learning download
|
||||||
|
service and focused tests.
|
||||||
|
- This is a feature task, so canonical project memory remains unchanged until a
|
||||||
|
later Integration Gate.
|
||||||
|
|
||||||
|
## Outcome
|
||||||
|
|
||||||
|
- Electron Main no longer reads or compares archive `Content-Length`, project
|
||||||
|
`archiveBytes`, actual streamed bytes, or the former 512 MiB ceiling when
|
||||||
|
saving a Learning project.
|
||||||
|
- Learning project DTO projection accepts positive safe-integer archive sizes
|
||||||
|
above the former client limit so those projects can reach the download flow.
|
||||||
|
- SHA-256, ZIP signature, MIME, account-binding, redirect/origin, temporary-file
|
||||||
|
cleanup, and atomic rename checks remain unchanged.
|
||||||
|
- Added regressions for omitted `Content-Length`, inconsistent reported sizes,
|
||||||
|
and metadata above the former limit.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
- Red phase: the two new size-removal regressions failed against the old
|
||||||
|
implementation with `LEARNING_PROJECT_INVALID` and
|
||||||
|
`LEARNING_INVALID_RESPONSE`.
|
||||||
|
- Focused Learning tests: `2` files, `17 passed`.
|
||||||
|
- Full unit suite via pinned pnpm 10.33.4: `176` files, `2061 passed`.
|
||||||
|
- `pnpm run typecheck`: passed.
|
||||||
|
- `pnpm run lint:check`: passed with zero errors and six pre-existing warnings.
|
||||||
|
- `pnpm run build:vite`: passed; existing dynamic-import and chunk-size
|
||||||
|
advisories only.
|
||||||
|
- `git diff --check`: passed.
|
||||||
|
- Independent sub-agent review was not run because the user explicitly required
|
||||||
|
that no sub-agents be created; final review was performed in the primary task.
|
||||||
|
|
||||||
|
## Follow-ups
|
||||||
|
|
||||||
|
- Integrate the source commit into `main`.
|
||||||
|
- During Integration Gate, reconcile all canonical documentation and remove the
|
||||||
|
obsolete size-failure smoke cases listed in the promotion candidate below.
|
||||||
|
|
||||||
|
## Promotion Candidates
|
||||||
|
|
||||||
|
- Target: ADR-005, Learning system overview/domain/current state, `README.md`,
|
||||||
|
`docs/learning-project-catalog-server-contract.md`, success criteria, and the
|
||||||
|
Learning release commitment.
|
||||||
|
- Proposal: remove the client download byte ceiling and all declared/
|
||||||
|
transport size consistency requirements while retaining same-origin
|
||||||
|
redirects, SHA-256, ZIP signature, temporary-file cleanup, and atomic save.
|
||||||
|
- Evidence: explicit user direction and focused download regressions from
|
||||||
|
this task.
|
||||||
|
- Future impact: the desktop client may consume disk space according to the
|
||||||
|
upstream archive size; server-side publication/storage policy may still set
|
||||||
|
independent upload limits, but they are not enforced during client download.
|
||||||
|
- Semantic conflicts: supersedes the 512 MiB and declared-size requirements
|
||||||
|
in accepted ADR-005 and its derivative documentation and smoke checklist.
|
||||||
|
- Human confirmation required: no; the user explicitly requested removal of
|
||||||
|
download size validation.
|
||||||
@@ -2,7 +2,6 @@ import { app, dialog, type SaveDialogOptions } from 'electron';
|
|||||||
import type { IncomingMessage, ServerResponse } from 'node:http';
|
import type { IncomingMessage, ServerResponse } from 'node:http';
|
||||||
import { join } from 'node:path';
|
import { join } from 'node:path';
|
||||||
import {
|
import {
|
||||||
LEARNING_ARCHIVE_MAX_BYTES,
|
|
||||||
LEARNING_MEDIA_MAX_BYTES,
|
LEARNING_MEDIA_MAX_BYTES,
|
||||||
type LearningProjectDetail,
|
type LearningProjectDetail,
|
||||||
} from '../../../shared/learning';
|
} from '../../../shared/learning';
|
||||||
@@ -166,7 +165,7 @@ function projectSummary(value: unknown): Record<string, unknown> {
|
|||||||
cover: projectImage(project.cover),
|
cover: projectImage(project.cover),
|
||||||
tags,
|
tags,
|
||||||
version: nullableString(project.version, 64),
|
version: nullableString(project.version, 64),
|
||||||
archiveBytes: boundedInteger(project.archiveBytes, 1, LEARNING_ARCHIVE_MAX_BYTES),
|
archiveBytes: boundedInteger(project.archiveBytes, 1, Number.MAX_SAFE_INTEGER),
|
||||||
publishedAt: isoTimestamp(project.publishedAt),
|
publishedAt: isoTimestamp(project.publishedAt),
|
||||||
updatedAt: isoTimestamp(project.updatedAt),
|
updatedAt: isoTimestamp(project.updatedAt),
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -1,10 +1,7 @@
|
|||||||
import { createHash, randomUUID } from 'node:crypto';
|
import { createHash, randomUUID } from 'node:crypto';
|
||||||
import { open, rename, rm } from 'node:fs/promises';
|
import { open, rename, rm } from 'node:fs/promises';
|
||||||
import { basename, dirname, extname, join } from 'node:path';
|
import { basename, dirname, extname, join } from 'node:path';
|
||||||
import {
|
import type { LearningProjectDetail } from '../../shared/learning';
|
||||||
LEARNING_ARCHIVE_MAX_BYTES,
|
|
||||||
type LearningProjectDetail,
|
|
||||||
} from '../../shared/learning';
|
|
||||||
import type { WorksSquareAccountBinding } from './works-square-session';
|
import type { WorksSquareAccountBinding } from './works-square-session';
|
||||||
|
|
||||||
const PROJECT_ID_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$/;
|
const PROJECT_ID_PATTERN = /^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$/;
|
||||||
@@ -53,9 +50,6 @@ function assertCurrentAccount(
|
|||||||
|
|
||||||
function validateProject(project: LearningProjectDetail): void {
|
function validateProject(project: LearningProjectDetail): void {
|
||||||
if (!PROJECT_ID_PATTERN.test(project.id)
|
if (!PROJECT_ID_PATTERN.test(project.id)
|
||||||
|| !Number.isSafeInteger(project.archiveBytes)
|
|
||||||
|| project.archiveBytes <= 0
|
|
||||||
|| project.archiveBytes > LEARNING_ARCHIVE_MAX_BYTES
|
|
||||||
|| !SHA256_PATTERN.test(project.archiveSha256)) {
|
|| !SHA256_PATTERN.test(project.archiveSha256)) {
|
||||||
throw new LearningProjectDownloadError(502, 'LEARNING_PROJECT_INVALID', '项目下载信息无效');
|
throw new LearningProjectDownloadError(502, 'LEARNING_PROJECT_INVALID', '项目下载信息无效');
|
||||||
}
|
}
|
||||||
@@ -122,26 +116,15 @@ async function writeVerifiedArchive(input: {
|
|||||||
await input.response.body.cancel().catch(() => undefined);
|
await input.response.body.cancel().catch(() => undefined);
|
||||||
throw new LearningProjectDownloadError(502, 'LEARNING_ARCHIVE_MIME_INVALID', '项目压缩包类型无效');
|
throw new LearningProjectDownloadError(502, 'LEARNING_ARCHIVE_MIME_INVALID', '项目压缩包类型无效');
|
||||||
}
|
}
|
||||||
const declaredLength = Number(input.response.headers.get('content-length'));
|
|
||||||
if (Number.isFinite(declaredLength) && declaredLength !== input.project.archiveBytes) {
|
|
||||||
await input.response.body.cancel().catch(() => undefined);
|
|
||||||
throw new LearningProjectDownloadError(502, 'LEARNING_ARCHIVE_SIZE_MISMATCH', '项目压缩包大小校验失败');
|
|
||||||
}
|
|
||||||
|
|
||||||
const handle = await open(input.temporaryPath, 'wx');
|
const handle = await open(input.temporaryPath, 'wx');
|
||||||
const hash = createHash('sha256');
|
const hash = createHash('sha256');
|
||||||
const signature: number[] = [];
|
const signature: number[] = [];
|
||||||
let bytes = 0;
|
|
||||||
const reader = input.response.body.getReader();
|
const reader = input.response.body.getReader();
|
||||||
try {
|
try {
|
||||||
while (true) {
|
while (true) {
|
||||||
const { done, value } = await reader.read();
|
const { done, value } = await reader.read();
|
||||||
if (done) break;
|
if (done) break;
|
||||||
assertCurrentAccount(input.binding, input.isCurrentAccountBinding);
|
assertCurrentAccount(input.binding, input.isCurrentAccountBinding);
|
||||||
bytes += value.byteLength;
|
|
||||||
if (bytes > input.project.archiveBytes || bytes > LEARNING_ARCHIVE_MAX_BYTES) {
|
|
||||||
throw new LearningProjectDownloadError(502, 'LEARNING_ARCHIVE_SIZE_MISMATCH', '项目压缩包大小校验失败');
|
|
||||||
}
|
|
||||||
for (const byte of value.subarray(0, Math.max(0, 4 - signature.length))) signature.push(byte);
|
for (const byte of value.subarray(0, Math.max(0, 4 - signature.length))) signature.push(byte);
|
||||||
hash.update(value);
|
hash.update(value);
|
||||||
await handle.write(value);
|
await handle.write(value);
|
||||||
@@ -151,9 +134,6 @@ async function writeVerifiedArchive(input: {
|
|||||||
await handle.close();
|
await handle.close();
|
||||||
}
|
}
|
||||||
|
|
||||||
if (bytes !== input.project.archiveBytes) {
|
|
||||||
throw new LearningProjectDownloadError(502, 'LEARNING_ARCHIVE_SIZE_MISMATCH', '项目压缩包大小校验失败');
|
|
||||||
}
|
|
||||||
if (signature.length < 4 || signature[0] !== 0x50 || signature[1] !== 0x4b
|
if (signature.length < 4 || signature[0] !== 0x50 || signature[1] !== 0x4b
|
||||||
|| !((signature[2] === 0x03 && signature[3] === 0x04)
|
|| !((signature[2] === 0x03 && signature[3] === 0x04)
|
||||||
|| (signature[2] === 0x05 && signature[3] === 0x06)
|
|| (signature[2] === 0x05 && signature[3] === 0x06)
|
||||||
|
|||||||
@@ -2,8 +2,6 @@
|
|||||||
|
|
||||||
export const LEARNING_API_PATH = '/api/works/learning';
|
export const LEARNING_API_PATH = '/api/works/learning';
|
||||||
|
|
||||||
/** Main rejects larger project archives before writing any bytes. */
|
|
||||||
export const LEARNING_ARCHIVE_MAX_BYTES = 512 * 1024 * 1024;
|
|
||||||
export const LEARNING_MEDIA_MAX_BYTES = 10 * 1024 * 1024;
|
export const LEARNING_MEDIA_MAX_BYTES = 10 * 1024 * 1024;
|
||||||
|
|
||||||
export type LearningProjectImage = {
|
export type LearningProjectImage = {
|
||||||
|
|||||||
@@ -67,6 +67,46 @@ describe('Learning project verified download', () => {
|
|||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('downloads without comparing metadata or transport-reported archive sizes', async () => {
|
||||||
|
const fetchImpl = vi.fn<typeof fetch>().mockResolvedValue(new Response(archive, {
|
||||||
|
status: 200,
|
||||||
|
headers: { 'Content-Type': 'application/zip', 'Content-Length': String(archive.length + 100) },
|
||||||
|
}));
|
||||||
|
const destinationPath = join(root, 'large-project.zip');
|
||||||
|
|
||||||
|
await saveLearningProjectArchive({
|
||||||
|
project: project({ archiveBytes: 512 * 1024 * 1024 + 1 }),
|
||||||
|
destinationPath,
|
||||||
|
binding,
|
||||||
|
fetchImpl,
|
||||||
|
getAccessToken: vi.fn().mockResolvedValue('works-token'),
|
||||||
|
isCurrentAccountBinding: () => true,
|
||||||
|
apiBaseUrl: 'https://square.example',
|
||||||
|
});
|
||||||
|
|
||||||
|
await expect(readFile(destinationPath)).resolves.toEqual(archive);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('downloads when the archive response omits Content-Length', async () => {
|
||||||
|
const fetchImpl = vi.fn<typeof fetch>().mockResolvedValue(new Response(archive, {
|
||||||
|
status: 200,
|
||||||
|
headers: { 'Content-Type': 'application/zip' },
|
||||||
|
}));
|
||||||
|
const destinationPath = join(root, 'unknown-size-project.zip');
|
||||||
|
|
||||||
|
await saveLearningProjectArchive({
|
||||||
|
project: project({ archiveBytes: 1 }),
|
||||||
|
destinationPath,
|
||||||
|
binding,
|
||||||
|
fetchImpl,
|
||||||
|
getAccessToken: vi.fn().mockResolvedValue('works-token'),
|
||||||
|
isCurrentAccountBinding: () => true,
|
||||||
|
apiBaseUrl: 'https://square.example',
|
||||||
|
});
|
||||||
|
|
||||||
|
await expect(readFile(destinationPath)).resolves.toEqual(archive);
|
||||||
|
});
|
||||||
|
|
||||||
it('rejects an unsafe redirect and never forwards Bearer credentials to redirects', async () => {
|
it('rejects an unsafe redirect and never forwards Bearer credentials to redirects', async () => {
|
||||||
const fetchImpl = vi.fn<typeof fetch>().mockResolvedValue(new Response(null, {
|
const fetchImpl = vi.fn<typeof fetch>().mockResolvedValue(new Response(null, {
|
||||||
status: 302,
|
status: 302,
|
||||||
|
|||||||
@@ -161,6 +161,26 @@ describe('Learning project Main route boundary', () => {
|
|||||||
expect(JSON.stringify(response.json)).not.toContain('private/project.zip');
|
expect(JSON.stringify(response.json)).not.toContain('private/project.zip');
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('accepts project metadata above the former client archive-size limit', async () => {
|
||||||
|
getAccessToken.mockResolvedValue('works-token');
|
||||||
|
const archiveBytes = 512 * 1024 * 1024 + 1;
|
||||||
|
fetchImpl.mockResolvedValue(envelope(project({ archiveBytes })));
|
||||||
|
const handler = createLearningRouteHandler({ fetchImpl, getAccessToken });
|
||||||
|
const response = createResponse();
|
||||||
|
|
||||||
|
await handler(
|
||||||
|
{ method: 'GET' } as IncomingMessage,
|
||||||
|
response.res,
|
||||||
|
new URL('http://127.0.0.1/api/works/learning/projects/project-1'),
|
||||||
|
{} as never,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(response.json).toMatchObject({
|
||||||
|
success: true,
|
||||||
|
data: { id: 'project-1', archiveBytes },
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
it('rejects malformed identities and unsafe media URLs in successful DTOs', async () => {
|
it('rejects malformed identities and unsafe media URLs in successful DTOs', async () => {
|
||||||
getAccessToken.mockResolvedValue('works-token');
|
getAccessToken.mockResolvedValue('works-token');
|
||||||
fetchImpl.mockResolvedValue(envelope({
|
fetchImpl.mockResolvedValue(envelope({
|
||||||
|
|||||||
Reference in New Issue
Block a user