simplify production database configuration
This commit is contained in:
@@ -41,11 +41,10 @@ ZHINIAN_WORKER_REQUEST_TIMEOUT_MS=120000
|
||||
# Production must explicitly select postgres; it never falls back to container-local JSON.
|
||||
ZHINIAN_DATA_BACKEND=local
|
||||
DATABASE_URL=
|
||||
# Migration runner only: PostgreSQL role used by the Web DATABASE_URL.
|
||||
# Migration runner only: PostgreSQL role used by the Go API DATABASE_URL.
|
||||
DATABASE_APP_ROLE=
|
||||
DATABASE_SSL_MODE=disable
|
||||
# For RDS SSL, set verify-full and mount the downloaded CA certificate at this path.
|
||||
DATABASE_CA_CERT_PATH=
|
||||
# Optional URL query for an explicit RDS CA, for example:
|
||||
# postgresql://user:password@rds-host:5432/app?sslmode=verify-full&sslrootcert=/etc/zhinian/rds/ca.pem
|
||||
DATABASE_POOL_MAX=10
|
||||
DATABASE_IDLE_TIMEOUT_MS=30000
|
||||
DATABASE_CONNECTION_TIMEOUT_MS=5000
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
# Task: Simplify production database and deployment environment configuration
|
||||
|
||||
## Identity
|
||||
|
||||
- Task ID: 20260816-simplify-prod-env-8a4c
|
||||
- Mode: Feature
|
||||
- Branch: codex/20260816-simplify-prod-env-8a4c-simplify-prod-env
|
||||
- Worktree: D:\Datas\OthersProjects\NianAIGC-simplify-prod-env-8a4c
|
||||
- Base commit: 8cb8b5e463b752230fc675f05de3d910f8c90332
|
||||
- Owner: codex
|
||||
- Status: Ready for Integration
|
||||
|
||||
## Scope
|
||||
|
||||
- Simplify PostgreSQL connection configuration so `DATABASE_URL` is the only
|
||||
database connection setting; allow `sslmode` and optional `sslrootcert` in
|
||||
that URL while retaining full certificate verification by default.
|
||||
- Remove non-essential production ACK environment and Secret injections while
|
||||
retaining the Go API, Next session, bootstrap, RDS CA mount, and runtime
|
||||
storage settings required by the first deployment.
|
||||
- Synchronize Go/Node clients, ACK checks, deployment documentation, README
|
||||
references, and configuration tests.
|
||||
|
||||
## Intent And Constraints
|
||||
|
||||
- Do not weaken TLS verification or introduce `InsecureSkipVerify`.
|
||||
- Keep local JSON development behavior and existing database pool defaults.
|
||||
- Preserve the existing split topology: Next.js serves pages, Go owns the
|
||||
database and embedded WorkerLoop.
|
||||
- Feature mode may update task-scoped code/deployment/docs, but not canonical
|
||||
`.project-docs` memory.
|
||||
|
||||
## Outcome
|
||||
|
||||
- Completed the single-variable database contract. Go, the legacy TypeScript
|
||||
adapter, and migration scripts now read TLS settings from `DATABASE_URL`;
|
||||
production defaults to `sslmode=verify-full`, and `sslrootcert` is optional
|
||||
in the URL when the RDS CA is mounted. Separate
|
||||
`DATABASE_SSL_MODE`/`DATABASE_CA_CERT_PATH` environment variables are no
|
||||
longer consumed.
|
||||
- Reduced ACK runtime configuration to startup essentials, added the missing
|
||||
`GO_BACKEND_HOST=0.0.0.0`, removed unused provider/webhook/API-key/legacy
|
||||
worker Secret injections, and retained the CA Secret as a file mount rather
|
||||
than an environment variable.
|
||||
- Updated `.env.example`, deployment docs, READMEs, ACK assertions, and
|
||||
database configuration tests.
|
||||
|
||||
## Verification
|
||||
|
||||
- `go test ./...` passed in `backend/`.
|
||||
- `go test ./internal/postgres` passed after the final TLS assertion update.
|
||||
- `npm run deploy:check` passed (9 ACK manifests).
|
||||
- `node --check scripts/postgres-client.mjs` passed.
|
||||
- `node --check scripts/check-ack-manifests.mjs` passed.
|
||||
- `git diff --check` passed; only Git line-ending warnings were reported.
|
||||
- Full Vitest/Next typecheck was not run because this isolated worktree has no
|
||||
`node_modules` installation.
|
||||
|
||||
## Follow-ups
|
||||
|
||||
- Replace all ACK placeholders, especially the RDS CA Secret and the
|
||||
`DATABASE_URL` values. For strict RDS verification, include
|
||||
`?sslmode=verify-full&sslrootcert=/etc/zhinian/rds/ca.pem` in both the Go and
|
||||
migration connection URLs.
|
||||
- Add provider/OSS/API-key/Webhook Secret references only when those optional
|
||||
capabilities are enabled.
|
||||
|
||||
## Promotion Candidates
|
||||
|
||||
- None. The deployment simplification is task-scoped; canonical architecture
|
||||
already describes the same two-workload production topology.
|
||||
@@ -235,7 +235,7 @@ cp .env.example .env.local
|
||||
- `ALI_OSS_*`:用于上传素材和生成结果转存
|
||||
- `ZHINIAN_DATA_BACKEND`:生产使用 `postgres`,开发可使用 `local`
|
||||
- `DATABASE_URL`:仅服务端读取的 PostgreSQL 连接串
|
||||
- `DATABASE_SSL_MODE` / `DATABASE_CA_CERT_PATH`:RDS TLS 验证配置
|
||||
- `DATABASE_URL` 的 `sslmode` / `sslrootcert`:在同一个连接串中配置 RDS TLS;默认使用 `sslmode=verify-full`
|
||||
|
||||
当 `ZHINIAN_DATA_BACKEND=local` 时,应用使用 `.runtime/data/web-app-state.json` 作为单实例开发数据层。生产 `postgres` 模式缺少连接配置会直接失败,不会静默写入本地 JSON。如果 OSS 未配置,上传和 mock 结果会保存到 `.runtime/uploads` 和 `.runtime/generated-results`,并通过 Web 路由提供访问。
|
||||
|
||||
|
||||
@@ -329,7 +329,7 @@ cp .env.example .env.local
|
||||
| `ALI_OSS_*` | 上传素材和生成结果转存配置 |
|
||||
| `ZHINIAN_DATA_BACKEND` | `postgres` 或 `local` |
|
||||
| `DATABASE_URL` | PostgreSQL 连接串(仅放 Secret) |
|
||||
| `DATABASE_SSL_MODE` / `DATABASE_CA_CERT_PATH` | RDS TLS 验证配置 |
|
||||
| `DATABASE_URL` 的 `sslmode` / `sslrootcert` | 在同一个连接串中配置 RDS TLS;默认使用 `sslmode=verify-full` |
|
||||
|
||||
`ZHINIAN_DATA_BACKEND=local` 时,应用使用 `.runtime/data/web-app-state.json` 作为单实例开发数据层;生产 `postgres` 模式配置错误会直接失败。未配置 OSS 时,上传和生成结果会写入 `.runtime/uploads` 与 `.runtime/generated-results`。
|
||||
|
||||
|
||||
@@ -49,7 +49,10 @@ func ParseConfig(getenv Getenv, readFile ReadFile) (Config, error) {
|
||||
ConnectionTimeout: 10 * time.Second,
|
||||
StatementTimeout: 30 * time.Second,
|
||||
ApplicationName: "zhinian-go",
|
||||
SSLMode: SSLDisable,
|
||||
// DATABASE_URL is the only database setting. Production defaults to
|
||||
// full TLS verification; sslrootcert may be supplied in the URL when
|
||||
// the RDS CA is not part of the container's system trust store.
|
||||
SSLMode: SSLVerifyFull,
|
||||
}
|
||||
|
||||
backend := strings.ToLower(strings.TrimSpace(getenv("ZHINIAN_DATA_BACKEND")))
|
||||
@@ -93,37 +96,36 @@ func ParseConfig(getenv Getenv, readFile ReadFile) (Config, error) {
|
||||
if parsed.Scheme != "postgres" && parsed.Scheme != "postgresql" {
|
||||
return Config{}, fmt.Errorf("DATABASE_URL must use the postgres:// or postgresql:// scheme")
|
||||
}
|
||||
for key := range parsed.Query() {
|
||||
if strings.HasPrefix(strings.ToLower(key), "ssl") {
|
||||
return Config{}, fmt.Errorf("DATABASE_URL must not contain SSL query parameters (%s); use DATABASE_SSL_MODE and DATABASE_CA_CERT_PATH", key)
|
||||
query := parsed.Query()
|
||||
for key := range query {
|
||||
lower := strings.ToLower(key)
|
||||
if strings.HasPrefix(lower, "ssl") && lower != "sslmode" && lower != "sslrootcert" {
|
||||
return Config{}, fmt.Errorf("DATABASE_URL contains unsupported SSL query parameter %s; use sslmode and optional sslrootcert", key)
|
||||
}
|
||||
}
|
||||
|
||||
mode := strings.ToLower(strings.TrimSpace(getenv("DATABASE_SSL_MODE")))
|
||||
if mode != "" {
|
||||
if mode := strings.ToLower(strings.TrimSpace(query.Get("sslmode"))); mode != "" {
|
||||
cfg.SSLMode = SSLMode(mode)
|
||||
}
|
||||
switch cfg.SSLMode {
|
||||
case SSLDisable:
|
||||
case SSLVerifyFull:
|
||||
path := strings.TrimSpace(getenv("DATABASE_CA_CERT_PATH"))
|
||||
if path == "" {
|
||||
return Config{}, fmt.Errorf("DATABASE_CA_CERT_PATH is required when DATABASE_SSL_MODE=verify-full")
|
||||
cfg.TLSConfig = &tls.Config{MinVersion: tls.VersionTLS12}
|
||||
if path := strings.TrimSpace(query.Get("sslrootcert")); path != "" {
|
||||
if readFile == nil {
|
||||
return Config{}, fmt.Errorf("a certificate reader is required when DATABASE_URL contains sslrootcert")
|
||||
}
|
||||
pem, readErr := readFile(path)
|
||||
if readErr != nil {
|
||||
return Config{}, fmt.Errorf("read DATABASE_URL sslrootcert: %w", readErr)
|
||||
}
|
||||
roots := x509.NewCertPool()
|
||||
if !roots.AppendCertsFromPEM(pem) {
|
||||
return Config{}, fmt.Errorf("DATABASE_URL sslrootcert does not contain a valid CA certificate")
|
||||
}
|
||||
cfg.TLSConfig.RootCAs = roots
|
||||
}
|
||||
if readFile == nil {
|
||||
return Config{}, fmt.Errorf("CA certificate reader is required")
|
||||
}
|
||||
pem, readErr := readFile(path)
|
||||
if readErr != nil {
|
||||
return Config{}, fmt.Errorf("read DATABASE_CA_CERT_PATH: %w", readErr)
|
||||
}
|
||||
roots := x509.NewCertPool()
|
||||
if !roots.AppendCertsFromPEM(pem) {
|
||||
return Config{}, fmt.Errorf("DATABASE_CA_CERT_PATH does not contain a valid CA certificate")
|
||||
}
|
||||
cfg.TLSConfig = &tls.Config{RootCAs: roots, MinVersion: tls.VersionTLS12}
|
||||
default:
|
||||
return Config{}, fmt.Errorf("DATABASE_SSL_MODE must be 'disable' or 'verify-full'")
|
||||
return Config{}, fmt.Errorf("DATABASE_URL sslmode must be 'disable' or 'verify-full'")
|
||||
}
|
||||
return cfg, nil
|
||||
}
|
||||
|
||||
@@ -7,6 +7,7 @@ import (
|
||||
"crypto/x509/pkix"
|
||||
"encoding/pem"
|
||||
"math/big"
|
||||
"net/url"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
@@ -38,14 +39,13 @@ func TestParseConfigRequiresPostgresURL(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseConfigAcceptsOnlyPostgresSchemesAndRejectsSSLQueryParameters(t *testing.T) {
|
||||
func TestParseConfigAcceptsOnlyPostgresSchemesAndRejectsUnsupportedSSLQueryParameters(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
url string
|
||||
}{
|
||||
{name: "wrong scheme", url: "https://db.example/app"},
|
||||
{name: "sslmode", url: "postgres://db.example/app?sslmode=require"},
|
||||
{name: "mixed case ssl parameter", url: "postgresql://db.example/app?SSLcert=x"},
|
||||
{name: "mixed case unsupported ssl parameter", url: "postgresql://db.example/app?SSLcert=x"},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
@@ -60,7 +60,7 @@ func TestParseConfigAcceptsOnlyPostgresSchemesAndRejectsSSLQueryParameters(t *te
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseConfigBuildsVerifyFullTLSFromCA(t *testing.T) {
|
||||
func TestParseConfigBuildsVerifyFullTLSFromURLCA(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
caPath := filepath.Join(dir, "ca.pem")
|
||||
const ca = "-----BEGIN CERTIFICATE-----\nMIIB\n-----END CERTIFICATE-----\n"
|
||||
@@ -71,10 +71,8 @@ func TestParseConfigBuildsVerifyFullTLSFromCA(t *testing.T) {
|
||||
return []byte(ca), nil
|
||||
}
|
||||
_, err := ParseConfig(env(map[string]string{
|
||||
"ZHINIAN_DATA_BACKEND": "postgres",
|
||||
"DATABASE_URL": "postgresql://db.example/app",
|
||||
"DATABASE_SSL_MODE": "verify-full",
|
||||
"DATABASE_CA_CERT_PATH": caPath,
|
||||
"ZHINIAN_DATA_BACKEND": "postgres",
|
||||
"DATABASE_URL": "postgresql://db.example/app?sslmode=verify-full&sslrootcert=" + url.QueryEscape(caPath),
|
||||
}), read)
|
||||
if err == nil || !strings.Contains(err.Error(), "CA certificate") {
|
||||
t.Fatalf("ParseConfig() error = %v, want invalid CA certificate error", err)
|
||||
@@ -97,10 +95,8 @@ func TestParseConfigVerifyFullBuildsRootsWithoutDisablingVerification(t *testing
|
||||
}
|
||||
ca := pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der})
|
||||
cfg, err := ParseConfig(env(map[string]string{
|
||||
"ZHINIAN_DATA_BACKEND": "postgres",
|
||||
"DATABASE_URL": "postgresql://db.example/app",
|
||||
"DATABASE_SSL_MODE": "verify-full",
|
||||
"DATABASE_CA_CERT_PATH": "/ca.pem",
|
||||
"ZHINIAN_DATA_BACKEND": "postgres",
|
||||
"DATABASE_URL": "postgresql://db.example/app?sslmode=verify-full&sslrootcert=%2Fca.pem",
|
||||
}), func(string) ([]byte, error) { return ca, nil })
|
||||
if err != nil {
|
||||
t.Fatalf("ParseConfig() error = %v", err)
|
||||
@@ -124,9 +120,12 @@ func TestParseConfigDefaultsAndNumericValidation(t *testing.T) {
|
||||
if cfg.PoolMax != 10 || cfg.IdleTimeout != 30*time.Second || cfg.ConnectionTimeout != 10*time.Second || cfg.StatementTimeout != 30*time.Second {
|
||||
t.Fatalf("unexpected defaults: %+v", cfg)
|
||||
}
|
||||
if cfg.ApplicationName != "zhinian-go" || cfg.SSLMode != SSLDisable {
|
||||
if cfg.ApplicationName != "zhinian-go" || cfg.SSLMode != SSLVerifyFull {
|
||||
t.Fatalf("unexpected identity/TLS defaults: %+v", cfg)
|
||||
}
|
||||
if cfg.TLSConfig == nil || cfg.TLSConfig.InsecureSkipVerify {
|
||||
t.Fatal("default PostgreSQL configuration must verify the server certificate")
|
||||
}
|
||||
|
||||
for name, value := range map[string]string{
|
||||
"DATABASE_POOL_MAX": "0",
|
||||
|
||||
@@ -42,7 +42,7 @@ func Open(ctx context.Context, config Config) (*Module, error) {
|
||||
poolConfig.ConnConfig.Fallbacks = nil
|
||||
case SSLVerifyFull:
|
||||
if config.TLSConfig == nil {
|
||||
return nil, fmt.Errorf("TLS configuration is required when DATABASE_SSL_MODE=verify-full")
|
||||
return nil, fmt.Errorf("TLS configuration is required when DATABASE_URL sslmode=verify-full")
|
||||
}
|
||||
tlsConfig := config.TLSConfig.Clone()
|
||||
if tlsConfig.ServerName == "" {
|
||||
@@ -51,7 +51,7 @@ func Open(ctx context.Context, config Config) (*Module, error) {
|
||||
poolConfig.ConnConfig.TLSConfig = tlsConfig
|
||||
poolConfig.ConnConfig.Fallbacks = nil
|
||||
default:
|
||||
return nil, fmt.Errorf("unsupported DATABASE_SSL_MODE %q", config.SSLMode)
|
||||
return nil, fmt.Errorf("unsupported DATABASE_URL sslmode %q", config.SSLMode)
|
||||
}
|
||||
pool, err := pgxpool.NewWithConfig(ctx, poolConfig)
|
||||
if err != nil {
|
||||
|
||||
@@ -37,8 +37,6 @@ func TestOpenPostgresDoesNotProbeBeforeReadiness(t *testing.T) {
|
||||
func TestParseConfigIgnoresPostgresTLSSettingsForLocalBackend(t *testing.T) {
|
||||
cfg, err := ParseConfig(env(map[string]string{
|
||||
"ZHINIAN_DATA_BACKEND": "local",
|
||||
"DATABASE_SSL_MODE": "verify-full",
|
||||
"DATABASE_CA_CERT_PATH": "/missing/ca.pem",
|
||||
"DATABASE_POOL_MAX": "0",
|
||||
"DATABASE_CONNECTION_TIMEOUT_MS": "invalid",
|
||||
}), nil)
|
||||
@@ -56,7 +54,7 @@ func TestOpenPostgresRejectsInvalidConfiguredTLSMode(t *testing.T) {
|
||||
DatabaseURL: "postgresql://app:secret@db.example/app",
|
||||
SSLMode: SSLMode("prefer"),
|
||||
})
|
||||
if err == nil || !strings.Contains(err.Error(), "DATABASE_SSL_MODE") {
|
||||
t.Fatalf("Open() error = %v", err)
|
||||
if err == nil || !strings.Contains(err.Error(), "unsupported DATABASE_URL sslmode") {
|
||||
t.Fatalf("Open() error = %v, want DATABASE_URL sslmode error", err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -6,19 +6,11 @@ metadata:
|
||||
data:
|
||||
NODE_ENV: production
|
||||
PORT: "3000"
|
||||
ZHINIAN_DATA_BACKEND: postgres
|
||||
ZHINIAN_AUTH_REQUIRED: auto
|
||||
ZHINIAN_WORKER_BASE_URL: http://zhinian-web:3000
|
||||
DATABASE_SSL_MODE: verify-full
|
||||
DATABASE_CA_CERT_PATH: /etc/zhinian/rds/ca.pem
|
||||
DATABASE_POOL_MAX: "10"
|
||||
DATABASE_CONNECTION_TIMEOUT_MS: "5000"
|
||||
DATABASE_IDLE_TIMEOUT_MS: "30000"
|
||||
DATABASE_STATEMENT_TIMEOUT_MS: "30000"
|
||||
DATABASE_APPLICATION_NAME: zhinian-web
|
||||
ZHINIAN_AUTH_REQUIRED: "1"
|
||||
ZHINIAN_PUBLIC_BASE_URL: https://REPLACE_WITH_PUBLIC_HOST
|
||||
---
|
||||
# Go API runtime settings. Provider endpoints/models use code defaults unless
|
||||
# overridden here; credentials always come from zhinian-go-providers.
|
||||
# optional provider/OSS credentials are added to the Deployment.
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
@@ -26,21 +18,13 @@ metadata:
|
||||
namespace: zhinian
|
||||
data:
|
||||
NODE_ENV: production
|
||||
GO_BACKEND_HOST: 0.0.0.0
|
||||
GO_BACKEND_PORT: "8080"
|
||||
ZHINIAN_DATA_BACKEND: postgres
|
||||
ZHINIAN_AUTH_REQUIRED: "true"
|
||||
ZHINIAN_AUTH_COOKIE_SECURE: "true"
|
||||
ZHINIAN_PUBLIC_BASE_URL: https://REPLACE_WITH_PUBLIC_HOST
|
||||
DATABASE_SSL_MODE: verify-full
|
||||
DATABASE_CA_CERT_PATH: /etc/zhinian/rds/ca.pem
|
||||
DATABASE_POOL_MAX: "10"
|
||||
DATABASE_CONNECTION_TIMEOUT_MS: "5000"
|
||||
DATABASE_IDLE_TIMEOUT_MS: "30000"
|
||||
DATABASE_STATEMENT_TIMEOUT_MS: "30000"
|
||||
DATABASE_APPLICATION_NAME: zhinian-go-api
|
||||
ZHINIAN_GO_EMBEDDED_WORKER: "true"
|
||||
ZHINIAN_WORKER_ID: zhinian-go-api-embedded
|
||||
ZHINIAN_BILLING_REQUIRED: "1"
|
||||
ZHINIAN_RUNTIME_DIR: /var/lib/zhinian/runtime
|
||||
ZHINIAN_LOG_DIR: /var/lib/zhinian/logs
|
||||
ZHINIAN_SETTINGS_FILE: /var/lib/zhinian/settings.env
|
||||
|
||||
@@ -62,83 +62,6 @@ spec:
|
||||
secretKeyRef:
|
||||
name: zhinian-go-bootstrap
|
||||
key: ZHINIAN_BOOTSTRAP_ADMIN_PASSWORD
|
||||
- name: ZHINIAN_BOOTSTRAP_ADMIN_NAME
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: zhinian-go-bootstrap
|
||||
key: ZHINIAN_BOOTSTRAP_ADMIN_NAME
|
||||
optional: true
|
||||
- name: VOLCENGINE_ACCESS_KEY_ID
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: zhinian-go-providers
|
||||
key: VOLCENGINE_ACCESS_KEY_ID
|
||||
optional: true
|
||||
- name: VOLCENGINE_SECRET_ACCESS_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: zhinian-go-providers
|
||||
key: VOLCENGINE_SECRET_ACCESS_KEY
|
||||
optional: true
|
||||
- name: JIMENG_IMAGE_GENERATE_46_REQ_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: zhinian-go-providers
|
||||
key: JIMENG_IMAGE_GENERATE_46_REQ_KEY
|
||||
optional: true
|
||||
- name: EVOLINK_API_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: zhinian-go-providers
|
||||
key: EVOLINK_API_KEY
|
||||
optional: true
|
||||
- name: SEEDANCE_API_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: zhinian-go-providers
|
||||
key: SEEDANCE_API_KEY
|
||||
optional: true
|
||||
- name: BAILIAN_API_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: zhinian-go-providers
|
||||
key: BAILIAN_API_KEY
|
||||
optional: true
|
||||
- name: DASHSCOPE_API_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: zhinian-go-providers
|
||||
key: DASHSCOPE_API_KEY
|
||||
optional: true
|
||||
- name: ALI_OSS_ACCESS_KEY_ID
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: zhinian-go-providers
|
||||
key: ALI_OSS_ACCESS_KEY_ID
|
||||
optional: true
|
||||
- name: ALI_OSS_ACCESS_KEY_SECRET
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: zhinian-go-providers
|
||||
key: ALI_OSS_ACCESS_KEY_SECRET
|
||||
optional: true
|
||||
- name: ZHINIAN_WEBHOOK_SECRET
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: zhinian-go-secrets
|
||||
key: ZHINIAN_WEBHOOK_SECRET
|
||||
- name: ZHINIAN_API_KEYS
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: zhinian-go-secrets
|
||||
key: ZHINIAN_API_KEYS
|
||||
optional: true
|
||||
- name: ZHINIAN_INTERNAL_WORKER_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: zhinian-go-secrets
|
||||
key: ZHINIAN_INTERNAL_WORKER_TOKEN
|
||||
optional: true
|
||||
volumeMounts:
|
||||
- name: rds-ca
|
||||
mountPath: /etc/zhinian/rds
|
||||
|
||||
@@ -31,7 +31,7 @@ spec:
|
||||
value: production
|
||||
- name: ZHINIAN_DATA_BACKEND
|
||||
value: postgres
|
||||
# Must match the username in zhinian-web-db/DATABASE_URL.
|
||||
# Must match the username in zhinian-go-db/DATABASE_URL.
|
||||
- name: DATABASE_APP_ROLE
|
||||
value: REPLACE_WITH_RDS_APP_ROLE
|
||||
- name: DATABASE_URL
|
||||
@@ -39,14 +39,6 @@ spec:
|
||||
secretKeyRef:
|
||||
name: zhinian-migration-db
|
||||
key: DATABASE_URL
|
||||
- name: DATABASE_SSL_MODE
|
||||
value: verify-full
|
||||
- name: DATABASE_CA_CERT_PATH
|
||||
value: /etc/zhinian/rds/ca.pem
|
||||
- name: DATABASE_CONNECTION_TIMEOUT_MS
|
||||
value: "5000"
|
||||
- name: DATABASE_STATEMENT_TIMEOUT_MS
|
||||
value: "60000"
|
||||
volumeMounts:
|
||||
- name: rds-ca
|
||||
mountPath: /etc/zhinian/rds
|
||||
|
||||
@@ -1,15 +1,16 @@
|
||||
# Example only. Replace every placeholder and keep the populated file out of Git.
|
||||
# Secrets marked "(local development only)" are not referenced by the first
|
||||
# production deployment; keep or drop them as your local workflow requires.
|
||||
# Provider, OSS, public API, and webhook secrets are optional add-ons and are
|
||||
# intentionally not injected by the minimal production Deployment.
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: zhinian-web-db
|
||||
name: zhinian-rds-ca
|
||||
namespace: zhinian
|
||||
type: Opaque
|
||||
stringData:
|
||||
# Local development only: the production Web workload holds no RDS credentials.
|
||||
DATABASE_URL: postgresql://APP_USER:APP_PASSWORD@RDS_INTERNAL_HOST:5432/APP_DATABASE
|
||||
# Public CA certificate for the RDS endpoint used in DATABASE_URL.
|
||||
ca.pem: |
|
||||
REPLACE_WITH_RDS_CA_PEM
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
@@ -19,17 +20,7 @@ metadata:
|
||||
type: Opaque
|
||||
stringData:
|
||||
# Manual schema execution only: run database/migrations/*.sql with this role.
|
||||
DATABASE_URL: postgresql://MIGRATION_USER:MIGRATION_PASSWORD@RDS_INTERNAL_HOST:5432/APP_DATABASE
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: zhinian-worker-auth
|
||||
namespace: zhinian
|
||||
type: Opaque
|
||||
stringData:
|
||||
# Local development only: the Node Worker is not deployed in production.
|
||||
ZHINIAN_INTERNAL_WORKER_TOKEN: REPLACE_WITH_A_LONG_RANDOM_VALUE
|
||||
DATABASE_URL: "postgresql://MIGRATION_USER:MIGRATION_PASSWORD@RDS_INTERNAL_HOST:5432/APP_DATABASE?sslmode=verify-full&sslrootcert=/etc/zhinian/rds/ca.pem"
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
@@ -50,7 +41,7 @@ metadata:
|
||||
type: Opaque
|
||||
stringData:
|
||||
# Application role (least privilege): grants applied manually after the SQL.
|
||||
DATABASE_URL: postgresql://APP_USER:APP_PASSWORD@RDS_INTERNAL_HOST:5432/APP_DATABASE
|
||||
DATABASE_URL: "postgresql://APP_USER:APP_PASSWORD@RDS_INTERNAL_HOST:5432/APP_DATABASE?sslmode=verify-full&sslrootcert=/etc/zhinian/rds/ca.pem"
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
@@ -63,35 +54,3 @@ stringData:
|
||||
# administrator exists. Password must be at least 8 characters.
|
||||
ZHINIAN_BOOTSTRAP_ADMIN_PHONE: REPLACE_WITH_ADMIN_PHONE
|
||||
ZHINIAN_BOOTSTRAP_ADMIN_PASSWORD: REPLACE_WITH_STRONG_PASSWORD
|
||||
ZHINIAN_BOOTSTRAP_ADMIN_NAME: 平台超级管理员
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: zhinian-go-providers
|
||||
namespace: zhinian
|
||||
type: Opaque
|
||||
stringData:
|
||||
# Provider credentials. Delete keys for providers you do not use; the Go
|
||||
# workload tolerates missing optional keys and fails closed when an enabled
|
||||
# engine has no credentials.
|
||||
VOLCENGINE_ACCESS_KEY_ID: REPLACE_OR_REMOVE
|
||||
VOLCENGINE_SECRET_ACCESS_KEY: REPLACE_OR_REMOVE
|
||||
JIMENG_IMAGE_GENERATE_46_REQ_KEY: REPLACE_OR_REMOVE
|
||||
EVOLINK_API_KEY: REPLACE_OR_REMOVE
|
||||
SEEDANCE_API_KEY: REPLACE_OR_REMOVE
|
||||
BAILIAN_API_KEY: REPLACE_OR_REMOVE
|
||||
DASHSCOPE_API_KEY: REPLACE_OR_REMOVE
|
||||
ALI_OSS_ACCESS_KEY_ID: REPLACE_OR_REMOVE
|
||||
ALI_OSS_ACCESS_KEY_SECRET: REPLACE_OR_REMOVE
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: zhinian-go-secrets
|
||||
namespace: zhinian
|
||||
type: Opaque
|
||||
stringData:
|
||||
ZHINIAN_WEBHOOK_SECRET: REPLACE_WITH_A_LONG_RANDOM_VALUE
|
||||
ZHINIAN_API_KEYS: REPLACE_WITH_PUBLIC_API_KEYS
|
||||
ZHINIAN_INTERNAL_WORKER_TOKEN: REPLACE_OR_REMOVE
|
||||
|
||||
@@ -6,13 +6,12 @@
|
||||
`DATABASE_URL`。优先使用 RDS 内网连接地址;ACK 节点/Pod 与 RDS 必须位于同一
|
||||
或网络可达的 VPC,并在 RDS 白名单或安全组中仅放行实际工作负载网段。
|
||||
|
||||
启用 RDS SSL 后,下载实例对应的 CA,创建 `zhinian-rds-ca` Secret,并将其挂载到
|
||||
`/etc/zhinian/rds/ca.pem`;同时设置 `DATABASE_SSL_MODE=verify-full` 和
|
||||
`DATABASE_CA_CERT_PATH=/etc/zhinian/rds/ca.pem`。不要使用关闭证书校验的配置。
|
||||
数据库连接只配置一个 `DATABASE_URL`。默认按 `sslmode=verify-full` 建立 TLS 连接;如果
|
||||
容器内没有 RDS CA 根证书,请把 CA 路径直接写进连接串,例如:
|
||||
`...?sslmode=verify-full&sslrootcert=/etc/zhinian/rds/ca.pem`,并按需挂载 CA 文件。
|
||||
不再设置 `DATABASE_SSL_MODE` 或 `DATABASE_CA_CERT_PATH`。
|
||||
|
||||
```bash
|
||||
kubectl -n zhinian create secret generic zhinian-rds-ca \
|
||||
--from-file=ca.pem=./path/to/downloaded-rds-ca.pem
|
||||
kubectl apply -f deploy/ack/configmap.yaml
|
||||
kubectl apply -f deploy/ack/secrets.example.yaml # 仅作模板;先替换全部占位值
|
||||
kubectl apply -f deploy/ack/web.yaml -f deploy/ack/go-api.yaml \
|
||||
@@ -98,8 +97,7 @@ Secret/ConfigMap 后触发 Deployment 滚动更新。
|
||||
| `ZHINIAN_DATA_BACKEND` | 生产固定为 `postgres`;配置错误不会降级到本地 JSON |
|
||||
| `DATABASE_URL` | PostgreSQL URI,仅存 Secret;不要写入镜像、ConfigMap 或日志 |
|
||||
| `DATABASE_APP_ROLE` | 应用角色名;手工执行授权语句时使用,与 Web/Go 的 RDS 用户名一致 |
|
||||
| `DATABASE_SSL_MODE` | `disable` 或 `verify-full`;RDS SSL 生产建议 `verify-full` |
|
||||
| `DATABASE_CA_CERT_PATH` | 已挂载 CA 文件路径 |
|
||||
| `DATABASE_URL` 的 `sslmode` / `sslrootcert` | 可在同一个连接串中指定 TLS 模式和 CA 路径;默认 `sslmode=verify-full` |
|
||||
| `DATABASE_POOL_MAX` | 单个 Web Pod 最大连接数 |
|
||||
| `DATABASE_CONNECTION_TIMEOUT_MS` | 建连超时;模板为 5000 ms |
|
||||
| `DATABASE_IDLE_TIMEOUT_MS` | 空闲连接回收时间 |
|
||||
@@ -155,8 +153,7 @@ ZHINIAN_AUTH_REQUIRED=auto
|
||||
ZHINIAN_AUTH_SESSION_SECRET=请替换为强随机会话密钥
|
||||
ZHINIAN_DATA_BACKEND=postgres
|
||||
DATABASE_URL=postgresql://应用账号:密码@RDS内网地址:5432/数据库名
|
||||
DATABASE_SSL_MODE=verify-full
|
||||
DATABASE_CA_CERT_PATH=/etc/zhinian/rds/ca.pem
|
||||
# 可选:把 sslmode/sslrootcert 直接写入 DATABASE_URL
|
||||
|
||||
ZHINIAN_API_KEYS=partner-a:请替换为强随机key
|
||||
ZHINIAN_INTERNAL_WORKER_TOKEN=请替换为强随机token
|
||||
|
||||
@@ -97,7 +97,7 @@ function getPool(): Pool {
|
||||
|
||||
const connectionString = process.env.DATABASE_URL?.trim();
|
||||
if (!connectionString) throw new Error("DATABASE_URL is required when ZHINIAN_DATA_BACKEND=postgres");
|
||||
assertConnectionStringContract(connectionString);
|
||||
const parsed = assertConnectionStringContract(connectionString);
|
||||
const config: PoolConfig = {
|
||||
connectionString,
|
||||
max: positiveInteger("DATABASE_POOL_MAX", 10),
|
||||
@@ -106,20 +106,25 @@ function getPool(): Pool {
|
||||
statement_timeout: positiveInteger("DATABASE_STATEMENT_TIMEOUT_MS", 30_000),
|
||||
application_name: process.env.DATABASE_APPLICATION_NAME?.trim() || "zhinian-web"
|
||||
};
|
||||
const sslMode = process.env.DATABASE_SSL_MODE?.trim().toLowerCase() || "disable";
|
||||
const sslMode = parsed.searchParams.get("sslmode")?.trim().toLowerCase() || "verify-full";
|
||||
if (sslMode === "verify-full") {
|
||||
const caPath = process.env.DATABASE_CA_CERT_PATH?.trim();
|
||||
if (!caPath) throw new Error("DATABASE_CA_CERT_PATH is required when DATABASE_SSL_MODE=verify-full");
|
||||
config.ssl = { ca: readFileSync(caPath, "utf8"), rejectUnauthorized: true };
|
||||
const caPath = parsed.searchParams.get("sslrootcert")?.trim();
|
||||
config.ssl = {
|
||||
...(caPath ? { ca: readFileSync(caPath, "utf8") } : {}),
|
||||
rejectUnauthorized: true
|
||||
};
|
||||
} else if (sslMode !== "disable") {
|
||||
throw new Error("DATABASE_SSL_MODE must be 'disable' or 'verify-full'");
|
||||
throw new Error("DATABASE_URL sslmode must be 'disable' or 'verify-full'");
|
||||
}
|
||||
parsed.searchParams.delete("sslmode");
|
||||
parsed.searchParams.delete("sslrootcert");
|
||||
config.connectionString = parsed.toString();
|
||||
pool = new Pool(config);
|
||||
pool.on("error", (error) => console.error("Unexpected PostgreSQL pool error", error));
|
||||
return pool;
|
||||
}
|
||||
|
||||
function assertConnectionStringContract(connectionString: string): void {
|
||||
function assertConnectionStringContract(connectionString: string): URL {
|
||||
let parsed: URL;
|
||||
try {
|
||||
parsed = new URL(connectionString);
|
||||
@@ -130,11 +135,17 @@ function assertConnectionStringContract(connectionString: string): void {
|
||||
throw new Error("DATABASE_URL must use the postgres:// or postgresql:// scheme");
|
||||
}
|
||||
const sslParameters = [...parsed.searchParams.keys()].filter((key) => key.toLowerCase().startsWith("ssl"));
|
||||
if (sslParameters.length > 0) {
|
||||
const unsupportedSSLParameters = sslParameters.filter((key) => !["sslmode", "sslrootcert"].includes(key.toLowerCase()));
|
||||
if (unsupportedSSLParameters.length > 0) {
|
||||
throw new Error(
|
||||
`DATABASE_URL must not contain SSL query parameters (${sslParameters.join(", ")}); use DATABASE_SSL_MODE and DATABASE_CA_CERT_PATH`
|
||||
`DATABASE_URL contains unsupported SSL query parameters (${unsupportedSSLParameters.join(", ")}); use sslmode and optional sslrootcert`
|
||||
);
|
||||
}
|
||||
const sslMode = parsed.searchParams.get("sslmode")?.trim().toLowerCase();
|
||||
if (sslMode && sslMode !== "disable" && sslMode !== "verify-full") {
|
||||
throw new Error("DATABASE_URL sslmode must be 'disable' or 'verify-full'");
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
|
||||
function positiveInteger(name: string, fallback: number): number {
|
||||
|
||||
@@ -13,7 +13,9 @@ for (const file of files) {
|
||||
const migrationJob = read("migration-job.yaml");
|
||||
assert(migrationJob.includes("name: ZHINIAN_DATA_BACKEND\n value: postgres"), "migration Job must select postgres");
|
||||
assert(migrationJob.includes("name: DATABASE_APP_ROLE"), "migration Job must provision the Web role");
|
||||
assert(migrationJob.includes("secretName: zhinian-rds-ca"), "migration Job must mount the RDS CA");
|
||||
assert(migrationJob.includes("secretName: zhinian-rds-ca"), "migration Job must mount the optional RDS CA used by DATABASE_URL");
|
||||
assert(!migrationJob.includes("DATABASE_SSL_MODE"), "migration Job must keep database TLS inside DATABASE_URL");
|
||||
assert(!migrationJob.includes("DATABASE_CA_CERT_PATH"), "migration Job must keep database TLS inside DATABASE_URL");
|
||||
|
||||
const web = read("web.yaml");
|
||||
assert(/^\s*replicas: 1\s*$/m.test(web), "Web must default to one replica until object storage is shared");
|
||||
@@ -27,10 +29,15 @@ assert(goApi.includes("path: /api/ready"), "Go API must use database-aware readi
|
||||
assert(goApi.includes("runAsNonRoot: true"), "Go API must run as a non-root user");
|
||||
assert(goApi.includes("name: zhinian-go-runtime"), "Go API must consume the Go runtime ConfigMap");
|
||||
assert(goApi.includes("name: zhinian-go-bootstrap"), "Go API must receive bootstrap administrator credentials");
|
||||
assert(goApi.includes("secretName: zhinian-rds-ca"), "Go API must mount the RDS CA");
|
||||
assert(goApi.includes("secretName: zhinian-rds-ca"), "Go API must mount the optional RDS CA used by DATABASE_URL");
|
||||
assert(!goApi.includes("DATABASE_SSL_MODE"), "Go API must keep database TLS inside DATABASE_URL");
|
||||
assert(!goApi.includes("DATABASE_CA_CERT_PATH"), "Go API must keep database TLS inside DATABASE_URL");
|
||||
|
||||
const configMap = read("configmap.yaml");
|
||||
assert(configMap.includes("ZHINIAN_GO_EMBEDDED_WORKER: \"true\""), "Go runtime ConfigMap must embed the WorkerLoop");
|
||||
assert(configMap.includes("GO_BACKEND_HOST: 0.0.0.0"), "Go runtime ConfigMap must listen on the Pod interface");
|
||||
assert(!configMap.includes("DATABASE_SSL_MODE"), "ConfigMaps must not carry database TLS settings");
|
||||
assert(!configMap.includes("DATABASE_CA_CERT_PATH"), "ConfigMaps must not carry database CA paths");
|
||||
|
||||
const ingress = read("ingress.yaml");
|
||||
assert(ingress.includes("path: /api/internal/worker"), "Ingress must intercept the internal worker prefix");
|
||||
|
||||
@@ -12,7 +12,7 @@ export function getScriptDataBackend(env = process.env) {
|
||||
export function createPostgresPool({ env = process.env, applicationName = "zhinian-script" } = {}) {
|
||||
const connectionString = env.DATABASE_URL?.trim();
|
||||
if (!connectionString) throw new Error("DATABASE_URL is required when ZHINIAN_DATA_BACKEND=postgres");
|
||||
assertConnectionStringContract(connectionString);
|
||||
const parsed = assertConnectionStringContract(connectionString);
|
||||
|
||||
const config = {
|
||||
connectionString,
|
||||
@@ -23,15 +23,21 @@ export function createPostgresPool({ env = process.env, applicationName = "zhini
|
||||
application_name: applicationName
|
||||
};
|
||||
|
||||
const sslMode = env.DATABASE_SSL_MODE?.trim().toLowerCase() || "disable";
|
||||
const sslMode = parsed.searchParams.get("sslmode")?.trim().toLowerCase() || "verify-full";
|
||||
if (sslMode === "verify-full") {
|
||||
const caPath = env.DATABASE_CA_CERT_PATH?.trim();
|
||||
if (!caPath) throw new Error("DATABASE_CA_CERT_PATH is required when DATABASE_SSL_MODE=verify-full");
|
||||
config.ssl = { ca: readFileSync(caPath, "utf8"), rejectUnauthorized: true };
|
||||
const caPath = parsed.searchParams.get("sslrootcert")?.trim();
|
||||
config.ssl = {
|
||||
...(caPath ? { ca: readFileSync(caPath, "utf8") } : {}),
|
||||
rejectUnauthorized: true
|
||||
};
|
||||
} else if (sslMode !== "disable") {
|
||||
throw new Error("DATABASE_SSL_MODE must be 'disable' or 'verify-full'");
|
||||
throw new Error("DATABASE_URL sslmode must be 'disable' or 'verify-full'");
|
||||
}
|
||||
|
||||
parsed.searchParams.delete("sslmode");
|
||||
parsed.searchParams.delete("sslrootcert");
|
||||
config.connectionString = parsed.toString();
|
||||
|
||||
return new Pool(config);
|
||||
}
|
||||
|
||||
@@ -57,11 +63,17 @@ function assertConnectionStringContract(connectionString) {
|
||||
throw new Error("DATABASE_URL must use the postgres:// or postgresql:// scheme");
|
||||
}
|
||||
const sslParameters = [...parsed.searchParams.keys()].filter((key) => key.toLowerCase().startsWith("ssl"));
|
||||
if (sslParameters.length > 0) {
|
||||
const unsupportedSSLParameters = sslParameters.filter((key) => !["sslmode", "sslrootcert"].includes(key.toLowerCase()));
|
||||
if (unsupportedSSLParameters.length > 0) {
|
||||
throw new Error(
|
||||
`DATABASE_URL must not contain SSL query parameters (${sslParameters.join(", ")}); use DATABASE_SSL_MODE and DATABASE_CA_CERT_PATH`
|
||||
`DATABASE_URL contains unsupported SSL query parameters (${unsupportedSSLParameters.join(", ")}); use sslmode and optional sslrootcert`
|
||||
);
|
||||
}
|
||||
const sslMode = parsed.searchParams.get("sslmode")?.trim().toLowerCase();
|
||||
if (sslMode && sslMode !== "disable" && sslMode !== "verify-full") {
|
||||
throw new Error("DATABASE_URL sslmode must be 'disable' or 'verify-full'");
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
|
||||
function positiveInteger(env, name, fallback) {
|
||||
|
||||
@@ -12,14 +12,25 @@ describe("PostgreSQL script configuration", () => {
|
||||
expect(getScriptDataBackend({ NODE_ENV: "test", ZHINIAN_DATA_BACKEND: "postgres" })).toBe("postgres");
|
||||
});
|
||||
|
||||
it("rejects connection-string SSL options that could override the verified CA configuration", () => {
|
||||
it("accepts SSL settings in the single DATABASE_URL value", async () => {
|
||||
const pool = createPostgresPool({
|
||||
env: {
|
||||
NODE_ENV: "test",
|
||||
ZHINIAN_DATA_BACKEND: "postgres",
|
||||
DATABASE_URL: "postgresql://app:secret@rds.example:5432/app?sslmode=verify-full"
|
||||
}
|
||||
});
|
||||
await pool.end();
|
||||
});
|
||||
|
||||
it("rejects unsupported connection-string SSL options", () => {
|
||||
expect(() => createPostgresPool({
|
||||
env: {
|
||||
NODE_ENV: "test",
|
||||
ZHINIAN_DATA_BACKEND: "postgres",
|
||||
DATABASE_URL: "postgresql://app:secret@rds.example:5432/app?sslmode=no-verify"
|
||||
DATABASE_URL: "postgresql://app:secret@rds.example:5432/app?sslcert=unexpected"
|
||||
}
|
||||
})).toThrow("must not contain SSL query parameters");
|
||||
})).toThrow("unsupported SSL query parameters");
|
||||
});
|
||||
|
||||
it("quotes PostgreSQL role identifiers without allowing SQL syntax injection", () => {
|
||||
|
||||
Reference in New Issue
Block a user