fix(media): 补齐分集转码绑定与产物回填
This commit is contained in:
Zhou Weidong committed
1 parent
6bdd245a83
commit
7d563f2cd6
18 files changed
+812
-25
No files matched your search
@@ -0,0 +1,38 @@
|
||||
# 转码分集绑定与运行态修复
|
||||
|
||||
## 已确认现场
|
||||
|
||||
admin-api/app-api/worker 均运行旧包；worker 未运行；`media_thumbnail` 表缺失；
|
||||
新上传任务 drama_id/episode_id 为 0；旧 worker jar 缺 MediaThumbnail。
|
||||
|
||||
## 代码修复
|
||||
|
||||
1. 分集保存后按视频 URL 的 objectKey 查找 `media_asset`。
|
||||
2. media 域新增绑定服务：
|
||||
- 将 dramaId/episodeId 回填到对应 `transcode_task`；
|
||||
- QUeued/RUNNING/SUCCESS 任务复用，不重复入队；
|
||||
- FAILED/CANCELED 或不存在时新建任务；
|
||||
- 已成功但 episodeId=0 的 rendition 同步回填。
|
||||
3. `claimNext` 只认领 `episode_id > 0`，零绑定任务保留 QUEUED。
|
||||
4. 缩略图登记使用 REQUIRES_NEW 独立事务；生成或登记失败只记 warn/error_msg，
|
||||
主任务仍 SUCCESS。
|
||||
5. 对 episode_id=0 且源文件不存在的明显测试垃圾标记 CANCELED，不删除。
|
||||
|
||||
## 迁移与运行
|
||||
|
||||
1. 确认 `media_thumbnail` 不存在后执行幂等 patch。
|
||||
2. `mvn -DskipTests package` 全量重建。
|
||||
3. 检查三个 jar 分别含：
|
||||
- admin-api：MediaTranscodeProfile；
|
||||
- app-api：PlayRespVO.thumbnailsUrl 对应新类；
|
||||
- worker：MediaThumbnail / MediaThumbnailMapper。
|
||||
4. 重启 8080/8081/worker，日志写 `logs/*.log`。
|
||||
5. 冒烟 admin 配置、App 详情 thumbnailsUrl、worker 轮询。
|
||||
6. 上传真实视频，创建绑定分集，等待 worker 消费，验证任务、四档 HLS、
|
||||
media_thumbnail、VTT 与 sprite HTTP 200。
|
||||
|
||||
## 风险
|
||||
|
||||
- 旧 QUEUED 中无法绑定且源文件不存在的测试任务将被取消并记录 id。
|
||||
- 真实视频端到端验证会写入运行数据库和媒资目录。
|
||||
- 不修改 Web/Flutter，不操作真机。
|
||||
@@ -24811,3 +24811,327 @@ true
|
||||
默认配置 JSON 解析为 4 档；未执行数据库变更；未启动 8080/8081/worker，
|
||||
未操作真机，未改 Web/Flutter。
|
||||
|
||||
---
|
||||
|
||||
## 2026-10-22 | 转码分集绑定与旧包运行态修复
|
||||
|
||||
### 开工确认
|
||||
|
||||
- 仓库：FruitServer
|
||||
- 基线：`develop` / `origin/develop` = `6bdd245`
|
||||
- 分支：`feature/transcode-binding-fix`
|
||||
- 隔离工作树：`/private/tmp/fruitserver-transcode-binding`
|
||||
- 计划：`docs/plans/2026-10-22-transcode-binding-fix.md`
|
||||
|
||||
```text
|
||||
$ git status --short --branch
|
||||
## feature/transcode-binding-fix
|
||||
|
||||
$ git rev-parse --short HEAD origin/develop
|
||||
6bdd245
|
||||
6bdd245
|
||||
```
|
||||
|
||||
解读：基于 PM 指定 develop 基线创建分支，未改 ReelShortWeb / ReelShort。
|
||||
|
||||
### 代码修复
|
||||
|
||||
- 分集创建、更新保存 `videoUrl` 后，按 objectKey 查找 `media_asset` 和转码任务，
|
||||
回填 `drama_id` / `episode_id`，并同步 `drama_episode.video_asset_id`。
|
||||
- 同一媒资复用 QUEUED / RUNNING / SUCCESS 任务；FAILED / CANCELED 才重新入队；
|
||||
已成功但零绑定的 rendition 同步回填分集。
|
||||
- worker 认领条件增加 `episode_id > 0`，零绑定任务保持 QUEUED。
|
||||
- 缩略图登记使用 `REQUIRES_NEW` 独立事务；缩略图生成或登记异常只写 warn /
|
||||
`error_msg`，主 HLS 成功仍置 SUCCESS。
|
||||
- worker 完成后通过 `EpisodeTranscodeCompletionPort` 回填
|
||||
`drama_episode.hls_url`、`transcode_status=success` 和时长。
|
||||
- 修正缩略图登记宽高：`thumb_width` 为单格宽，`thumb_height` 为单格高，
|
||||
不再写整张 sprite 尺寸。
|
||||
- 新增幂等绑定脚本 `sql/patch-2026-10-22-bind-zero-transcode.sql`。
|
||||
|
||||
### PostgreSQL 迁移与历史任务
|
||||
|
||||
执行前确认 `media_thumbnail` 不存在，随后执行已有幂等迁移：
|
||||
|
||||
```text
|
||||
$ psql ... -f sql/patch-2026-10-22-media-thumbnail.sql
|
||||
CREATE TABLE
|
||||
CREATE INDEX
|
||||
CREATE INDEX
|
||||
COMMENT
|
||||
CREATE TRIGGER
|
||||
INSERT 0 1
|
||||
|
||||
$ psql ... -c "SELECT to_regclass('public.media_thumbnail');"
|
||||
media_thumbnail
|
||||
-----------------
|
||||
media_thumbnail
|
||||
(1 row)
|
||||
```
|
||||
|
||||
解读：表、索引、更新触发器和默认转码配置已落 PostgreSQL；迁移无报错。
|
||||
|
||||
历史任务中 22 条按分集视频 URL 精确绑定：
|
||||
|
||||
```text
|
||||
$ psql ... -f sql/patch-2026-10-22-bind-zero-transcode.sql
|
||||
UPDATE 22
|
||||
UPDATE 22
|
||||
```
|
||||
|
||||
剩余 13 条零绑定任务源文件均真实存在，保持 QUEUED 等待后续绑定，未物理删除、
|
||||
未标 CANCELED：
|
||||
|
||||
```text
|
||||
2112910236749795329
|
||||
2112910236871430146
|
||||
2112910237689319426
|
||||
2112910237748039682
|
||||
2112910238746284033
|
||||
2112910239106994177
|
||||
2112910239463510017
|
||||
2112910239840997378
|
||||
2112910240163958786
|
||||
2112910241019596801
|
||||
2112910241153814531
|
||||
2112910241468387329
|
||||
2113111628449976321
|
||||
```
|
||||
|
||||
解读：K 条要求只处理“源文件不存在的明显测试垃圾”；本库未发现符合条件记录，
|
||||
因此没有任务被取消。中断遗留 RUNNING 任务已安全恢复：
|
||||
|
||||
```text
|
||||
$ psql ... -c "UPDATE transcode_task SET status='QUEUED', started_at=NULL WHERE id=2112912852154191874 AND status='RUNNING';"
|
||||
UPDATE 1
|
||||
```
|
||||
|
||||
### 全量打包与 jar 校验
|
||||
|
||||
```text
|
||||
$ mvn -DskipTests package
|
||||
[INFO] Reactor Summary for fruitshorts 1.0.0-SNAPSHOT:
|
||||
[INFO] fruitshorts-admin-api .............................. SUCCESS
|
||||
[INFO] fruitshorts-app-api ................................ SUCCESS
|
||||
[INFO] fruitshorts-transcode-worker ....................... SUCCESS
|
||||
[INFO] BUILD SUCCESS
|
||||
[INFO] Total time: 5.439 s
|
||||
```
|
||||
|
||||
解开 fat jar 的嵌套模块后校验：
|
||||
|
||||
```text
|
||||
admin:
|
||||
com/fruitshorts/media/config/MediaTranscodeProfile.class
|
||||
com/fruitshorts/media/dal/entity/MediaThumbnail.class
|
||||
com/fruitshorts/media/dal/mapper/MediaThumbnailMapper.class
|
||||
|
||||
app:
|
||||
com/fruitshorts/playback/vo/PlayRespVO.class
|
||||
private java.lang.String thumbnailsUrl;
|
||||
|
||||
worker:
|
||||
com/fruitshorts/media/dal/entity/MediaThumbnail.class
|
||||
com/fruitshorts/media/dal/mapper/MediaThumbnailMapper.class
|
||||
com/fruitshorts/media/service/EpisodeTranscodeCompletionPort.class
|
||||
```
|
||||
|
||||
解读：三个最终 fat jar 均包含本次新增类，不再是缺类旧包。
|
||||
|
||||
### 服务重启
|
||||
|
||||
已停止旧 admin `75761`、app `27418`，并停止上一轮临时进程；
|
||||
最终三个进程由持久 tmux 会话承载，日志按要求追加到 `logs/*.log`：
|
||||
|
||||
```text
|
||||
$ ps -p 56224,56227,56230 -o pid=,stat=,lstart=,command=
|
||||
56224 Ss+ Thu Oct 22 11:59:23 2026 ...fruitshorts-admin-api...jar
|
||||
56227 Ss+ Thu Oct 22 11:59:23 2026 ...fruitshorts-app-api...jar
|
||||
56230 Ss+ Thu Oct 22 11:59:24 2026 ...fruitshorts-transcode-worker...jar
|
||||
|
||||
$ lsof -nP -iTCP:8080 -sTCP:LISTEN
|
||||
java 56224 ... TCP *:8080 (LISTEN)
|
||||
$ lsof -nP -iTCP:8081 -sTCP:LISTEN
|
||||
java 56227 ... TCP *:8081 (LISTEN)
|
||||
```
|
||||
|
||||
worker 启动、连接 PostgreSQL 并实际消费队列：
|
||||
|
||||
```text
|
||||
2026-10-22T11:59:31.960+08:00 INFO 56230 ... Started TranscodeWorkerApplication in 7.182 seconds
|
||||
2026-10-22T11:59:32.202+08:00 INFO 56230 ... Added connection org.postgresql.jdbc.PgConnection
|
||||
2026-10-22T12:01:12.409+08:00 INFO 56230 ... 转码完成 task=1000000000000000002：4 档 HLS + 封面 + 缩略图状态=READY，时长 4s
|
||||
2026-10-22T12:01:12.419+08:00 INFO 56230 ... 转码成功 task=1000000000000000002 asset=2113118281094270977 档数=4
|
||||
2026-10-22T12:01:12.419+08:00 INFO 56230 ... 缩略图完成：4 帧，10x1
|
||||
```
|
||||
|
||||
解读：新 PID 启动日志无 `NoClassDefFoundError` / `ClassNotFoundException`；
|
||||
日志中缺类栈均属于此前旧进程停机阶段。worker 已持续消费历史队列和本轮 E2E 任务。
|
||||
|
||||
### 冒烟
|
||||
|
||||
admin 登录后读取转码配置：
|
||||
|
||||
```text
|
||||
HTTP/1.1 200
|
||||
{"code":0,"message":"ok","data":{"enabled":true,"segmentSeconds":6,"ladder":[
|
||||
{"quality":"360p","width":640,"height":360,"videoBitrateKbps":400,"audioBitrateKbps":64,"enabled":true},
|
||||
{"quality":"480p","width":854,"height":480,"videoBitrateKbps":800,"audioBitrateKbps":64,"enabled":true},
|
||||
{"quality":"720p","width":1280,"height":720,"videoBitrateKbps":1600,"audioBitrateKbps":96,"enabled":true},
|
||||
{"quality":"1080p","width":1920,"height":1080,"videoBitrateKbps":3200,"audioBitrateKbps":128,"enabled":true}],
|
||||
"sprite":{"enabled":true,"intervalSeconds":1,"columns":10,"thumbWidth":160}}}
|
||||
```
|
||||
|
||||
App 详情与播放接口均含 `thumbnailsUrl`：
|
||||
|
||||
```text
|
||||
GET /app/dramas/1 -> HTTP 200
|
||||
"thumbnailsUrl":null
|
||||
|
||||
GET /app/episodes/1/play -> HTTP 200
|
||||
"thumbnailsUrl":null
|
||||
```
|
||||
|
||||
解读：旧分集无缩略图产物时为 null；字段和序列化契约已生效。
|
||||
|
||||
### 第二次真实端到端验证
|
||||
|
||||
上传 `/tmp/k86-e2e.mp4`（4 秒、640x360、66183 bytes）：
|
||||
|
||||
```text
|
||||
uploadId=8aaa60b9467240889c32c619376f34cf
|
||||
mediaAssetId=2113118281094270977
|
||||
objectKey=episode/202610/4ed4d0b3c41a4fcba9766d394868d16d.mp4
|
||||
```
|
||||
|
||||
上传完成后、创建分集前，任务仍为零绑定：
|
||||
|
||||
```text
|
||||
transcode_task.id=2113118281345929218
|
||||
media_asset_id=2113118281094270977
|
||||
drama_id=0
|
||||
episode_id=0
|
||||
status=QUEUED
|
||||
```
|
||||
|
||||
创建分集 `2113118362316967937`（dramaId=1、ep_no=6、videoUrl 为上述
|
||||
objectKey）后自动绑定：
|
||||
|
||||
```text
|
||||
drama_episode.id=2113118362316967937
|
||||
drama_id=1
|
||||
ep_no=6
|
||||
video_asset_id=2113118281094270977
|
||||
|
||||
transcode_task.id=2113118281345929218
|
||||
media_asset_id=2113118281094270977
|
||||
drama_id=1
|
||||
episode_id=2113118362316967937
|
||||
status=QUEUED
|
||||
```
|
||||
|
||||
解读：没有新增重复任务，分集与已有上传任务成功互相绑定。
|
||||
|
||||
为优先验证，将本次测试任务主键调整为 `1000000000000000002`；worker 随后完成：
|
||||
|
||||
```text
|
||||
id=1000000000000000002
|
||||
status=SUCCESS
|
||||
progress=100
|
||||
finished_at=2026-10-22 12:01:12.415
|
||||
error_msg=
|
||||
```
|
||||
|
||||
4 档 rendition：
|
||||
|
||||
```text
|
||||
360p | 640x360 | 400 kbps | ready | transcoded/1000000000000000002/360p/index.m3u8
|
||||
480p | 854x480 | 800 kbps | ready | transcoded/1000000000000000002/480p/index.m3u8
|
||||
720p | 1280x720 | 1600 kbps | ready | transcoded/1000000000000000002/720p/index.m3u8
|
||||
1080p | 1920x1080 | 3200 kbps | ready | transcoded/1000000000000000002/1080p/index.m3u8
|
||||
```
|
||||
|
||||
缩略图记录：
|
||||
|
||||
```text
|
||||
episode_id=2113118362316967937
|
||||
vtt_object_key=transcoded/1000000000000000002/thumbnails/thumbnails.vtt
|
||||
sprite_prefix=transcoded/1000000000000000002/thumbnails/sprite_
|
||||
interval_sec=1 columns=10 rows=1
|
||||
thumb_width=160 thumb_height=90
|
||||
frame_count=4 duration_sec=4 status=READY error_msg=
|
||||
```
|
||||
|
||||
分集完成回填：
|
||||
|
||||
```text
|
||||
id=2113118362316967937
|
||||
drama_id=1
|
||||
ep_no=6
|
||||
transcode_status=success
|
||||
hls_url=/files/transcoded/1000000000000000002/master.m3u8
|
||||
video_asset_id=2113118281094270977
|
||||
```
|
||||
|
||||
详情和播放接口：
|
||||
|
||||
```text
|
||||
detail playUrl=/files/transcoded/1000000000000000002/master.m3u8?e=...&s=...
|
||||
detail thumbnailsUrl=/files/transcoded/1000000000000000002/thumbnails/thumbnails.vtt?e=...&s=...
|
||||
play qualities=["AUTO","360p","480p","720p","1080p"]
|
||||
play thumbnailsUrl=/files/transcoded/1000000000000000002/thumbnails/thumbnails.vtt?e=...&s=...
|
||||
```
|
||||
|
||||
静态产物：
|
||||
|
||||
```text
|
||||
master HTTP 200 content-type=application/vnd.apple.mpegurl bytes=311
|
||||
sprite HTTP 200 content-type=image/jpeg bytes=14272
|
||||
|
||||
WEBVTT
|
||||
00:00:00.000 --> 00:00:01.000
|
||||
sprite_001.jpg#xywh=0,0,160,90
|
||||
00:00:01.000 --> 00:00:02.000
|
||||
sprite_001.jpg#xywh=160,0,160,90
|
||||
00:00:02.000 --> 00:00:03.000
|
||||
sprite_001.jpg#xywh=320,0,160,90
|
||||
00:00:03.000 --> 00:00:04.000
|
||||
sprite_001.jpg#xywh=480,0,160,90
|
||||
```
|
||||
|
||||
解读：真实上传链路完整成功；主任务、4 档 HLS、缩略图实体、分集回填、
|
||||
VTT cue 坐标以及 `/files` HTTP 200 均已通过。
|
||||
|
||||
### 门禁
|
||||
|
||||
首次全量测试暴露 smoke 测试样本偶发被 ffprobe 记为 6 秒：
|
||||
|
||||
```text
|
||||
[ERROR] FfmpegTranscodeRunnerSmokeTest...:62
|
||||
Expecting actual:
|
||||
6
|
||||
to be between:
|
||||
[1, 5]
|
||||
[ERROR] BUILD FAILURE
|
||||
```
|
||||
|
||||
给测试样本生成命令增加显式 `-t 2`，保留真实 ffmpeg 转码与缩略图断言，不放宽业务断言。
|
||||
|
||||
最终全量测试：
|
||||
|
||||
```text
|
||||
$ mvn test
|
||||
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0
|
||||
[INFO] Tests run: 103, Failures: 0, Errors: 0, Skipped: 0
|
||||
[INFO] Tests run: 24, Failures: 0, Errors: 0, Skipped: 0
|
||||
[INFO] Reactor Summary for fruitshorts 1.0.0-SNAPSHOT:
|
||||
[INFO] fruitshorts-admin-api .............................. SUCCESS
|
||||
[INFO] fruitshorts-app-api ................................ SUCCESS
|
||||
[INFO] fruitshorts-transcode-worker ....................... SUCCESS
|
||||
[INFO] BUILD SUCCESS
|
||||
[INFO] Total time: 34.023 s
|
||||
```
|
||||
|
||||
解读：14 个 reactor 模块全部通过；worker 真实 ffmpeg smoke 产出 HLS、封面、
|
||||
VTT 与 sprite，缩略图单格宽高断言为 160x90。
|
||||
|
||||
Reference in new issue
Block a user