video-streaming

video-streaming (SyncByte) is a backend platform for ingesting large video files, transcoding them into adaptive HLS streams, and serving them efficiently. The upload flow avoids routing video bytes through the application server: clients get presigned MinIO PUT URLs per chunk, upload directly to object storage, and acknowledge each chunk's ETag back to the API. Once all chunks are confirmed, the server assembles them with MinIO's compose API and queues a transcode job. The transcoder runs FFmpeg asynchronously to produce four video quality variants and a separate audio track, each split into 6-second HLS segments. Manifests and segments live in MinIO (cold), and a promotion service moves hot content to local disk for low-latency repeated access.

Multipart Upload

Video files are too large to upload through the application server. The upload flow has four steps: initiate, presign parts, acknowledge chunks, complete. Initiating creates an UploadSession in PostgreSQL and a VideoChunk record per part (10 MB each). Presigning generates one PUT URL per requested part number directly to MinIO. Each acknowledgment records the chunk's ETag and flips its status to UPLOADED.

On completion, the server verifies all chunks are acknowledged, then calls MinIO's composeObject to assemble the ordered chunks into the final object in one atomic operation. After assembly, statObject verifies the assembled size matches the expected total. If sizes match, chunk objects are deleted and a transcode job is queued.

public CompleteMultipartResponse completeMultipartUpload(
        CompleteMultipartRequest request, User user) {

    List<VideoChunk> chunks = videoChunkRepository
            .findByUploadIdOrderByPartNumberAsc(request.uploadId());

    boolean allUploaded = chunks.stream()
            .allMatch(c -> c.getStatus() == ChunkStatus.UPLOADED);
    if (!allUploaded)
        throw new AppException("Not all chunks acknowledged", BAD_REQUEST);

    // Assemble in MinIO — one atomic compose from ordered chunk objects
    List<ComposeSource> sources = chunks.stream()
            .map(c -> ComposeSource.builder()
                    .bucket(session.getBucket())
                    .object(buildChunkKey(session.getObjectKey(),
                                         session.getUploadId(), c.getPartNumber()))
                    .build())
            .collect(Collectors.toList());
    minioClient.composeObject(ComposeObjectArgs.builder()
            .bucket(session.getBucket())
            .object(session.getObjectKey())
            .sources(sources).build());

    transcodeService.queueJob(lesson.getId(),
            lesson.getCourse().getId(), session.getObjectKey());
}

HLS Transcoding

TranscodeService runs asynchronously on a dedicated thread pool. It downloads the raw video from MinIO to a temp directory, then runs FFmpeg once per quality preset and once for the audio track. Video variants use libx264 with a fixed bitrate ceiling and buffer size; the audio track is extracted separately as AAC at 128 kbps. Each pass produces 6-second .ts segments and an HLS playlist.

The master manifest is written and uploaded to MinIO after each quality completes — not only at the end. This means a player can start streaming at 360p within minutes of upload completion while 720p and 1080p are still encoding. Lesson status is set to READY as soon as the first quality (360p) is available. If a quality fails, its partial segments are deleted and the manifest continues without it. The job retries up to three times before being marked permanently failed.

private static final List<QualityPreset> VIDEO_PRESETS = List.of(
    new QualityPreset("360p",   640,  360,  "800k",  "1600k",   800_000),
    new QualityPreset("480p",   842,  480,  "1400k", "2800k", 1_400_000),
    new QualityPreset("720p",  1280,  720,  "2800k", "5600k", 2_800_000),
    new QualityPreset("1080p", 1920, 1080,  "5000k","10000k", 5_000_000)
);

List<String> readyQualities = new ArrayList<>();
for (QualityPreset preset : VIDEO_PRESETS) {
    boolean ready = transcodeVideoQuality(lessonId, inputPath, tempDir, hlsBase, preset);
    if (ready) {
        readyQualities.add(preset.name());
        uploadMasterManifest(tempDir, hlsBase, readyQualities, audioReady);
        if (readyQualities.size() == 1) {
            setLessonStatusAndHlsKey(lessonId, VideoStatus.READY, hlsBase);
        }
    }
}

Storage Tiering & Segment Delivery

After transcoding, HLS content lives in MinIO (cold tier). StreamController serves manifests by fetching bytes directly from MinIO and returning them in the response. Segment requests are handled differently: the controller generates a presigned GET URL for the .ts object and returns HTTP 302, letting the player or CDN fetch the segment bytes directly from MinIO without touching the API server.

PromotionService moves content to hot (local disk) asynchronously when a lesson is accessed. It lists all objects under the HLS prefix in MinIO, streams each one to disk via HotStorageService.writeStream, and updates the storage tier in PostgreSQL. On subsequent requests, HotStorageService.getResource returns a FileSystemResource from disk. DiskMonitorJob watches disk usage and triggers DemotionJob to evict cold-accessed content back to MinIO-only when the disk threshold is exceeded.

// Manifests: fetch from MinIO, return inline
@GetMapping("/{lessonId}/master.m3u8")
public ResponseEntity<byte[]> masterManifest(@PathVariable UUID lessonId) {
    Lesson lesson = getReadyLesson(lessonId);
    byte[] content = fetchFromMinio(lesson.getHlsKey() + "/master.m3u8");
    return ResponseEntity.ok()
            .header(CACHE_CONTROL, "no-cache")
            .contentType(MediaType.parseMediaType("application/vnd.apple.mpegurl"))
            .body(content);
}

// Segments: redirect to presigned URL — API never touches the bytes
@GetMapping("/{lessonId}/{quality}/{segment}")
public ResponseEntity<Void> segment(...) {
    String objectKey = lesson.getHlsKey() + "/" + quality + "/" + segment;
    String presignedUrl = presignGet(objectKey);
    return ResponseEntity.status(HttpStatus.FOUND)
            .header(HttpHeaders.LOCATION, presignedUrl)
            .build();
}