Video Processing & Editing

Expert in FFmpeg-based video editing, processing automation, and export optimization for modern content creation workflows.

When to Use

✅ Use for:

• Automated video editing pipelines (script-to-video)
• Cutting, trimming, concatenating clips
• Adding transitions, effects, overlays
• Audio mixing and normalization
• Subtitle/caption handling
• Export optimization for platforms
• Batch video processing
• Color grading and correction

❌ NOT for:

• Real-time video editing UI (use DaVinci Resolve/Premiere)
• 3D compositing (use After Effects/Blender)
• Motion graphics animation (use After Effects)
• Basic screen recording (use OBS)

Technology Selection

Video Editing Tools

Decision tree:

Need automation? → FFmpeg
Need Python API? → MoviePy
Need frame-level control? → PyAV
Need manual editing? → DaVinci Resolve

Common Anti-Patterns

Anti-Pattern 1: Not Using Keyframe-Aligned Cuts

Novice thinking: "Just cut the video at any timestamp"

Problem: Causes artifacts, black frames, and playback issues.

Wrong approach:

bash
1# ❌ Cut at arbitrary timestamp (not keyframe-aligned)
2ffmpeg -i input.mp4 -ss 00:01:23.456 -to 00:02:45.678 -c copy output.mp4
3
4# Result: Black frames, artifacts, sync issues

Why wrong:

• Video codecs use keyframes (I-frames) every 2-10 seconds
• Non-keyframe cuts require re-encoding
• Using -c copy (stream copy) without keyframe alignment breaks playback
• GOP (Group of Pictures) structure depends on keyframes

Correct approach 1: Re-encode for precise cuts

bash
1# ✅ Re-encode for frame-accurate cutting
2ffmpeg -i input.mp4 -ss 00:01:23.456 -to 00:02:45.678 \
3  -c:v libx264 -crf 18 -preset medium \
4  -c:a aac -b:a 192k \
5  output.mp4
6
7# Frame-accurate, but slower (re-encoding)

Correct approach 2: Keyframe-aligned stream copy

bash
1# ✅ Fast cutting with keyframe alignment
2# Step 1: Find keyframes near cut points
3ffprobe -select_streams v -show_frames -show_entries frame=pkt_pts_time,key_frame \
4  -of csv input.mp4 | grep ",1$" | awk -F',' '{print $2}'
5
6# Step 2: Cut at nearest keyframes (fast, no re-encoding)
7ffmpeg -i input.mp4 -ss 00:01:22.000 -to 00:02:46.000 -c copy output.mp4
8
9# Blazing fast, no quality loss, but not frame-accurate

Correct approach 3: Two-pass for best of both worlds

bash
1# ✅ Fast seek + precise cut
2ffmpeg -ss 00:01:20.000 -i input.mp4 \
3  -ss 00:00:03.456 -to 00:01:25.678 \
4  -c:v libx264 -crf 18 -preset medium \
5  -c:a aac -b:a 192k \
6  output.mp4
7
8# -ss BEFORE -i: Fast seek to keyframe (no decode)
9# -ss AFTER -i: Precise trim (only decode needed portion)

Performance comparison:

Timeline context:

• 2010: FFmpeg required full re-encoding for cuts
• 2015: -c copy added for stream copying
• 2020: Two-pass cutting became best practice
• 2024: Hardware acceleration (NVENC) makes re-encoding viable

Anti-Pattern 2: Re-encoding Unnecessarily

Novice thinking: "Apply all edits in one FFmpeg command"

Problem: Multiple re-encodings cause cumulative quality loss.

Wrong approach:

bash
1# ❌ Re-encode for each operation (quality degradation)
2# Operation 1: Trim
3ffmpeg -i input.mp4 -ss 00:01:00 -to 00:05:00 \
4  -c:v libx264 -crf 23 temp1.mp4
5
6# Operation 2: Add audio
7ffmpeg -i temp1.mp4 -i audio.mp3 -c:v libx264 -crf 23 \
8  -map 0:v -map 1:a temp2.mp4
9
10# Operation 3: Add subtitles
11ffmpeg -i temp2.mp4 -vf subtitles=subs.srt \
12  -c:v libx264 -crf 23 output.mp4
13
14# Result: 3x re-encoding = significant quality loss

Why wrong:

• Each re-encode is lossy (even with high CRF)
• Cumulative quality loss (generation loss)
• 3x encoding time
• Wasted disk I/O

Correct approach 1: Chain operations in single command

bash
1# ✅ Single-pass encoding with all operations
2ffmpeg -ss 00:01:00 -i input.mp4 -i audio.mp3 \
3  -to 00:04:00 \
4  -vf "subtitles=subs.srt" \
5  -map 0:v -map 1:a \
6  -c:v libx264 -crf 18 -preset medium \
7  -c:a aac -b:a 192k \
8  output.mp4
9
10# Single re-encode, all operations applied at once

Correct approach 2: Use stream copy when possible

bash
1# ✅ Lossless operations with stream copy
2# Trim (stream copy)
3ffmpeg -i input.mp4 -ss 00:01:00 -to 00:05:00 -c copy temp.mp4
4
5# Add audio (stream copy video, encode audio)
6ffmpeg -i temp.mp4 -i audio.mp3 \
7  -map 0:v -map 1:a \
8  -c:v copy -c:a aac -b:a 192k \
9  temp2.mp4
10
11# Burn subtitles (must re-encode video)
12ffmpeg -i temp2.mp4 -vf subtitles=subs.srt \
13  -c:v libx264 -crf 18 -preset medium \
14  -c:a copy \
15  output.mp4
16
17# Only 1 video re-encode (for subtitles)

Quality comparison:

Anti-Pattern 3: Ignoring Color Space Conversions

Novice thinking: "Just concatenate videos together"

Problem: Color shifts, mismatched brightness, broken playback.

Wrong approach:

bash
1# ❌ Concatenate videos with different color spaces
2# clip1.mp4: BT.709 (HD), yuv420p
3# clip2.mp4: BT.601 (SD), yuvj420p (full range)
4# clip3.mp4: BT.2020 (HDR), yuv420p10le
5
6# Create concat list
7echo "file 'clip1.mp4'" > list.txt
8echo "file 'clip2.mp4'" >> list.txt
9echo "file 'clip3.mp4'" >> list.txt
10
11# Concatenate without color normalization
12ffmpeg -f concat -safe 0 -i list.txt -c copy output.mp4
13
14# Result: Color shifts between clips, broken HDR metadata

Why wrong:

• Different color spaces (BT.601 vs BT.709 vs BT.2020)
• Different pixel formats (yuv420p vs yuvj420p)
• Different color ranges (limited vs full)
• Metadata conflicts

Correct approach:

bash
1# ✅ Normalize color space before concatenation
2
3# Step 1: Analyze color space of each clip
4ffprobe -v error -select_streams v:0 \
5  -show_entries stream=color_space,color_transfer,color_primaries,pix_fmt \
6  -of default=noprint_wrappers=1 clip1.mp4
7
8# Step 2: Normalize all clips to common color space
9# Target: BT.709 (HD), yuv420p, limited range
10
11# Normalize clip1 (already BT.709)
12ffmpeg -i clip1.mp4 -c copy clip1_normalized.mp4
13
14# Normalize clip2 (BT.601 SD → BT.709 HD)
15ffmpeg -i clip2.mp4 \
16  -vf "scale=in_range=full:out_range=limited,colorspace=bt709:iall=bt601:fast=1" \
17  -color_primaries bt709 \
18  -color_trc bt709 \
19  -colorspace bt709 \
20  -c:v libx264 -crf 18 -preset medium \
21  -c:a copy \
22  clip2_normalized.mp4
23
24# Normalize clip3 (BT.2020 HDR → BT.709 SDR)
25ffmpeg -i clip3.mp4 \
26  -vf "zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=limited,format=yuv420p" \
27  -color_primaries bt709 \
28  -color_trc bt709 \
29  -colorspace bt709 \
30  -c:v libx264 -crf 18 -preset medium \
31  -c:a copy \
32  clip3_normalized.mp4
33
34# Step 3: Concatenate normalized clips
35echo "file 'clip1_normalized.mp4'" > list.txt
36echo "file 'clip2_normalized.mp4'" >> list.txt
37echo "file 'clip3_normalized.mp4'" >> list.txt
38
39ffmpeg -f concat -safe 0 -i list.txt -c copy output.mp4

Color space guide:

Anti-Pattern 4: Poor Audio Sync

Novice thinking: "Video and audio are separate, just overlay them"

Problem: Lip sync issues, audio drift, broken playback.

Wrong approach:

bash
1# ❌ Replace audio without sync consideration
2ffmpeg -i video.mp4 -i audio.mp3 \
3  -map 0:v -map 1:a \
4  -c:v copy -c:a copy \
5  output.mp4
6
7# Problems:
8# - Audio duration ≠ video duration
9# - No audio stretching/compression
10# - Drift over time

Why wrong:

• Audio and video have different durations
• No timebase synchronization
• No drift correction
• Ignores original audio sync

Correct approach 1: Stretch/compress audio to match video

bash
1# ✅ Adjust audio speed to match video duration
2
3# Get durations
4VIDEO_DUR=$(ffprobe -v error -show_entries format=duration \
5  -of default=noprint_wrappers=1:nokey=1 video.mp4)
6AUDIO_DUR=$(ffprobe -v error -show_entries format=duration \
7  -of default=noprint_wrappers=1:nokey=1 audio.mp3)
8
9# Calculate speed ratio
10RATIO=$(echo "$VIDEO_DUR / $AUDIO_DUR" | bc -l)
11
12# Stretch audio to match video (with pitch correction)
13ffmpeg -i video.mp4 -i audio.mp3 \
14  -filter_complex "[1:a]atempo=${RATIO}[a]" \
15  -map 0:v -map "[a]" \
16  -c:v copy -c:a aac -b:a 192k \
17  output.mp4

Correct approach 2: Precise offset and trim

bash
1# ✅ Sync audio with offset and trim
2
3# Audio starts 0.5s late, trim to match video
4ffmpeg -i video.mp4 -itsoffset 0.5 -i audio.mp3 \
5  -map 0:v -map 1:a \
6  -shortest \
7  -c:v copy -c:a aac -b:a 192k \
8  output.mp4
9
10# -itsoffset: Delay audio by 0.5s
11# -shortest: Trim to shortest stream

Correct approach 3: Mix multiple audio tracks with sync

bash
1# ✅ Mix dialogue, music, effects with precise timing
2
3ffmpeg -i video.mp4 -i dialogue.wav -i music.mp3 -i sfx.wav \
4  -filter_complex "
5    [1:a]adelay=0|0[dlg];
6    [2:a]volume=0.3,adelay=500|500[mus];
7    [3:a]adelay=1200|1200[sfx];
8    [dlg][mus][sfx]amix=inputs=3:duration=first[a]
9  " \
10  -map 0:v -map "[a]" \
11  -c:v copy -c:a aac -b:a 256k \
12  output.mp4
13
14# adelay: Precise millisecond timing
15# amix: Mix multiple audio streams
16# volume: Normalize levels

Audio sync checklist:

□ Verify video and audio durations match
□ Use -shortest to prevent excess audio
□ Apply adelay for precise timing offsets
□ Use atempo for speed adjustment (maintains pitch)
□ Set audio bitrate appropriately (128k-256k)
□ Test lip sync at beginning, middle, end

Anti-Pattern 5: Wrong Codec/Bitrate for Platform

Novice thinking: "One export settings for everything"

Problem: Wasted bandwidth, poor quality, rejected uploads, compatibility issues.

Wrong approach:

bash
1# ❌ Export everything at 4K 50 Mbps
2ffmpeg -i input.mp4 \
3  -c:v libx264 -b:v 50M -s 3840x2160 \
4  -c:a aac -b:a 320k \
5  output.mp4
6
7# For Instagram story: 2 GB file, rejected (max 100 MB)
8# For YouTube: Could use 10 Mbps and look identical
9# For Twitter: Exceeds bitrate limits

Why wrong:

• Platform-specific size/bitrate limits
• Over-encoding wastes bandwidth
• Wrong resolution for platform
• Incompatible codecs

Correct approach: Platform-optimized exports

YouTube (recommended settings):

bash
1# ✅ YouTube 1080p upload
2ffmpeg -i input.mp4 \
3  -c:v libx264 -preset slow -crf 18 \
4  -s 1920x1080 -r 30 \
5  -pix_fmt yuv420p \
6  -color_primaries bt709 -color_trc bt709 -colorspace bt709 \
7  -movflags +faststart \
8  -c:a aac -b:a 192k -ar 48000 \
9  youtube_1080p.mp4
10
11# YouTube 4K upload
12ffmpeg -i input.mp4 \
13  -c:v libx264 -preset slow -crf 18 \
14  -s 3840x2160 -r 60 \
15  -pix_fmt yuv420p \
16  -movflags +faststart \
17  -c:a aac -b:a 256k -ar 48000 \
18  youtube_4k.mp4

Instagram (Stories, Reels, Feed):

bash
1# ✅ Instagram Story (9:16, max 100 MB, 15s)
2ffmpeg -i input.mp4 \
3  -c:v libx264 -preset medium -crf 23 \
4  -s 1080x1920 -r 30 -t 15 \
5  -pix_fmt yuv420p \
6  -movflags +faststart \
7  -c:a aac -b:a 128k \
8  instagram_story.mp4
9
10# ✅ Instagram Reel (9:16, max 90s)
11ffmpeg -i input.mp4 \
12  -c:v libx264 -preset medium -crf 23 \
13  -s 1080x1920 -r 30 -t 90 \
14  -pix_fmt yuv420p \
15  -movflags +faststart \
16  -c:a aac -b:a 128k \
17  instagram_reel.mp4
18
19# ✅ Instagram Feed (1:1 or 4:5)
20ffmpeg -i input.mp4 \
21  -c:v libx264 -preset medium -crf 23 \
22  -s 1080x1080 -r 30 \
23  -pix_fmt yuv420p \
24  -movflags +faststart \
25  -c:a aac -b:a 128k \
26  instagram_feed.mp4

Twitter/X:

bash
1# ✅ Twitter video (max 512 MB, 2:20)
2ffmpeg -i input.mp4 \
3  -c:v libx264 -preset medium -crf 23 \
4  -s 1280x720 -r 30 -t 140 \
5  -maxrate 5000k -bufsize 10000k \
6  -pix_fmt yuv420p \
7  -movflags +faststart \
8  -c:a aac -b:a 128k \
9  twitter.mp4

TikTok:

bash
1# ✅ TikTok (9:16, max 287 MB, 10 min)
2ffmpeg -i input.mp4 \
3  -c:v libx264 -preset medium -crf 23 \
4  -s 1080x1920 -r 30 -t 600 \
5  -pix_fmt yuv420p \
6  -movflags +faststart \
7  -c:a aac -b:a 128k \
8  tiktok.mp4

Web (HTML5 video):

bash
1# ✅ Web optimized (fast load, broad compatibility)
2ffmpeg -i input.mp4 \
3  -c:v libx264 -preset medium -crf 23 \
4  -s 1920x1080 -r 30 \
5  -pix_fmt yuv420p \
6  -profile:v baseline -level 3.0 \
7  -movflags +faststart \
8  -c:a aac -b:a 128k -ar 48000 \
9  web.mp4

Platform specs table:

Export optimization checklist:

□ Use -movflags +faststart for web (progressive download)
□ Use -pix_fmt yuv420p for broad compatibility
□ Set -r 30 for most platforms (avoid variable framerate)
□ Use -preset slow for final exports (better quality)
□ Use -preset ultrafast for drafts
□ Apply -maxrate and -bufsize for streaming
□ Test playback on target platform before bulk export

Production Checklist

□ Align cuts to keyframes (or two-pass seek)
□ Chain operations in single FFmpeg command
□ Normalize color spaces before concatenating
□ Verify audio/video sync (test at multiple points)
□ Use platform-specific export presets
□ Apply -movflags +faststart for web delivery
□ Set proper color metadata (bt709 for HD)
□ Test output file on target platform
□ Keep lossless intermediate files (ProRes, FFV1)
□ Use hardware acceleration for batch jobs (NVENC, VideoToolbox)

When to Use vs Avoid

References

• /references/ffmpeg-guide.md - Complete FFmpeg command reference
• /references/timeline-editing.md - Timeline concepts, multi-track editing
• /references/export-optimization.md - Platform-specific export settings

Scripts

• scripts/video_editor.py - Cut, trim, concatenate, transitions, effects
• scripts/batch_processor.py - Parallel batch video processing

This skill guides: Video editing | FFmpeg | Timeline editing | Transitions | Export optimization | Audio mixing | Color grading | Automated video production