The earliest automatic clip-finding tools worked by scanning transcripts for specific words or phrases the creator designated as important. If you said "this is huge" or "let's go," the tool marked those timestamps as clip candidates. This approach is simple to implement and easy to explain. It is also limited in ways that become obvious quickly when you look at what it finds.
What keyword detection gets right
Keyword detection works well for content with a predictable verbal structure. Tutorial content where the creator announces the key insight ("the answer is..."), review content where they deliver a verdict ("my final take is..."), or sports commentary where specific calls are clearly voiced ("and he scores!") -- all of these have verbal signals that are reliably present when a notable moment occurs. For creators with highly structured content formats, keyword matching against a custom vocabulary can produce reasonable clip candidates.
Where it misses
Most notable moments in live content are not announced verbally before they happen. A surprise reaction, a peak in competitive game play, a moment of genuine laughter, or a strong non-verbal response from the creator all produce shareable clips without a predicting keyword. The content that generates these moments is often visually and acoustically distinct from baseline -- face visibility changes, audio energy spikes, pace changes, or sudden silence followed by a peak. None of these signals are captured by matching against a word list.
Keyword detection also produces false positives at high rates for creators who use their target vocabulary conversationally rather than as punctuation for key moments. If a gaming creator says "let's go" 30 times per stream as filler, a keyword detector marks 30 potential clips of widely varying quality. The filtering burden shifts to the creator, which partially defeats the purpose of automated detection.
Signal-based detection
Moment detection that uses audio-visual signals rather than keywords operates on a different input set. Audio energy -- measured as peak volume, variance from baseline, and duration of elevated energy -- identifies windows where something acoustically significant happened. Face visibility and expression change, tracked through the video frames, identifies windows where the creator's visual reaction was present and visible. Speech pace change, derived from the transcript timing rather than the words, identifies windows where the speaker was excited, surprised, or emotionally elevated.
These signals do not require the creator to say a specific word. They measure the acoustic and visual character of the content at each point in the recording and rank windows by how strongly the signals indicate an elevated moment. A reaction moment that produces no verbal signal but a significant audio energy spike and a face-visible window with visible expression change will score highly on this model and miss entirely in a keyword model.
The transcript still matters
Signal-based detection does not discard the transcript -- it uses it differently. The transcript provides timing structure, allows detection of silence-to-speech transitions, and gives the system the ability to ensure the clip starts at a sentence boundary rather than mid-phrase. The content of the words matters less than the pattern: where the speech starts after a silence, where the pace changes, where the creator introduces a new thought. The words carry context for trimming and labeling, but they are not the primary quality signal.
The combination -- audio energy signals, visual signals, and transcript timing -- produces clip candidates that better match the intuition an experienced editor would apply when scrubbing the same recording. It finds the moments that stand out acoustically and visually, which is how a human editor's eye and ear naturally move when scanning a timeline.