Working With Videos

Working with videos

Process and analyze video content using Unbody’s Video API (powered by Mux) for video processing.

Fetch Video Assets

When you want summaries, thumbnails, and animated previews from your videos, use Unbody’s videoFile. Learn more about video in our Video API Guide.

const {
  data: { payload: videoFiles },
} = await unbody.get.videoFile
    .where(({ LessThan }) => ({
        size: LessThan(10000000), // 10 MB in bytes
    }))
    .select(
        "thumbnailUrl.png",
        "animatedImageUrl.gif",
        "autoSummary",
        "animatedImageUrl.webp",
        "size"
    )
    .exec();

Access Video Subtitles

Sometimes you want to extract subtitles from your videos along with their timing information. Here’s how to fetch and filter subtitle content.

const {
  data: { payload: subtitleFiles },
} = await unbody.get.subtitleFile
    .where(({ LessThan }) => ({
        media: {
            VideoFile: {
                mimeType: "video/mp4",
                size: LessThan(10000000), // 10 MB in Bytes
            },
        },
    }))
    .select(
        "entries.SubtitleEntry.start",
        "entries.SubtitleEntry.end",
        "entries.SubtitleEntry.text"
    )
    .exec();
 
const { entries } = subtitleFiles[0];

Analyze Video Content

Often after extracting subtitles, you’ll want to analyze what’s happening in your videos. Use extracted video transcriptions to generate structured analysis of story elements, key moments, and narrative context.

    const {
      data: { payload: subtitleFiles },
    } = await unbody.get.subtitleFile
        .where(({ LessThan }) => ({
            media: {
                VideoFile: {
                    mimeType: "video/mp4",
                    size: LessThan(10000000),
                },
            },
        }))
        .select(
            "entries.SubtitleEntry.start",
            "entries.SubtitleEntry.end",
            "entries.SubtitleEntry.text"
        )
        .exec();
 
    const { entries } = subtitleFiles[0];
 
    // Now analyze the content using generate.json
    const analysis = await unbody.generate.json(
        [
            {
                role: "user",
                content: `Analyze this story from a video transcript: ${JSON.stringify(entries)}`,
            },
            {
                role: "system",
                content: "Extract key story elements and create a structured analysis.",
            },
        ],
        {
            schema: z.object({
                storyContext: z.object({
                    location: z.string(),
                    mainCharacters: z.array(z.string()),
                    timeframe: z.string(),
                }),
                keyMoments: z.array(
                    z.object({
                        moment: z.string(),
                        significance: z.string(),
                    })
                ),
                humorElement: z.object({
                    setup: z.string(),
                    punchline: z.string(),
                }),
                overallTone: z.string(),
            }),
        }
    );

Learn more about advanced video features in our Video Api Guide.

©2024 Unbody