Video API

Unbody leverages Mux Video to provide video hosting and streaming services. With Mux you can expect a reliable video streaming experience with many more features to enhance your video content.

Core Capabilities

• Streaming: Deliver your videos to any device. Use Mux’s multi-platform player or your preferred HLS player.
• Automatic Transcription: Generate captions for your videos automatically (English only)
• Instant Thumbnail Generation: Obtain thumbnails and animated images directly from your videos.
• Static Renditions: Access static MP4 renditions of your video files for offline viewing, direct downloads, and other use-cases.

Usage

Example 1 - Fetch Your Video

Every video in Unbody is associated with a Mux asset. To stream a video with Mux player, you need the playbackId associated with the asset, or you can use the hlsUrl directly with your own HLS player of choice.

await unbody.get
            .videoFile
            .select("playbackId", "hlsUrl")
            .exec();

query {
  Get {
    VideoFile {
      playbackId
      hlsUrl
    }
  }
}

Example 2 - Setup Your Player

You can use the playbackId to setup a Mux player for your video. Alternatively, you can use the hlsUrl with your preferred HLS player.

Here are some examples:

import MuxVideo from "@mux/mux-video-react";
 
const VideoPlayer = ({ title, playbackId }) => {
  return (
    <div>
      <MuxVideo
        controls
        playbackId={playbackId}
        streamType="on-demand"
        style={{ height: "100%", maxWidth: "100%" }}
      />
    </div>
  );
};

 
<script src="https://cdn.jsdelivr.net/npm/hls.js@1"></script>
 
<video id="video"></video>
<script>
  var video = document.getElementById('video');
  var hlsUrl = 'your-hls-url.m3u8';
  
  if (video.canPlayType('application/vnd.apple.mpegurl')) {
    video.src = hlsUrl;
  } else if (Hls.isSupported()) {
    var hls = new Hls();
    hls.loadSource(hlsUrl);
    hls.attachMedia(video);
  }
</script>

Please refer to Mux Documentation for more guides and examples.

Example 3 - Get Thumbnails and Animated Images

To retrieve a thumbnail or an animated image of your video, utilize the URLs provided in the thumbnailUrl and animatedImageUrl fields respectively. Customize these images using query string parameters to meet your specific requirements. For instance, you can adjust parameters such as width, height for the thumbnail:

{thumbnailUrl}?time=10&width=800

Similarly, for the animated image:

{animatedImageUrl}?start=10&end=15&width=320&fps=24

For more detailed guides, please consult the Mux Documentation.

Below are the steps to retrieve the thumbnail and animated image URLs:

const { data: { payload: videoFiles }, } = await unbody.get
                                                       .videoFile
                                                       .select("thumbnailUrl.png", "animatedImageUrl.gif")
                                                       .exec();
 
const { thumbnailUrl: { png }, animatedImageUrl: { gif }, } = videoFiles[0];

query {
  Get {
    VideoFile {
      thumbnailUrl {
        png
      }
      animatedImageUrl {
        gif
      }
    }
  }
}

You can integrate the thumbnail URLs directly into your application as placeholders for the video. Below is an example demonstrating how to utilize the animated image URL with the Mux React Player:

<MuxVideo
  {
    /* Other props */
  }
  poster={animatedImageUrl}
  placeholder={animatedImageUrl}
/>

Example 4 - Auto Generated Captions (English Only)

Mux uses OpenAI’s Whisper model to automatically generate captions for your videos - read more.

Unbody stores these captions as SubtitleFile and SubtitleEntry objects, here’s how you can retrieve them:

const { data: { payload: subtitleFiles } } = await unbody.get
                                                          .subtitleFile
                                                          .where({
                                                            media: {
                                                              VideoFile: {
                                                                originalName: "your-video.mp4",
                                                              },
                                                            },
                                                          })
                                                          .select(
                                                            "entries.SubtitleEntry.start",
                                                            "entries.SubtitleEntry.end",
                                                            "entries.SubtitleEntry.text"
                                                          )
                                                          .exec();
 
const { entries } = subtitleFiles[0];

query {
  Get {
    SubtitleFile(
      where: {
        operator: Equal
        path: ["media", "VideoFile", "originalName"]
        valueText: "your-video.mp4"
      }
    ) {
      entries {
        ... on SubtitleEntry {
          start
          end
          text
        }
      }
    }
  }
}

Here are some examples of how you can leverage the auto-generated captions:

• Perform semantic search to find videos or specific parts of videos based on your query.
• Use the ask operator to get answers based on the video transcription.
• Use the generate operator to perform RAG (Retrieval-Augmented Generation) on video transcriptions.

Furthermore, if you have the auto-generated fields feature enabled in your project, Unbody utilizes the transcriptions to generate summaries, extract keywords, named entities, and more. These additional fields are added to both the VideoFile and SubtitleFile objects.

Example 5 - Static Renditions

Mux generates multiple MP4 renditions of video files in different resolutions and bitrates for offline viewing, direct downloads, scenarios where streaming HLS streaming is not possible, and other use-cases. Here’s how you can access these files:

const { data: { payload: videoFiles }, } = await unbody.get
                                                       .videoFile
                                                       .select("files.url", "files.name", "files.width", "files.height")
                                                       .exec();
 
const mp4Files = videoFiles[0].files;

query {
  Get {
    VideoFile {
      files {
        url
        name
        width
        height
      }
    }
  }
}

User Limits

Known Issues and Limitations

• Auto-generated captions are currently available exclusively for English videos.
• Subtitles embedded within video files are not stored and are unavailable via our API.
• Accessing playback metrics is not currently supported.
• Direct access to the Mux API is not currently supported.