Audio API

Unbody leverages Mux to provide audio hosting and streaming services. With Mux you can expect a reliable audio streaming experience with many more features to enhance your audio content.

Core Capabilities

• Streaming: Deliver your audios to any device. Use Mux’s multi-platform player or your preferred HLS player.
• Automatic Transcription: Generate captions for your audio files (English only).
• Static Renditions: Access static renditions of your audio files for offline viewing, direct downloads and other use-cases.

Getting Started

Prerequisites

Before you begin, make sure you have the following:

• An active Unbody account.
• Access to the Unbody GraphQL endpoint.

Fetch Your Audio

Every audio in Unbody is associated with a Mux asset. To stream an audio with Mux player, you need the playbackId associated with the asset, or you can use the hlsUrl directly with your preferred HLS player.

query {
  Get {
    AudioFile {
      playbackId
      hlsUrl
    }
  }
}

import { Unbody } from "@unbody-io/ts-client";
 
const unbody = new Unbody({
  apiKey: "your-api-key",
  projectId: "your-project-id",
});
 
const {
  data: { payload: audioFiles },
} = await unbody.get.audioFile.select("assetId", "playbackId", "hlsUrl").exec();
 
const { assetId, hlsUrl } = audioFiles[0];

Setup Your Player

Here’s an example of how you can use Mux React Player to stream your audio:

import MuxAudio from "@mux/mux-audio-react";
 
const AudioPlayer = ({ title, assetId, playbackId }) => {
  return (
    <div>
      <MuxAudio
        controls
        playbackId={playbackId}
        streamType="on-demand"
        style={{ height: "100%", maxWidth: "100%" }}
      />
    </div>
  );
};

Please refer to Mux Documentation for more guides and examples.

Auto Generated Captions (English Only)

Mux uses OpenAI’s Whisper model to automatically generate captions for your audios - read more.

Unbody stores these captions as SubtitleFile and SubtitleEntry objects, here’s how you can retrieve them:

query {
  Get {
    SubtitleFile(
      where: {
        operator: Equal
        path: ["media", "AudioFile", "originalName"]
        valueText: "your-audio-file.mp3"
      }
    ) {
      entries {
        ... on SubtitleEntry {
          start
          end
          text
        }
      }
    }
  }
}

import { Unbody } from "@unbody-io/ts-client";
 
const unbody = new Unbody({
  apiKey: "your-api-key",
  projectId: "your-project-id",
});
 
const {
  data: { payload: subtitleFiles },
} = await unbody.get.subtitleFile
  .where({
    media: {
      AudioFile: {
        originalName: "your-audio-file.mp3",
      },
    },
  })
  .select(
    "entries.SubtitleEntry.start",
    "entries.SubtitleEntry.end",
    "entries.SubtitleEntry.text"
  )
  .exec();
 
const { entries } = subtitleFiles[0];

Here are some examples of how you can leverage the auto-generated captions:

• Perform semantic search to find audio files or specific parts of audios based on your query.
• Use the ask operator to get answers based on the audio transcription.
• Use the generate operator to perform RAG (Retrieval-Augmented Generation) on audio transcription.

Furthermore, if you have the auto-generated fields feature enabled in your project, Unbody utilizes the transcriptions to generate summaries, extract keywords, named entities, and more. These additional fields are added to both the AudioFile and SubtitleFile objects.

Static Renditions

Mux generates static renditions of assets for offline viewing, direct downloads, scenarios where streaming HLS streaming is not possible, and other use cases. For audio files, Mux produces a single .m4a file. Here’s how you can access it:

query {
  Get {
    AudioFile {
      files {
        url
        name
        ext
        size
        bitrate
      }
    }
  }
}

import { Unbody } from "@unbody-io/ts-client";
 
const unbody = new Unbody({
  apiKey: "your-api-key",
  projectId: "your-project-id",
});
 
const {
  data: { payload: audioFiles },
} = await unbody.get.audioFile
  .select("files.url", "files.name", "files.ext", "files.size", "files.bitrate")
  .exec();
 
const audioFile = audioFiles[0].files[0];

User Limits

Free Plan:

• Audio files longer than 10 minutes are not supported.
• Audio files larger than 60MB are not supported.
• A maximum of 15 minutes of audio in total can be stored at once.

For an extended limit, consider upgrading your plan or contact support.

Paid Plan:

• Audio files larger than 2GB are not currently supported.

Known Issues and Limitations

• Auto-generated captions are currently available exclusively for English audio content.
• Accessing playback metrics is not currently supported.
• Direct access to the Mux API is not currently supported.