How to create an audio/video widget extension for H5P’s editor form?

The H5P editor specification includes audio fields and video fields that you can use in your semantics.json file in order to request an input field for an audio file or a video file. This request results in an audio or video widget being rendered in the editor form. This is handled by one single module. I call this the “A/V widget”. The audio widget, for instance, will look like this by default:

Have you noticed that sometimes it’s different? Take a look at editor form of Dictation. The audio widget looks like this:

Not only does it put the upload button and the link into separate tabs, it also has an Audio Recorder tab that lets you record audio in the browser. The audio recorder is an extension. Creating such an extension for your own purposes is not rocket science. However, there is no documentation on how to do this. So let’s break it all down.

The A/V widget in the H5P Editor Core allows you to add extensions that enable you to add media to H5P contents differently. What might that entail? Obviously, you can record your audio directly. You could also create an extension that provides predefined audio or video files. You could also allow users to choose existing media from a local repository or from a remote repository somewhere on the web or the cloud. Or…

The API is pretty simple, but, as mentioned, it is not documented. You can implement your extension as complex as you like, but the minimum requirements are:

1. a library.json file (as always) to tell H5P what JavaScript, CSS and potential dependencies to load,
2. a JavaScript file with a class that represents your extension, and
3. a language file language/en.json that holds your extension’s title.

Building your extension

library.json

Your (rudimentary) library.json file could look like this:

{
  "title": "My AV widget extension",
  "machineName": "H5PEditor.MyAVWidgetExtension",
  "majorVersion": 1,
  "minorVersion": 0,
  "patchVersion": 0,
  "runnable": 0,
  "preloadedJs": [
    {
      "path": "h5p-editor-my-av-widget-extension.js"
    }
  ],
  "preloadedCss": [
    {
      "path": "h5p-editor-my-av-widget-extension.css"
    }
  ]
}

You’d probably have organized your JavaScriot/CSS files in a dist folder or something similar. You may also need preloadedDependencies and you should add a description, etc. — but this would suffice.

The language file

As with any H5P editor widget, you need to add language/en.json. This file needs to contain at least the title property with the name of your widget as the value. That’s what will be displayed as the tab label in the editor. This will be displayed as the tab label in the editor. You can, of course, add translation files for other languages as usual, but the English version is required as a fallback.

{
  "libraryStrings": {
    "title": "My AV widget extension"
  }
}

Your JavaScript file

In your JavaScript file(s), you will need to meet a couple of requirements.

Extend H5P.EventDispatcher

H5P has its own tiny event system that we need to use. In particular, we need to use the trigger method to send an event to whoever is listening to our widget instance. Here it will be the H5P core at least.

So, your MyAVWidgetExtension class will need to extend H5P.EventDispatcher.

Implement required methods

H5P core assumes five exposed methods to be implemented in your class.

appendTo(container:HTMLElement):void

The H5P editor core will call this method to allow your widget to render in the editor. The container HTML element will be passed to you, and you can append your DOM elements to it.

Note that, unlike the way the H5P core handles this for content types, the container is actually a standard HTML element, not a jQuery element. Nice!

/**
 * Append your widget's DOM to the container element.
 * @param {HTMLElement} container The container to append to.
 * @returns {void}
 */
appendTo(container) {
  // Append your DOM to the container element here
}

hasMedia():boolean

The H5P editor core will call this method when the user switches to your tab to determine if your widget is already providing a media file without requiring the user to select anything. Simply return true or false as appropriate.

/**
 * Determine if media file was selected/provided in your widget.
 * @returns {boolean} True if media file is present, false otherwise.
 */
hasMedia() {
  // Change as appropriate
  return false;
}

getMedia():object

The H5P editor Core calls this function when the user clicks the “Insert” button. The function expects to receive an object with two properties as the argument:

1. The first one with the key data needs to be a Blob with the proper MIME type for the medium, e.g. audio/mpeg.
2. The second property must have the key name and a string for the target file name. The string needs to end with a file extension such as .mp3.

Note that by default, H5P accepts .mp3, .m4a, .wav, and .ogg for audio, and .mp4, .webm or .ogv for video. However, the list of permitted suffixes can be customized in the H5P integration, so that additional suffixes may be possible or not all of them may be possible. Unfortunately, this cannot be queried on the client side.

Note that the MIME type of the Blob must match its file extension; otherwise, the H5P editor core will reject the file.

/**
 * @typedef {object} MediaFile
 * @property {Blob} data - Blob with proper MIME type (e.g., audio/mpeg).
 * @property {string} Filename usually ending with .mp3, .m4a, .wav, .ogg, mp4, webm, or ogv.
 */

/*
 * Get the media file provided in your widget.
 * @returns {MediaFile} Media file data.
 */
getMedia() {
  /*
   * Just an example! Change as appropriate.
   * Note that by default, H5P accepts mp3, m4a, wav, and ogg for audio, and mp4, webm or ogv for video,
   * but the list of allowed extensions could be changed in the H5P integration.
   * Also note that the MIME type must match the file extension, of course.
   */
  return {
    data: new Blob([], { type: 'audio/mpeg' }),
    name: 'audio.mp3',
  };
}

pause():void

The H5P editor core calls this method when the user switches tabs. The editor expects the extension to stop what it is doing. For example, in the existing AudioRecorder extension, the recording stops.

Note that this may not be relevant to your extension, but the H5P editor core will still expect the pause method to be present.

/**
 * Pause whatever the widget is doing. Note that it's mandatory to have this method.
 * @returns {void}
 */
pause() {
  // Implement as required.
}

reset():void

The H5P editor core calls this method when the audio or video dialog is closed. The editor expects the extension to return to its initial state so that the user has a fresh start when opening the instance tab again.

Note that this may not be relevant to your extension, but the H5P Editor Core will still expect it to be present.

/**
 * Reset the widget to its initial state. Note that it's mandatory to have this method.
 * @returns {void}
 */
reset() {
  // Implement as required.
}

Implement expected behavior

The H5P editor core expects you to trigger a hasMedia event via the H5P.EventDispatcher when the availability of a media file changes in your extension. The event data should be set to true if a media object is selected or set, and to false if a media object was deselected or none is set. In the former case, the AV widget enables the “Insert” button for the user. In the latter case, the AV widget disables the button.

this.trigger('hasMedia', true); // Signal that a media file is selected/provided

this.trigger('hasMedia', false); // Signal that no media file is selected/provided

Add your extension to the AV widget

Finally, you need to inform H5P editor core’s AV widget that your extension is available. Assuming that your class is named MyAVWidgetExtension and that this is also the name of the extension that should be put in semantics.json, you just need to add it to the global H5PEditor object and its  AV property:

H5PEditor.MyAVWidgetExtension = MyAVWidgetExtension;
H5PEditor.AV.MyAVWidgetExtension = MyAVWidgetExtension;

Your CSS

Do whatever you need in your CSS file 🙂

Using your extension …

…in a content type directly

Simply add your extension to the list of required editor libraries in the library.json file of the content type:

{
  "editorDependencies": [
    {
      "machineName": "H5PEditor.MyAVWidgetExtension",
      "majorVersion": 1,
      "minorVersion": 0
    }
  ]
}

Then add the widgetExtensions property to your audio field or video field and enter the name of your extension in the extensions list:

{
  "name": "myAudioField",
  "type": "audio",
  "label": "My audio field",
  "widgetExtensions": [
    "MyAVWidgetExtension"
  ]
}

The H5P Editor core will then load all the editor dependency libraries including yours and instantiate them as extensions for the media fields.

…in a content type indirectly

There will be cases where you cannot change the content type, but you will still want your extension to be available for a media field. Perhaps you want it to be available for any audio field. For example, this could be useful if your system provides access to an audio library, allowing you to fetch files rather than upload them.

That’s possible as well. It’s specific to your system, so I won’t describe it in detail here. However, you can use H5P’s hooks to accomplish this:

1. Use the alter_scripts hook to add your extension’s JavaScript file(s)
2. Use the alter_styles hook to add your extension’s CSS file(s)
3. Use the alter_semantics hook to amend (or add) the widgetExtensions property and include your extension.