TL;DR
Here’s a form that you can use to create a bug report for H5P. Following the “More information” links inside will give you explanations if needed.
Introduction
Where there’s software, there’s bugs. Or let’s say something doesn’t work the way someone expects it to. This does not necessarily have to be a mistake made by a developer. Especially on the Web, there are a lot of different components that have to work together. And sometimes things don’t work out.
When that happens, it’s great to have a place to report a problem. If there’s a person or a group of people who can probably help fix things. These people, who may even want to help out in their spare time, should be able to reproduce the problem in order to fix it. To do that, they need the right information. You can provide that information! And that’s what this article is about.
The post was created in cooperation with NDLA.
The Problem with Reporting Problems
If you browse forums, ticket systems or github issues of open source projects, you will inevitably find posts somewhat like this:
“I did X. Now all I get is a blank screen (see screenshot [of a blank browser page]). How can this be fixed?”
How useful would you rate the following message if it was sent to a repair shop by email?
“I tried to start my motorcycle. I cannot get it moving (see picture [of a motorcycle standing next to a road]). How can this be fixed?”
A repair shop could determine the model of the bike from the photo. And maybe there are common problems with that model that could lead to a random best guess. But that’s it. No information about the age of the bike, the mileage, how it was used before, no way to inspect it, etc. You probably would not give that report too many stars.
It’s the same with software bug reports, maybe even worse. So let’s take a look at what you can do to write good bug reports. Writing a thorough and clear bug report is essential for effective communication between users and developers. A well-documented bug report helps developers understand and resolve problems efficiently. I know, I know, this is a long post. But investing some time here can help save a lot of time later. And at the end of the post, you will also find an H5P Documentation Tool template for creating your reports.
We’ll focus on H5P as a web-based software, of course 😉
Before Reporting
You probably want to get your report out right away – things should be fixed as soon as possible. But you might want to take a step back and do a few checks before you start your report.
Check a reference
H5P is versatile and adaptable across platforms, allowing for extensive customization. It’s a good idea to troubleshoot any issues by testing on a reference platform or another H5P integration. This approach helps to eliminate problems that may be platform-specific, such as using outdated versions of content types, using stylesheet overrides, or introducing additional scripts that may interfere with content. You should also consider any editor customizations that are intended to add functionality but could potentially cause problems.
For example, if you’re using H5P content and suspect low color contrast or irregular keyboard behavior, it’s a good idea to check against the sample content at https://h5p.org. This way you can ensure that troubles are all the same everywhere.
If possible, export your content and upload it to https://h5p.org or https://h5p.com (considered the optimal reference). Alternatively, you could test it in a public Moodle sandbox, such as https://sandbox.moodledemo.net/.
It’s worth noting that many bug reports in the past have been related to issues that have already been resolved (such as users running outdated versions of the content type libraries), or issues arising from local customizations. So try not to inadvertently report issues that have already been resolved or are specific to local configurations. Just be careful not to be the boy who cried wolf…
Check for duplicates
Before submitting a problem, you should check to see if it has been reported before. There are numerous platforms where such notes may exist, and a quick Google search can be a helpful first step. If you’re unsure, it’s better to be safe than sorry – it’s better to report something a second time than not at all!
Report Structure
Following a good structure for your report can help you not forget important information, and it can help developers find relevant information quickly. So let’s look at a suggestion.
Basic Information
Title
Provide a clear and concise title that summarizes the issue for effective ticket handling. While not critical, ticket platforms often rely on accurate flags to automatically place reports in the correct queue. A well-defined title helps with this process.
Description
Start by briefly outlining the problem. Clearly state your expectations and describe the actual outcome. Be specific and avoid unnecessary details.
Steps to Reproduce
Clearly outline the steps that lead to the bug. Provide a step-by-step guide for developers to reproduce the problem, including any relevant settings or configurations.
Clearly state your expected result (at each step) and contrast it with the actual result. This may not be necessary for routine steps (e.g., filling in the title field of a content type-unless there’s something wrong with that, of course), but include it when there’s a problem. This approach helps developers quickly identify deviations from expected behavior.
Example
- Create a Question Set (1.20.12) content with a True False (1.8.x) question.
- Disable the “Retry” option in either the Question Set behavior settings or the True False question behavior settings.
- Save the content.
- Launch the question set, navigate to the true/false question, and click the “Check” button without selecting “true” or “false” as the answer choice for the true/false question.
Result: You are stuck and cannot finish the question set because the “Finish” button does not appear.
Expected behavior: The “Finish” button appears to complete the question set.
Impact and Frequency
Describe the frequency of the bug and its impact on the user experience. Is it just a minor annoyance? Does it prevent something from working at all? This information will help developers prioritize and assess the severity of the problem.
Clearly state whether the problem is reproducible consistently (following a pattern) or whether it occurs intermittently only. This information helps developers identify patterns and root causes.
Environment Details
Sometimes a brief description is enough to identify a problem. Typically, however, someone will need to investigate further. This someone will try to reproduce the problem on their own machine based on your description. Providing a detailed description of your setup increases the developers’ ability to understand and resolve the problem.
H5P Content Type Name (or part of H5P)
Despite its apparent simplicity, ensuring clarity about the specific content type is critical. Confusion has arisen in the past regarding the exact identification of the content type associated with a report. It’s important to use the correct English content type name, keeping in mind that some H5P integrations, such as Lumi, may use translated content type names.
Sometimes you may only believe to know the correct English name. For example, specifying “Flashcards content type” may be ambiguous given the existence of the “Dialog Cards” content type, which is more similar to traditional flashcards than the actual “Flashcards” content type. To avoid such misunderstandings, check the original English name at https://h5p.org/content-types-and-applications before reporting a problem with a specific content type.
It can also be important to know the version number of the main library that your content uses. However, it is not obvious to users how this can be determined. And although there are some “simple” ways to do this, the understanding of “simple” is probably different here. If you provide the file (either directly or via a link), developers will find this information themselves.
Browser
Web browsers use various engines to render web pages (interpreting HTML and CSS code for the appearance of the page) and execute JavaScript (the programming language of web pages). Despite standards set by committees like the W3C, interpretation is not always straightforward. Even when a standard is clear and perfectly implemented, some browsers may go beyond it for robustness or convenience. This flexibility can result in code that works on one browser but fails on others.
Standards evolve quickly, and not every browser may have adopted all the changes, resulting in inconsistencies. In addition, browsers can implement different policies, and user configurations can further affect results. The dynamic nature of these factors underscores the need for careful consideration when developing for multiple browser environments.
So, what are you using (unless it’s just a server-side issue)? I know there are gazillions of browsers, but you’re probably using one of these …
- Chrome
- Edge
- Firefox
- Safari
It’s also important to know the version number, especially for Safari. There is usually a menu/settings item called “Help” or “About” that will give you this information.
H5P Integration
Specifying the H5P integration (or plugin) you’re using is critical. Different integrations inherently can’t behave 100% the same because they need to adapt to the host platform in different ways. Therefore, it’s important to include information about the H5P integration you’re using. For example:
- H5P plugin for Drupal 7
- H5P plugin for Drupal 10
- H5P plugin for WordPress
- H5P plugin for moodle (sometimes called “black” H5P derived from the black H5P symbol showing when adding activities to a course)
- moodle’s custom H5P integration (sometimes called “blue” H5P derived from the blue H5P symbol showing when adding activities to a course; the one with the “content bank”)
- Edlib
- Lumi Desktop
- Lumi Cloud
- …
If you have administrative access, it’s also a good idea to check the version number of the H5P integration. You can usually find this information on the plugin/component management page of your platform.
H5P Core Version
It is is possible that someone following up on your report needs information about the H5P core version that you are using. You can check the current version of the H5P core if you’re using a specialized H5P integration that allows you to choose between different versions.
moodle’s custom H5P integration
Go to “Site administration > H5P > H5P settings”. There you can find the version number of H5P core in the “h5plibraryhandler” setting (e.g. v1.24).
Device
If you’re using a desktop, just saying that should be enough. If you’re using an Apple mobile device, it can make a difference which device it is. So please be as specific as possible.
Operating system
In most cases, the operating system should not be a significant factor. If the problem is with the execution of a content type on your end, you can generally disregard information about the operating system – unless it’s iOS on a mobile phone, perhaps. However, if the problem is server-side, especially in the backend of an H5P plugin, the operating system may be relevant.
If your situation doesn’t involve iOS, feel free to skip this section. Otherwise, please specify that you are using iOS and provide its version. You can find the “software version” or “iOS version” information by navigating to “Preferences > General > About”, for example 16.6.1.
Screen Reader
This information is important when dealing with issues related to screen readers. There are several options available, including system-level options and browser plug-ins. Unfortunately, they may behave differently. The H5P Group focuses primarily on NVDA, which is widely known for its positive reputation. If you are experiencing problems with screen readers, providing details about the specific screen reader and its version will help to resolve the issue effectively.
Some of the potential screen reader options you may have include:
- Google Screen Reader (ChromeVox) (Chrome across operating systems)
- Jaws (Windows)
- Narrator (Windows)
- NVDA (Windows)
- Orca (Linux)
- Voice Over (MacOS)
Here, mentioning the version number may not be as important. But if you happen to have it at hand …
Server Software
When reporting issues related to the operation of a content type, details about the server software are generally not important, especially since such information is typically only available to platform administrators. However, if the problem may be rooted in the H5P integration (plugin) or elsewhere in the platform, providing information about the server software may be valuable. Here’s a loose ranking of the importance of such information:
- What version of PHP (or node.js in some cases) is being used?
- What type of database is used, and in what version?
- Are any services (e.g. storage, database) outsourced to the cloud, e.g. Amazon Web Services? If so, how?
- …
Technical Details
Error Messages on Screen
Include any on-screen error messages in your bug report. Copy and paste the complete error message to provide context about when and where it occurred. This additional information is critical for the developers to diagnose and resolve the problem effectively.
Browser Development Tools Information
Desktop browsers provide valuable bug reporting tools that provide a variety of information. Two particularly useful tools are the console log and the network log. These tools can help capture detailed information that can help diagnose and resolve problems.
In Chrome, Edge and Firefox, these tools are available right out of the box; in Safari, you’ll need to make sure they’re enabled (go to “Safari menu > Preferences > Advanced” and check “Show Developer menu in menu bar” if they’re not, or go to “Safari menu > Preferences > Advanced” and check “Show Web Developer features” if they’re not).
You can then toggle the development tools on and off by pressing ctrl + shift + i (or ⌥ + ⌘ + i on MacOS)
Console Log
Copying any error messages you find there when your problem occurs into your report will be one of the most important things to help solve the problem, and may even be the only clue. So please include this information right away.
- You can open the JavaScript console log by opening the development tools and clicking on the “Console” tab or by pressing
ctrl+shift+j(Chrome and Firefox) or⌥+⌘+c(MacOS). - Then reproduce your issue and check the console for error messages.
- Copy and paste the full error message.
Network Tab
Network logs can be valuable in certain cases, but unless you’re sure you need the information, don’t worry too much about them. A developer will tell you to look at the network logs if they think it will help you solve the problem. However, if you want to take a look, go ahead:
- Open the network tools by going to the development tools and clicking on the “Network” tab.
- Reproduce the issue and look for anything suspicious, typically highlighted in red.
Server Log
Including specific server logs can be useful for troubleshooting issues related to H5P integration or specific parts of the H5P core. There are situations where platform configuration or server settings may be contributing factors (e.g. file upload size limits affecting H5P’s file upload widget). In many cases, information from the browser console log is sufficient, so you can skip providing server logs unless a developer requests them.
It’s important to note that server logs are only accessible to those with sufficient administrative privileges on the server. If you do not have these privileges, you may need to work with a server administrator.
PHP Error Log
The PHP programming language, which is commonly used on servers running H5P, typically writes error messages to a log file unless this behavior is disabled. The location of the log file may vary depending on the server software and its configuration. You can find this information in your server’s php.ini file under a variable named error_log.
The error messages in the log file are preceded by a timestamp. When troubleshooting a problem, look for error messages in the log file that correspond to the time when you reproduced the error.
Server Error Log
It’s possible that H5P is not the root cause of the problem, even if it appears to manifest there. Server-level errors are usually logged in an error log on the server. The location of this error log can vary, so it’s a good idea to ask your server administrator or check the server software configuration.
In the log file, look for error messages that have a timestamp corresponding to when you reproduced the error. This can provide insight into any server-level issues that may be contributing to the problem.
Supplements
Screenshots or Screen Recordings
As mentioned above, there’s no need to include screenshots of something not displaying or a loading spinner, as they don’t provide any relevant information. However, it’s a good idea to include screenshots when reporting visual glitches, such as cut-off buttons or misaligned elements.
Screen captures can be useful because they provide additional information about the steps required to reproduce the problem. However, they may not be crucial if they result in capturing a blank screen that you want to report 🙂
Most likely, your operating system already includes tools for capturing screenshots and, in some cases, screen recordings. If not, you can easily find suitable free applications for these tasks.
Please make your screenshot or recording available online via a file hosting service of your choice, e.g. Dropbox, iCloud, etc.
Access to the Content
One of the most effective things you can do when reporting a bug is to include a link to the H5P content you are referring to. This gives developers direct access to your system, making it easier to investigate. Make sure the content is accessible without first requiring credentials.
If the problem is related to the H5P integration (plugin), it may not be possible to provide a link. Under no circumstances should you share credentials publicly. If necessary, share the credentials securely through a private channel.
Alternatively, you can download and share the problematic H5P content. This can save developers from recreating the same content locally. However, keep in mind that running it on a different platform may still result in the problem not being reproducible.
Additional Information
Include any additional details that might be relevant to understanding the bug. This could include the
- date and time the issue was discovered,
- recent changes to the server or platform or H5P integration or content type,
- other plugins that might interfere with H5P,
- …
Conclusion
Hopefully, this post can help you write better bug reports for H5P and better understand why some things are important to mention. Remember, a well-written bug report is a collaboration between users and developers, and facilitates a smoother resolution process. Providing detailed and organized information greatly improves the chances of a quick and accurate fix. And don’t forget to express your gratitude for the developer’s attention and offer to provide additional information if needed.
Happy bug reporting!
Report Template
We’re talking about H5P here, so why not put all the above information into an H5P documentation tool that can provide you with a template for your report…
The template has been created in collaboration with NDLA.