Usability of Qt5 QMediaPlayer for multimedia applications

I have developed several multimedia applications using Qt4 and Qt5. In past, when creating Karaoke Lyrics Editor with Qt4, I initially used Phonon, which was offered by Qt4 multimedia framework. Unfortunately I quickly found the issues with this framework, making it less useful for my purpose:

• It was mostly Linux-based framework, and Windows support was lacking. Notably the majority of “No audio is being heard” bugs came from Windows users.
• It lacked any diagnostics. Basically if you load an audio file which should be playable – such as MP3 file played by Windows Media Player – and it doesn’t load for some reason, all you get is an error code meaning “couldn’t load file”. You have no idea why the file couldn’t be loaded (lack of permissions? lack of codec? unsupported bitrate?)
• It was not flexible. Your soundcard does not support a specific sample rate the file is encoded in? Tough luck – the framework won’t resample it for you.
• Media position reporting could be off up to a second or even more after seeking, and was never exact for VBR MP3 files. This made it unusable for Karaoke purposes, as the exact lyric timing is required.

At that moment I abandoned Phonon and switched to proven combination of FFMpeg + SDL. Despite being hard to build (especially for Windows), it provided everything I needed: consistent cross-platform support, full diagnostic if the file couldn’t be loaded, flexible audio resampling and perfect position reporting.

So while developing a new cross-platform Karaoke player I found the new Qt Multimedia in Qt5, and QMediaPlayer based on native libraries. I have thoughtfully tested it in Qt 5.5.1 and after two months I have to abandon it again, for the similar reasons. I loved the simplified build procedure and common interface, but couldn’t deal anymore with its issues, notably:

1. Relying on standard frameworks leaves you at mercy of framework developers – and, in case of Linux, maintainers of specific Linux distro. Notably OpenSuSE is well-known of packaging a crippled GStreamer framework which can’t play even MP3s in default installation. Karaoke Player is supposed to be a user-friendly application for which the user should not be expected to know how to install codecs and configure frameworks.
2. It also had a bug on Windows (href) where half of my MP3 files just refuse to play (even with standard “player” example supplied with Qt, while Windows Media Player plays them without issues).
3. Inconsistency in supported formats. Ogg files couldn’t be played at all on Windows, but played well on Linux even on a crippled OpenSuSE. MP3s are played on Mac and (only some) on Windows, but not on Linux. Internet is full of “QMediaPlayer does not play flac/ogg/other formats”.
4. While using native frameworks is a requirement for Qt to support its dual-licensing model (and gives you the smallest distribution size), not only the playback support is limited – it is limited differently on different OS! As a result, there is no single common compressible audio or video format which is guaranteed to work on all supported platforms. So if you’re developing a communication application and need to ensure Mac users can communicate to Linux or Windows users, you better pack your own compression algorithms.
5. Troubleshooting is very limited. If QMediaPlayer plays the file, it is fine. If it doesn’t – all you get is “Failed to set source” error which tells you nothing; only on Linux it includes some GStreamer diagnostic messages. Again, some MP3 files with prepend ID3 tags are not playing on Windows (but play fine on Linux and Mac), while other MP3 files play fine on Windows too.
6. Inability to do any sound alterations. Nowhere during playback you get access to raw audio buffers which you could modify. QAudioProbe gives you the read-only access, which is good for visualization but not for effects. A solution would be combining QAudioDecoder with QAudioOutput, but QAudioDecoder is supported on very few platforms (only Windows and Linux; notably OS X is missing).

On the positive note, the timing issue seem to be fixed, and is working properly even on VBR MP3 files (at least those it could play). However the issues above make it not a good choice for a Karaoke player application. Even more, the decision to rely on platform-provided frameworks brings in the issues which cannot be resolved otherwise.