[wpe-2.46][MSE] Video playback does not start when switching visibility and muted state from 'hidden' and 'muted' to 'visible' and 'unmuted'

[filipe-norte]

When setting up a video element with 'visibility = hidden' and 'muted = true', switching it to 'visibility = visible' and 'muted = false' after pipeline is prerolled and ready to play, will keep the pipeline in the paused state rather than switching to playing state.

The attached player-mute.zip test app reproduces this issue.

Analysis

Logs show that the media player is not recognizing that it should be playing:

[WPEWebKit:Media:-] HTMLMediaElement::updatePlayState(2E03E3F2) shouldBePlaying = true, playerPaused = true 

0:00:04.121967004    27    0x4a920 DEBUG              webkitmse MediaPlayerPrivateGStreamerMSE.cpp:158:play:<MSE-media-player-1> Play requested

0:00:04.121998449    27    0x4a920 DEBUG              webkitmse MediaPlayerPrivateGStreamerMSE.cpp:469:updateStates:<MSE-media-player-1> shouldBePlaying = false, m_isPipelinePlaying = false, is seeking false 

The fact that 'shouldBePlaying = false' is due the 'm_isPausedByViewport = true' in the MediaPlayerPrivateGStreamerMSE::updateStates() call, not allowing the pipeline to change state. Checking the sequence around this variable shows:

0:13:46.121777424    27    0x4a920 INFO       webkitmediaplayer MediaPlayerPrivateGStreamer.cpp:4348:setVisibleInViewport:<MSE-media-player-7> Muted video player no longer visible in viewport
0:13:46.121824572    27    0x4a920 DEBUG      webkitmediaplayer MediaPlayerPrivateGStreamer.cpp:4359:setVisibleInViewport:<MSE-media-player-7> Media element is muted and not visible in viewport, pausing it to save resources. Will resume afterwards to VOID_PENDING state.

0:13:46.402898924    27    0x4a920 INFO       webkitmediaplayer MediaPlayerPrivateGStreamer.cpp:4348:setVisibleInViewport:<MSE-media-player-7> Un-muted video player visible in viewport

0:13:48.582387315    27    0x4a920 DEBUG              webkitmse MediaPlayerPrivateGStreamerMSE.cpp:471:updateStates:<MSE-media-player-7> shouldBePlaying = false, m_isPipelinePlaying = false, is seeking false

Although player is visible and unmuted, we remain with 'shouldBePlaying = false'.

Checking MediaPlayerPrivateGStreamer::setVisibleInViewport(), the following condition would seem incomplete, as it causes an early return from the function, not allowing setting 'm_isPausedByViewport = false' (and other relevant calls) when we are not muted and are visible.

    if ((player && !player->isVideoPlayer()) || !m_isMuted)
        return; 

However, changing the condition around '!m_isMuted' to consider the 'm_isPausedByViewport' and 'visible' state to allow correct transition to occur is not sufficient, as the variable 'm_invisiblePlayerState' does not properly reflect the state we should be in. This is because this variable is only set in MediaPlayerPrivateGStreamer::changePipelineState() (other place than in setVisibleInViewport()), but the changePipelineState() will never be called from MediaPlayerPrivateGStreamerMSE::updateStates(), because 'm_isPausedByViewport = true'.

Therefore, this needs checked in more details considering the wider impact changes around this may have.

This does not happen with wpe-2.38 as the updateStates() function does not restrict state changes based on the viewport visibility.

[vivienne-w]

I've reproduced this on upstream, and it looks like suspending the player is also in general a bit inconsistent currently (e.g. only suspending correctly if we mute, then hide, but not the other way around). I'm working on a fix for these issues.