Skip to content

experimental version of UxPlay with support for HLS video streaming (you tube movies)

Jump to bottom
fduncanh edited this page Sep 18, 2024 · 34 revisions

PLEASE TEST! Help welcomed in getting the experimental version with HLS support working well! GStreamer and HLS expertise would be useful, especially!

The experimental version of UxPlay-1.69 can be found at https://github.com/FDH2/UxPlay/tree/video2.

  • it will play a youtube video from the youtube app on an iOS client. using the airplay button. The protocol is HLS (Http Live Streaming).

issues:

  • Only Youtube video streaming from the YouTube app is tested

  • "scrub" (changing position in the video with the slider bar on the client) is not working properly.

  • a smooth transition from advertisements to the video isnt implemented. The video stops when the advertisement ends.
    Let the advertisement play on the iOS client, and just start video streaming when the content you want starts to play.

  • sometimes the audio restarts from the beginning, midway in the video.

  • use uxplay -d to see what is happening as the video plays.

Design:

The experimental UxPlay uses a third-party HLS parsing library with MIT license written in C : https://github.com/frejoel/hlsparse. The code is in UxPlay subdirectory lib/hlsparse. (REMOVED)

This supports version 7 HLS (August 2017): https://datatracker.ietf.org/doc/html/rfc8216

  • HLS support is supplied by GStreamer: all uxplay-experimental does is receive HLS playlists from the iOS client, and pass them to GStreamer (hlsdemux) for playing (playbin2). Eventually GStreamer's playbin3 may be used.

  • a C++ interface to this HLS parser is adapted from the C++ library https://github.com/air-display/apsdk-public , and is in UxPlay subdirectory lib/media_data_store. The license is GPL v3+, like UxPlay. (REMOVED)

  • The experimental version works, but needs work in terminating after the youtube video has finished and reached EOS (end of stream) .

In particular, if youtube plays an advertisement first, it is not understood how to transition to playing the main video after the ad has finished or been skipped. The GStreamer pipeline for HLS video is playbin3. Run UxPlay -d to see what is happening. Open the youtube app on the iOS client, click on the airplay button, and select a video.

the HLS AirPlay protocol:

  • There are four kind of RTSP/HTTP connections: You can see them with the -d option when UxPlay is running.

    1. "RAOP": a RTSP connection from the client to the server (UxPlay)
    2. AirPlay : a HTTP connection from the client to the server
    3. Airplay (reversed): a HTTP conection from the server to the client.
    4. HLS: a HTTP connection from the media player (Gstreamer) running on the server host to the server.

  • When UxPlay initially connects with a client, a standard RAOP RTSP connection is made from the client. GET /info and SETUP RTSP requests are made. The standard airplay non-http mirror services (raop_ntp (UDP), raop_rtp (UDP), raop_rtp_mirror (TCP)) are initialized, and raop_ntp time service is started.

  • starting a HLS video opens an Airplay HTTP connection. The raop_ntp,raop_rtp, raop_rtp_mirror services are closed, and a new pair_setup occurs.

  • After authentication, a GET /server-info HTTP request is made, to provide Airplay server options to the client.

  • Next, a second AirPlay HTTP connection is made with an "Upgrade /PTTH1.0" request, after which the connection is "reversed" (to provide an "Event" HTTP connection that allows UxPlay to send HTTP requests to the client. The connection type changes from "AirPlay" to "AirPlay (reversed)".

  • The client sends a POST /play HTTP request to the server, which starts the HLS process. The request contains a plist containing "Content-Location" for the master playlist (for YouTube it is "mlhls://localhost/master.mu38") and "Start-Position-Seconds" (a float), a uuid as well as other information.

  • The server then makes a Post /event HTTP request ("fcup_request") on the AirPlay (reverse) connection to ask the client for the HLS master Playlist data ("FCUP requests").

  • the client then sends various PUT /SetProperty?... POST /getProperty?... ,POST /rate?value=1.0000 , and GET playback-info HTTP requests.

  • next it sends a POST /action request that contain (in plist form) the master playlist mlhls://localhost/master.m3u8. The master playlist lists the media playlists and their locations (e.g. mlhls://localhost/itag/604/mediadata.m3u8). These can be EXT-X-STREAM-INF video playlists or EXT-X-MEDIA playlists (audio, subtitles, etc).

The client then sends a series of reverse-http POST /event "fcup" requests for each of the media playlists listed in the master playlist. The client sends each of them with a POST /action http request after receiving the fcup request. These may be either in regular form, or in YT-EXT-CONDENSED form (in which case they need to be adjusted before use by adding the BASE-URI specified in the YT-EXT-CONDENSED line to each #EXTINF entry). The server should only make the next fcup request after the client has responded to the previous request.

  • once the server has received all the media playlists from the client, it is ready to start the media player. The "mlhls://localhost/.." in the URI's in the master playlist must be adjusted to "http://localhost:xxxxx/...." where xxxx is the server port for the AirPlay connection. The master playlist location "http://locahost:xxx/master.m3u8" and start-position should be sent to the media player.

  • the media player will open its own http connection ("HLS") to the server, and make requests like GET /master.m3u8 for the (adjusted) master playlist; media playlists are requested by e.g. GET /itag/609/mediadata.m3u8. (Note a few of the STREAM-INF playlists have duplicate itag entries, but combine with different MEDIA playlists; it is not clear how to choose which one to send.) Only one STREAM-INF video playlist and the MEDIA playlists it references will be requested initially. Other STREAM-INF playlists (with the associated MEDIA playlists) may be requested if the video player optimizes for the internet connection.

  • There is an initial period where the media player buffers are being filled with downloaded media content from the internet. Once the buffer filling reaches 100%, the video starts playing.

  • Initially at one-second intervals, the client sends a GET /playback-info request on the AirPlay channel for "playback_data", which the server gets from the media player, including the current playing position relative to total duration, which it reports to the client. After about 2 minutes of playing, the request interval appears to change to once every thirty seconds. (It is not clear what use the client make of this information.)

Clone this wiki locally