OSVDI: Video stream quality measuring with ML
The trend for cloud solutions is going on and affects also desktop PCs, which can be replaced by a remote desktop VM. This VM remote access aims to come as close as possible to the local desktop PC behavior and experience. The list of existing and relevant remote access protocols and techniques (e.g., SPICE, RDP, VNC, ICA) is comparably short. One major difference between these techniques is the lossy transmission of the graphical desktop content as a picture or video stream from the VM to the end user’s client device. Image compressions, update frequencies, and network latencies are examples of parameters influencing the video stream received at a client.
Task
What video stream, recorded at the client, is closer to the original, from a human perspective?
The student should solve the problem stated above, by providing
- different test environments, each creating a reproducible behavior and video stream, and
- a tool, measuring the similarity between two video streams (e.g., VMAF).
Approach
- Milestone I: A basic setup for a remote access protocol measurement exists, e.g.:
- Create and prepare a VM (with GPU passthrough) and a remote client.
- On the VM, create some (GPU-demanding) changes in the graphical content.
- Connect the remote client to the remote server.
- Record these changes during timeframe T on the VM (ground truth) and on the remote client.
- Compute a similarity score between these two video streams.
- Milestone II: The task of comparing a video with its video streams is complex and (may) be delegated to a Machine Learning (ML) model. The student has therefore researched about existing pre-learned models, now adapted for this use case.
- Milestone III: Variable features in the test environment are eliminated, allowing reproducible tests.
Useful previous knowledge
Network protocols like SPICE, TCP; Programming languages like Python, C; ML basics