Have you ever been watching a movie from your laptop or computer, perhaps connected to a projector or TV, and wanted to move your mouse, but realised doing so meant abandoning your comfy watching spot? I have.

In that moment, I wished I had some kind of TV remote for my computer, so I decided to make one. Throughout that process, I learned about all of these things:

• 🖥️ Wayland (the display server protocol)
• 🌐 Network programming in C
• 🔒 Datagram TLS encryption
• 🤖 Android development
• 📡 The Multicast DNS (mDNS) advertising

Check it out at the links above, or keep reading to find out more about the development process.

Wayland

As I described in my very first blog post, I’m using Linux with a Wayland compositor called niri. It happens to support a Wayland protocol called wlr-virtual-pointer-unstable-v1 which allows programs to control the pointer and simulate clicks. I decided to write a simple test program that would move the mouse using this protocol. A bit of the Wayland book later and I had a working program to simply move the mouse by a few pixels then exit.

The next idea was then to hook this up to a network socket that would receive relative coordinates to move the pointer by, and mouse buttons to click, allowing remote control.

Network programming

I had been meaning to learn the basics for a while, so I was glad to have this opportunity to finally do so. I wanted to learn how to do network programming in a modern way, so I decided to write the main loop using the epoll syscall — even if it’s overkill for a small project like this with only a few sockets and total of ~30-50 Kbps peak bandwidth.

I opted for UDP datagram sockets instead of TCP, for lower latency and because there isn’t any need to retransmit lost packets of mouse deltas. Assuming little packet loss over the local network, dropping some packets shouldn’t affect the overall experience of moving the cursor.

With the networking code setup and hooked up to the virtual pointer code, I was able to move the mouse by directly using nc to issue the packets. I wrote a small test client to make the iteration process faster.

Datagram TLS encryption

With the above setup, anyone on the local network who knew the right packet format and port number could easily take control of my mouse. Now, this might seem like not much of a threat to some of you, but I live with two other Imperial Computing students, so you never know. I decided I wanted to have encryption, which led me to Datagram TLS (aka DTLS). If you haven’t guessed already, it’s TLS but for UDP.

I found a small C library which was perfect for my use case: tinydtls. It supports the pre-shared key (TLS-PSK) cipher — basically, a password — for encryption, which I thought was definitely secure enough. The usual downsides for PSK are

• You can’t differentiate users — but there’s no concept of that here
• It doesn’t have “perfect forward secrecy” i.e. an attacker can decrypt the packets by brute forcing the password at a later point — but the mouse packet data itself is meaningless without context, and the password can very easily be changed.

Also, I don’t have to deal with private key infrastructure, which is nice. Anyway, with tinydtls integrated and the addition of a password input, I had encrypted traffic working between the server and test client. I was worried halfway through writing it that I’d done this too early and that it would forever be impossible to debug anything because of the encryption, but fortunately, that was never a big headache.

Android development

Next, I started on the Android app. IntelliJ IDEA makes it simple to set up the Android SDK, which is helpful. From there, I was relying on Gemini to make some basic mockups, and then fixed them up the way I wanted¹. The UI uses the Material 3 design and I really liked how straightforward it is to edit the layout with the DSL that Jetpack Compose provides.

The main language used for modern Android development is Kotlin, which has interoperability with Java libraries. The “Bouncy Castle” Java package provides convenient DTLS client implementation which made the encryption support easy. Another consequence of using Kotlin was the ability to use Coroutines which nicely abstracts the asynchronous nature of network operations into clean, sequential code.

The first version of the app only had a manual connection input — specifying server IP address and port. But I was finding it annoying to input the IP every time I wanted to test, not to mention how unfriendly it is to end-users. It would be much better if my computer just showed up in a list of devices running the server. I knew I could do this with mDNS, so I decided to implement that next.

Multicast DNS (mDNS) advertising

mDNS is a protocol which allows devices to broadcast their services on the local network. Other devices can then know which IP and port to connect to. I implemented it into the server with the single-header mjansson/mdns library (with some minor modifications). On the Android side, it was also fairly straightforward to set up with the native Android NsdManager (Nsd = network service discovery). This feature is really what makes it feel like a proper usable app rather than just a simple programming project.

Future plans

I put the app to the test when my girlfriend visited and we were watching a movie from my computer, and it was great! One thing that was missing was a better UX for scrolling, since dragging the scroll bars manually with the cursor is a bit clunky.

A few other things I’d like to implement are:

• a libevdev backend for the virtual pointer input. At the cost of requiring sudo permissions, it would allow the server to work on any Linux desktop environment, not just Wayland compositors supporting the specific virtual pointer protocol.
• Keyboard input from the phone (I have a wireless keyboard so this wasn’t a priority).
• niri supports touch gestures, so I could perhaps handle those too.

I’d also like to make an iOS app but I don’t have any Apple devices to do that with, so if you’re reading this and you would like to give it a go, please do! Both wlr-remote and wlr-remote-control are released under the MIT license.