This is Shuochen from R&D research division from Flect. In this blog I would like to demonstrate how to setup PulseAudio so you can forward audio from local PC to the remote instance.
Introduction
What is audio forwarding? Why do we need it?
First of all, I would like to explain what is audio forwarding. Audio forwarding is the act of redirecting the sound stream from one device to another device. The way to achieve audio forwarding depends on the Operation System (OS), in this blog we focus on Linux based systems and Mac OS because PulseAudio is designed for the Linux system.
Now to answer the question on why do we need audio forwarding. Imagine you live in a two floor house and you have two laptops. One laptop is upstairs and one laptop is downstairs. Suppose you want both laptops to play the same music. How would you achieve that? This is done by audio forwarding, we are redirecting the audio output from one laptop to another laptop so both laptop can play the same music. The same idea can be extended to a remote device anywhere in the world that is connected to the internet.
Now suppose we have a remote cloud instance. The remote instance could be a Google instance, an AWS instance or an Oracle instance. However, they share the common characteristics of not having a sound card or microphone. Even if they do have a sound card, you cannot hear them due to physical distance is too far away! If you are developing an audio application, how can you hear what is playing on the remote instance? Again we can achieve this with audio forwarding. We stream the audio output from the remote instance to the local computer so we can hear it. It is also possible to do the reverse: we can stream our microphone input from the local computer to the remote instance.
There are multiple softwares which allow audio forwarding, with PulseAudio being one of the possible choice. The reason that I chose PulseAudio is that it is installed by default on Linux computers.
Types of audio forwarding
Before moving to explain on how to achieve audio forwarding, it is necessary to understand that there are 2 types of audio forwarding. Audio forwarding within the same network and audio forwarding for different networks.
Audio forwarding within the same network is the first example which we share the audio between laptops in the same house. The devices could be further away (for example same building), but the idea is the same. They are connected to the same network. When in the same network, the device can access to each other directly using the private IP address. Therefore, audio forwarding within the same network is easy. There are multiple ways to achieve audio forwarding: direction connection, tunneling, mDNS services and RTP protocol. In this blog I will focus on setting up audio forwarding in different networks.
Usually, it is common for devices to be in different networks. It is common for our devices at home to join a router and then connect to the internet. This means the devices are typically behind NAT (Network address translation). In other words, it is not directly accessible from the internet. This is desired for security reasons but it creates a problem for audio forwarding. If it is not directly accessible then we cannot forward any sound stream.
Therefore the first step for audio forwarding is to establish a connection between the remote instance and the local PC. There are 2 ways this can be achieved and I will explain each of the method in the next section.
Setting up connection
Port forwarding
The first method, which is straightforward in the concept but more complicated in the setup is to use port forwarding. You may already have done port forwarding before for certain games (for example Minecraft).
It is not a good idea to allow any traffic to access your local computer without any restrictions. What we will do instead, is to allow the internet to access the local computer only on one port. Because PulseAudio uses the port 4713 by default, this will be the port that will be port forwarded. Basically this means, any request for the port 4713 to your public ip address will be forwarded to your local computer.
Unfortunately, the steps of port forwarding depends on the router so there is no universal guide. The website https://portforward.com/, has a good comprehensive summary for how to do port forwarding.
After you have completed the port forwarding, please do check for the firewall and allow for access to the port 4713. The firewall setting includes both the firewall list of the router and the firewall options of the OS.
To test for connection, type telnet ip_address 4713 in the remote cloud instance.
SSH
SSH is the secure alternative for establish connection between the remote server and the client. In particular we will use SSH -R option to establish a reverse tunnel from the remote server back to the client. Normal SSH establishes the connection from the client to the server. However, we also require the reverse process for PulseAudio to work. That is, establish the connection from the server to the client. This is not difficult because usually remote instance requires SSH to connect to it anyway. We are just including additional parameters to also create a reverse tunnel.
In order to set up a reverse tunnel, run the command ssh –R port1:127.0.0.1:4713 username@ip_address when connecting from the client to the server. Port1 can be any port number of your choice, as long it is not used. 127.0.0.1 means local host, because we are forwarding the request from the localhost of the server back to the client. Finally, 4713 is the default port number used by PulseAudio. Unless you have a good reason to change it, use this port number.
Again to test the connection, type telnet ip_address 4713 in the remote cloud instance.
Installing required modules
Installing required modules should be rather simple. The required modules are SSH and PulseAudio itself. Both SSH and PulseAudio should be installed by default. If for some reason PulseAudio is not installed on the distribution, install it with sudo apt install pulseaudio
For Mac OS, install it with brew install pulseaudio
Setting up the server
Just to be clear, when I use the term server here it means the PulseAudio server, not the SSH server which you connect to. PulseAudio server needs to be the device with the actual sound device on it, otherwise we will not hear any sound output.
Edit the configuration file
After installing PulseAudio, we need to edit the configuration file. We edit the configuration file with sudo nano ~/.config/pulse/default.pa. In this file (default.pa), enable the line load-module module-native-protocol-tcp auth-anonymous=1. In other words, delete the * sign in front for the line to take effect. auth-anonymous=1 means we do not restrict on who can connect to the server. You can set the IP of the client to only allow the remote instance to connect it.
Restart the service
After changing the configuration, it is necessary to restart the PulseAudio service. For Mac OS, type brew services restart pulseaudio in the terminal.
For Linux distrubutions, first type pulseaudio -k, then type pulseauido --start
Setting up the client
Now we need to repeat the same steps to set up the PulseAudio client. In this case, PulseAudio client is the the device on the remote client, which will connect to the PulseAudio server. It is possible for multiple clients to connect to the same server, should it be required.
Edit the configuration file
Again, we need to edit the configuration file with sudo nano ~/.config/pulse/default.pa. In this file (default.pa), enable the line load-module module-native-protocol-tcp. In other words, delete the * sign in front for the line to take effect. auth-anonymous=1 is no longer required because the client side does not perform authentication.
In addition, 2 additional modules should be enabled. Module-tunnel-source and module-tunnel-sink. Module-tunnel-sink-new provides audio forwarding for the sink, which is the speaker output from the remote cloud instance (client), back to the server (local computer). Module-tunnel-source-new provides the audio forwarding of the microphone input from the server to the client.
To enable module-tunnel-sink-new module, enable the line load-module module-tunnel-sink-new server=127.0.0.1:6666 sink_name=Sound. This is assuming you have provided a reverse tunnel in SSH with the port number 6666. In the case of port forwarding, the server argument would your global IP of your local computer and the port number should be 4713.
For the load-module module-tunnel-source-new module, enable load-module module-tunnel-source-new server=127.0.0.1:6666 source_name=Mic.
Both the argument sink_name and source_name are optional. Restart the service as before.
Test the connection
That is all the steps required to set up the PulseAudio. Now any audio you play on the remote client will be forwarded back to the local computer and the any microphone input from the local computer will be forwarded to the remote client. This is very convenient for developing audio applications on the remote client.
Optional adjustment
If the PulseAudio server has multiple devices. It is possible to specify which device to use for audio forwarding. We can select the device by its index (the information is shown by pacmd list-sources or pacmd list-sinks) in this way:
pacmd set-default-sink 1, pacmd set-default-source 1
Restart the PulseAudio on both the server and client to take effect.
Reference links
Complete guide on port forwarding
https://portforward.com/
PulseAudio wiki
https://wiki.archlinux.org/title/PulseAudio#Networked_audio
PulseAudio official page
https://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/User/Modules/#module-cli-protocol-unixtcp
