Recently, I wanted to use OBS Studio (Open Broadcast Software) to record some of my coding sessions. As I switched from Windows to Debian a lot of things are still new to me. As I followed along there (unofficial) build instructions, I got OBS Studio up and running except the most important feature: Screen Capture.

That’s why I also tried building it from source and installing it via snap. However, audio and Window Capture just worked fine, but the Screen Capture did not. It seemed to be a display server related problem (Again, I’m fairly new to Linux), that’s why I specifically searched for terms I already came across during my search: Wayland and X11.

After reading a few user reports I conclude, that OBS is not working well together with Wayland. In fact, you won’t get OBS running properly on operating systems using Wayland as a display server protocol. However, most users revealed, that using X server helps. The loginctl command showed me, which type of display server protocol I was using: Wayland. A bit of researched revealed, that I simply needed to uncomment a single line in a configuration file to force my Debian system to use X11, which is already installed, instead of Wayland.

Simply go to /etc/gdm3/daemon.conf and uncomment the line containing WaylandEnable (at my system it was line 7) so it looks like this:


# GDM configuration storage
#
# See /usr/share/gdm/gdm.schemas for a list of available options.[daemon]
# Uncomment the line below to force the login screen to use Xorg
WaylandEnable=false# Enabling automatic login
#  AutomaticLoginEnable = true
#  AutomaticLogin = user1# Enabling timed login
#  TimedLoginEnable = true
#  TimedLogin = user1
#  TimedLoginDelay = 10[security][xdmcp][chooser][debug]
# Uncomment the line below to turn on debugging
# More verbose logs
# Additionally lets the X server dump core if it crashes
#Enable=true

Note: Depending on your setup the directory might be /etc/gdm/ and the configuration files name might be different as well. Please also keep in mind, that you need to reboot your system before changes take effect.

I hope this short article helps you on your own setup — keep coding!