i. Introduction & Table of Contents
ii. X Window System Basics
iii. Advanced Window Techniques
iv. Adding Transparency
v. Regional Geometry
vi. Basic 2D Rasterization
Feel like HN should start a collection to pay the author to finish the series, since it shows up here every few months, but the writer has (understandably) not had the time/inclination to complete it.
Author here. I left the Linux world 6 years ago, though my passion for making this kind of educational content has not faded. These days, I work in the games industry, and make videos explaining game graphics concepts.
Does anyone know of a project that puts an XML wrapper (markup) or command line interface around X Window? That way one can use or interface with an X Window "browser" without writing C.
XML interfaces are common in toolkits, look at Guile for GTK or Qt Quick/QML. One of the ancient X window toolkits was the Athena widget set (baked in most X servers, https://www.x.org/releases/X11R7.7/doc/libXaw/libXaw.html). I don't know from XML interface to design Athena GUIs.
I continue to be amazed that X is still around. I can sort of understand why it was done the way it was done back when it was first created (although even then, the concept of running the actual code remotely but the X server locally seems strange). There must be more efficient ways to organize a graphics library now; it seems like X continues to be the Linux standard from inertia.
I am amazed that people try to force limited and unfinished Wayland instead of X11. I recently had RHEL on ThinkPad T14 at work - which by default uses Wayland - yet even employer by default switches settings to X11 because EVERYTHING works there as desired - including screen sharing and/or recording.
... and for my personal stuff? I have ZERO issues with X11 on FreeBSD - why would I switch to Wayland if everything works as desired on X11? FreeBSD of course supports Wayland and you can use it there if you wish - but I just do not see the point of it. I do not have ANY screen flickering or other issues on X11.
Not to mention that A LOT of stuff is X11 only and not ported (and never will be) to Wayland.
> I am amazed that people try to force limited and unfinished Wayland
Can we just not do this useless flamewar under every similar topic’s thread? Especially that no, noone wants to force Wayland, they want something that is under active maintenance, which X11 is not anymore, since every maintainer of it left/migrated to Wayland.
> Not to mention that A LOT of stuff is X11 only and not ported (and never will be) to Wayland.
Hmm, only if the goddamn maintainers of X11 who came up with Wayland thought of that. They could have named it something like W11.. no, XWayland!
Legitimate concerns of people that have a severely degraded user experience are not flamewar. Xorg lost most of its corporate support which moved mostly to Wayland, that is true. Despite this Wayland has very poor adoption rate. The main reason for this is that Wayland has many unfixable flaws at the concept level that have been discussed here many times.
Gripes are unconstructive, though. I'm just as disenchanted as the next person, but in the FOSS world the ethos is rightly, put up or shut up.
Personally, I suspect that if the Wayland developers simply dropped maintenance of Xorg completely--no more patches, no more XWayland or at least completely disassociate XWayland from Xorg--that a community would step up and carry Xorg forward. As it is, that erstwhile community remains largely hidden, maintaining out-of-tree patches. But if that community faced the prospect of 1) being able to take back the reigns of Xorg, and 2) if not, Xorg vanishes today, then it would likely become much more motivated, organized, and proactive. (Whether it would be capable of carrying Xorg forward is a different question.) Alternatively, it might create the environment for projects like Arcan to shine.
Actually caring a tiny bit about security and not letting everything be a keylogger is not an “unfixable flaw”, there is working screen sharing for basically every mainstream desktop app, what exactly is that flaw then?
How true this is. In the 90's, at least one commercial Unix vendor shipped their systems with X11 open to the world (equivalent of "xhost +", full remote access, no security.) In the days of no firewalls, that was a fun time.
Wayland does not care about security. In fact it does not even offer standardized access control protocols. The flaw here (among many others) is that the protocol that is responsible for negotiating display output for multiple processes (which is what Wayland tries to be) should also contain standardized interfaces to exchange that output among processes, but Wayland doesn't.
Yes. Wayland is a display protocol, not a general-purpose IPC. Hence it must contain access control interfaces for buffers.
Only the compositor has access to the composited image therefore is the only program able to make screenshots and full screen sharing. Furthermore, efficient sharing of single application windows is only possible with direct memory access to the buffer which also is something the compositor does/has. The Display protocol is the only sensible place to negotiate access to GPU resources.
>The Display protocol is the only sensible place to negotiate access to GPU resources.
No, GPU resources can come from any number of other places. That's literally the purpose of direct rendering. Any applications using GPU compute can produce them without requiring a connection to the display server. There's no reason to put that in Wayland or restrict it to the compositor. There's already a much better protocol for sharing GPU buffers: Pipewire. Dbus is only there to act as the glue between a Wayland compositor and Pipewire.
No, with direct rendering the display server uses the same API as any other process that uses the GPU. The way to fix stuttering is to fix scheduling in the kernel. XGetImage is comparing apples to oranges. That doesn't use the GPU at all, that does a copy into main memory. Pipewire is complicated because it has to handle format negotiation and conversion and audio support and a lot of other things that make sense for generically transferring video between processes but are outside the scope of the display server.
Can you please knock off the conspiracy theories and flames? What are you hoping to accomplish by hurting people like this on the internet?
> The whole ecosystem is garbage and is either conceived by complete moronic retards or with the purpose of sabotage. The latter is more likely, nobody can be that stupid.
> This exactly the behaviour that I would expect from Wayland apologists, though. Your arguments are obviously wrong and pointless and the best thing you can come up with is shaming other people into self censorship.
Yeah, I'm going to shame you for that. That kind of language isn't going to convince anyone, and it's actively harmful. Please stop.
I didn't tell you what to say. I asked you nicely. Notice I used the word "please." You can make all the arguments against Wayland you want. Promoting baseless conspiracy theories and verbal abuse calling other people morons doesn't help get your argument across. It weakens your point. You're just hurting people as well as hurting your own argument by making these comments.
Hi, I'm maybe one of those moronic retards you talk about. Please, please help us out! The world is eagerly awaiting your super non-garbage solutions. Please show us the way... You could be a hero!
1. Scrap everything you have done so far and start over.
2. Use tried and tested engineering principles as basis for your development philosophy (such as "mechanism, not policy" or "Keep It Simple Stupid").
3. Choose the right target audience. The target audience of the FOSS-Desktop are neither car entertainment systems, nor Android consumer users that potentially download malware everyday. Your target audience are power users and developers that know what they are doing. If you don't agree, go work for a car manufacturer, work as an Android developer or do something else, just stay away from the FOSS-Desktop.
4. Do what developers and power users need! Just some examples: use simple and stable interfaces, use non-opaque low-level abstractions, standardize often used functionality, avoid redundant standards, don't litter my pstree with pointless daemons, use existing mechanism provided by the OS, use clear text for low bandwidth IPC, keep dependency trees as small as possible, keep build dependencies as small as possible, look at Plan9 as inspiration instead of Windows and MacOS.
Wayland already meets all those points. And Plan9 doesn't use either X11 or Wayland. Why don't you take your own advice and switch to Plan9 if Linux is causing you problems? I'll also point out that your statement about being only for developers and power users is untrue. There are plenty of good "newbie" distros like Ubuntu and Mint.
The issues with xorg that i’m aware of are that it’s awful to maintain and that it has security issues like everything being able to access your clipboard.
And xwayland exists so porting isn’t much of an issue.
But yeah, despite all that i’m on x11 still as well. I need to use proprietary nvidia drivers and I don’t want to run full gnome or whatever to make that run smoothly. I’ll probably be dead before dwm stops working on an updated system.
I don’t mean this in a flamey way, but I’ve always wondered what the importance of clipboard isolation is, given how bad of a user experience it results in.
I’m a security engineer, and I can’t think of many threat models in which an attacker can snoop clipboard contents but can’t do much worse things, which I assume is why neither macOS nor Windows has ever bothered with clipboard isolation.
So you think it is okay for a single npm install to fork a process and continuously query the clipboard and without any sort of notice just upload it to some random server? Or, even just log every single keystroke you press? Hell, it can even take screenshots of everything.
Afaik macOS doesn’t allow clipboard access in that naive way and you do get access in wayland, it is just not this “uncontrolled”.
> So you think it is okay for a single npm install to fork a process and continuously query the clipboard and without any sort of notice just upload it to some random server? Or, even just log every single keystroke you press? Hell, it can even take screenshots of everything.
No, I don't think it's "okay." I just think it's indistinguishable from running arbitrary code, which is much worse anyways.
Again, ask yourself: what is the specific threat model in which an attacker can run a process that continually polls the clipboard, but can't do anything else?
> Afaik macOS doesn’t allow clipboard access in that naive way and you do get access in wayland, it is just not this “uncontrolled”.
You can try this for yourself: the `pbcopy` and `pbpaste` utilities on macOS allow uncontrolled access to the clipboard without any entitlements. These are well-known utilities that are baked into macOS, and are widely used by tools like password managers to programmatically control the clipboard.
More generally, you can look up the NSPasteboard and UIPasteboard APIs that Apple offers on macOS and iOS. There are no special entitlements required for either.
I don’t own nvidia, but from what I read they finally managed to have proper DRM support in their drivers which was the only limiting factor for Wayland (as opposed to nvidia patching your X server binary to run the proprietary driver, Wayland in itself only depends on proper linux kernel APIs which weren’t supported by nvidia for a long time)
The technical capability does seem to be there, I was able to run Sway with the --unsupported-gpu flag for a little while, but it broke unfortunately, and my investment in Wayland isn't high enough to try to support it myself. I'm not sure if any other wlroots compositor based projects support nvidia.
Wayland is a complicated and sore topic for a lot of people, but I think I've gained some perspective over time.
From a historical perspective, the "why Wayland?" question is incredibly obvious IMO. The old Linux graphics stack consisted of the dumb Linux framebuffer and fbcon, and the separate Xorg stack with all its DDX/UMS drivers that required root-level permissions, and only provided a graphics API for ... Xorg. Your options were either Xorg, or the framebuffer, which IIRC didn't always support all of the available display modes.
Wayland was part of a wider push to rewrite the Linux graphics stack, with the new stack of KMS/DRM/DRI3/Mesa being a huge improvement over the 2000s Linux GUI stack. This came at the cost of old UMS and DRI drivers, making e.g. PowerPC Mac GPUs largely useless now (very unstable and incapable of suspend-to-RAM), while unifying mode-setting between the kernel and userland (no more praying the video card doesn't crash when ctrl-alt-f2'ing), also allowing for an arbitrary number of shared video buffers to be created (including by non GLX video applications) which makes compositors work in a not-hacked-together sense, but also allows for e.g. hardware video decoders to work in a not-hacky way.
Coupled with the addition of evdev to replace the Xorg DDX stuff like all the custom peripheral drivers, and you finally had a kernel-level API for accessing your video and input hardware. These were all prerequisites for the idea of "Wayland", decoupling the video and input layers from the display server and creating a low level API for window server things, so that you can have both small window servers (e.g. wlroots-based compositors) as well as "thick" environments (Mutter/KWin-based desktops).
Part of what makes this confusing is a lot of these benefits trickled down to Xorg itself. Nowadays evdev (and libinput) have replaced the old DDXs on Xorg, and I think most distros have switched to using xf86-video-modesetting, which could be thought of as "Wayland-style rendering on Xorg", in the sense it uses pure DRM and Mesa to render instead of GPU-specific DDX drivers.
So Xorg has gotten a lot of the benefits of Wayland's development "for free" anyway. This is intentional, as you mention, a lot of things will never be ported to Wayland. Wayland is not a replacement for X11, but an API for implementing the display server at a lower level than X11 allowed, while allowing us to implement full X11 compatibility "for free", by embedding Xorg in the form of Xwayland.
I think this makes a ton of sense for the majority of desktop-environment level compositors like Mutter (GNOME) and KWin (Plasma), which were already massive and implemented so much of their own functionality outside of Xorg, that it made sense to just let them do their own compositing. Same with GTK and Qt, which were already doing "their own rendering" outside of X.
Didn't mean for this to become an apologetic rant for Wayland, but as a "wayland skeptic" myself I feel like most anti-wayland arguments miss the forest for the trees (screen sharing not working). As an olive branch, though, I'll also add I don't think the wayland version of the big compositors like Mutter or KWin actually do much for end users, and it's pretty shameful how they've been launched with so many bugs, with no compelling user story beyond "better scaling" and "flicker".
To be fair I don't really care all that much whether my box is running something X11-based or something Wayland-based. What I care about is whether it can still do today the things it could do 15 years ago. I'm finding it extremely frustrating that the answer is tending towards "no" - even with Xorg - but my impression is that Wayland increases the "no" count somewhat.
True, though it does so using KMS/DRM adapted from Linux [0], which I would argue were created in the same spirit as Wayland, even if they're not strictly related.
In the examples of screen sharing and recording, they work on Wayland but the application needs to be aware and request permission. Zoom and Chrome both support this, AFAIK.
Actually they don't work "on Wayland". The Wayland protocol provides no mechanisms for screen sharing and it is not intended that it ever will. Screen sharing has to be done by external protocols that are either compositor specific or using a somewhat standardized dbus protocol that is completely separate from Wayland.
Not Semantics. It shows how Wayland forces a Gnome/GTK and KDE/Qt duopoly and cuts out the long tail of hobbyists and power users. The latter is the interesting part about the FOSS/Linux-Desktop ecosystem. With Wayland the FOSS-Desktop will be 100% corporate controlled. If essential things were properly standardized this wouldn't happen. Therefore it's obviously done on purpose as means for sabotage.
I remember using XFree86 in the late 90s and first ~3 years of the 2000s. When the X server started changing a lot circa 2003 it was really exciting, and I very eagerly anticipated those releases. I think Xorg is better than XFree86.
Some of the things they did off the top of my head:
* Render extension. This enabled all sorts of things, like easier drawing and scaling of pixmaps, or better font handling.
* Less monolithic build, phase out imake. This allowed iterating various components at different rates.
* Better auto configuration - less need to write a config file for the most common configurations.
XCB was also a good idea. I don't think it lived up to full potential because interest in writing code for X waned.
I think one thing that was lost to history are some of the proprietary X servers, such as those by Sun and SGI. The XFree86/Xorg lineage is missing their input.
> * Better auto configuration - less need to write a config file for the most common configurations
Better until you compile a new kernel and realize that you don't have keyboard and mouse in X because it insists using an obscure (event ?) interface. And then press the power button to restart.
Is this really necessary? Wayland blows ass and mostly functions as a launcher for x.org processes. Nobody needs, wants or asked for Wayland. It's nobody's fault but the assholes at FreeDesktop that x.org isn't accepting patches. Shit they THROW OUT good patches like xgl.
It's obvious that the plot to kill x.org is being advanced not just by insider "maintainers" who work for OTHER COMPANIES while doing NOTHING when it comes to maintenance of x.org, they're actively trying to kill it off with the help of people like you. Wayland disgusts me, and I go to great lengths to avoid trash that MS and Amazon want to shove up my ass or down my gullet.
You, your whole operation, it's as insufferable as the Rust "advocacy" that goes way beyond shilling at this point. It's not cute, fully, or clever. You're not helping.
Rust has a very real purpose. Make certain common classes of bug stop.
I don't see any real alternatives to it that go as far as Rust does for safety.
I'm not a Wayland fan due to the lack of keylogging (Needed for global shortcuts and easily writing daemons that use keyboard input) and screenshotting as standard features, but it's not like it's being forced on people, the community as a whole is demanding stronger and stronger security guarantees, "All your stuff has access to all your other stuff" doesn't seem to be good enough for many people.
Global shortcuts and screenshots are already available as standard features. The problem with your thinking is the system needs to be designed to have security by default for it to have an chance of being secure. We know from X11 that "insecure by default" is a mistake that leads to everything breaking later when someone tries to hack it to actually have security. When everything is secure by default, you can still turn off the security checks and allow everything to get access to everything else.
Can you please knock off the conspiracy theories and baseless personal attacks? Why are you doing this? You don't have to bring yourself down to that level.
Seems logical to me when the technology of the era is taken into account. Having the application code running on the most powerful machine i.e the server and then sending the graphics commands to be rendered on your specialist graphics workstation would make sense. I don't know enough about the actual efficiency of the X Window protocol though.
But don't we model this type of protocol today with the web?
Well, the web can shift rendering between server and client side depending on the developers’ needs, but graphics is done mostly on the client side with their hardware. And while things tend to cycle between “centralized” then “edge computing” and back, graphics does make more sense to stuck to the client device, especially with ever bigger screen resolutions/color spaces/refresh rates paired with low-latency requirements.
Yes, of course, high performance graphics is better closer to the display hardware.
But I've felt that the terminology of server side rendering w.r.t. the web seems to imply something else in my mind. I agree with your comment but, to me, nothing actually gets rendered on the server - the code TO render in the client gets constructed on the server, e.g. a web page, partial page etc and sent to the client to be interpreted for display in a graphical environment (the browser).
Splitting hairs, technically a bitmap graphic could be said to be rendered remotely and it's display is simply a transfer function - sorry, I realise I'm been overly pedantic.
> although even then, the concept of running the actual code remotely but the X server locally seems strange
I don't think it is strange when you consider that terminals (including some with graphical capabilities) where common back in the day, with multiple terminals connecting to a single beefy computer.
In fact there were even dedicated X terminals (basically cheap computers running an X server) in the early 90s:
I had a Tektronix dumb X terminal die with literal black smoke pouring out the back of it at work circa 1992. It turned out great cause I got a Sun as a replacement.
Part of it is, of course, that it is not just the Linux standard. It's also the FreeBSD, OpenBSD, NetBSD, DragonflyBSD, GNU HURD, Minix 3, Solaris, OpenSolaris/Illumos, AIX, HP/UX, z/OS, and OpenVMS standard.
I am sure I missed some currently-maintained OSes in there.
X is a core part of the interoperable, cross-platform, open systems world, as it has been for decades. Since most of those will never undertake large new development now, it will probably never be replaced in this, but some of those systems will keep running the world for decades to come, until the collapse of technological civilisation.
Personally, since nobody can kill off or totally replace X, my suggestion is that we collectively decide what bits we don't need any more, and cut it down to the core stuff that we need to keep.
Byte-swapped clients just were banished.
Why not get ruthless and strip out all the ancient stuff that nobody uses in the 21st century, keep just the essentials, and call the result X12?
I am no expert in this area but I gather that most of the font server stuff is obsolete now. It is probably safe to assume only 24-bit colour now. In the mid-1990s I ran X11 over DECnet on VAX/VMS. I am sure everything but TCP/IP can go now. I am willing to bet there is lots of other stuff that could be removed, leaving a smaller, more maintainable core that someone somewhere is going to need for years to come.
Why does it seem strange? It's the same as what you do when you run, say, a web browser locally. (The only difference is, we do not call the browser "web server.")
I'll be honest, it's embarrassing the amount of issues my friends who use Wayland have with basic things like sharing screens, global hotkeys for applications (push-to-talk in Discord is somehow an issue) and other things. I chose Xorg for a new install I did coming back to Linux and I'm super happy I did. Everything just works; even the stuff that I was worried wouldn't (differing resolutions on my monitors was something I expected to have to "deal with" somehow and I just never had to).
I think Wayland is an admirable project in a lot of ways, but it's practically speaking a bit of a disappointment. In 5-10 years they're likely to have sorted stuff out and it's probably going to be worth switching to by then.
In this era I have access to a lot of computers. I don’t want to run everything on the one under my desk that happens to have a GPU. It’s kind of crazy that popular frameworks are getting worse at remote rendering, and it’s one reason backend services are switching to Javascript and Wasm frontends.
Isn't the concept of remotely running graphic applications more important now than ever before? Everything runs through WSL, and different containers and virtual machines and emulators now.
I dunno. I only recently tried an X-Client with persistent sessions (that is, you can disconnect then reconnect later and all your stuff will still be running, right where you left off) and if it didn't feel like a kinda-janky barely-used-and-supported afterthought of a project, it'd have felt like the most-correct way to work in this, the year 2023 (well, I first tried it in like 2020, I guess).
... and I'm a happy Mac user, haven't been deep in Linux-on-the-Desktop land in over a decade, and still felt that way.
> There must be more efficient ways to organize a graphics library now
You can perhaps write a more efficient graphics library now, but what if in a few decades the servers are way more powerful again compared to the clients/terminals?
I don't think that is the case. Pretty much all distros have an X server available. A few distros install some desktop environment out of the box and some of those use a desktop environment that uses a Wayland compositor instead of an X server using the kernel graphics APIs, but even in those environments there is an X server (XWayland) right above the compositor (technically you can have a Wayland setup without XWayland but that'd just make the desktop less useful so i don't think there is any distro doing that by default - though even if there is, i don't think it'd be majority).
But TBH i think most distros either use X server based environments or do not provide a default OOTB environment at all.
i never said they don't have an x server available. They all do. All of the general purpose ones that provide wayland sessions still provide an x11 fallback session where both are possible, and of course for those DE/WM where wayland based solutions don't exist.
RHEL has deprecated xorg-server and will provide support for those x11 DE/WM via rootful xwayland sessions.
Can you please knock off the conspiracy theories? You don't have to bring yourself down to that level. Maybe consider how those things could actually be useful to people who don't use Red Hat?
Wayland seems more in line with Python3 than IPv6. With Valve now helping with the development and nearly all other issues addressed... seems it will be the default on all major distros within the next few years.
X runs on so many more platforms than Wayland, even if every single Linux distribution switched tomorrow there would still be good reason to support X. Python3 is strictly an improvement on every platform, I don't think they're analogous because there really is no good reason to keep Python2 around.
> X runs on so many more platforms than Wayland [...] Python3 is strictly an improvement on every platform, I don't think they're analogous because there really is no good reason to keep Python2 around
X runs on more platforms than Wayland because...it was ported to them. Just like things use Python 3 because they were ported to it.
This is also understating the reach of X I think: it's widely used in the embedded world, is seeing increasing support in BSDs, and has even been used on macOS (https://github.com/owl-compositor/owl). People have even used it to embed an entire compositor inside a GTK app (https://github.com/alexlarsson/wakefield).
That isn't to say that libwayland has a lot of Linux-isms in it, but afaik they're not really structural as much as there is lack of interest to generalize things more. Heck, the protocol-oriented architecture would even make it easier for anything Linux-esque to be removed in favor of alternative protocols.
IPv6 maybe, but Wayland's protocol choices were IMHO caused by that part of the UI stack getting nigh zero resources. So every iteration from XFree86 ended up deprecating and removing features, and Wayland turns not having features into a 'feature'; Actual standards move into Desktop environments where (some) investment was happening. The standard following economics, as it were.
There are two good books I can recommend regarding the design of X11 and how windowing, widget primitive, and compositing work:
- X Toolkit Intrinsics Programming Manual
- X Toolkit Intrinsics Reference Manual
However they have very little to do with how modern operating systems (Windows, MacOS) deal with that stuff.
As for toys, I'd recommend reading about Dear ImGui [1]. It handles everything you mentioned while being self-contained so you can see how everything works from top to bottom. Many talks and articles have been written about it though I can't vouch for any of them.
