True. We actually go our of our way to make the entire website path-relaitve, to make it just work in a variety of configurations, making the website portable to:
www or no www
http or https
testing or moving a site in a different folder
but 404 pages are problematic if you use sub-folders.
The base href is a possible trick.
All this ties into server configuration (via .htaccess or web.config), something we have dodged over the years because of its complexity and potential of breaking websites. What’s compex specifically is not creating a configuration, but dealing with a pre-existing configuration and interpreting what it does, particularly with mod_rewrite, or setups that rely on the combination of different server settings.
We need to bite the bullet and support server configuration, and for 404 pages support a special 404 page type that has absolute paths, conditioned to having set the correct website address, which as a consequence would break the website portability.