This is not strictly self-hosted per se, but I am looking for your guidance.
I am standing up a wiki. I want it to use available FOSS wiki frameworks, all our self-hosted loving things.
However: the probably of this becoming a moderately trafficked page are high. So I also need to consider scale (cloudflare, captcha, auth, do I use Vercel? etc.), light-weightness (this is a Wiki but not too massive), and all that jazz. This is also someone who won’t like alt-wikis and is a very layman, mainstream audience. So “wikipedia but SEO friendly/article style” is also a plus. It will be less deeply linked than Wikipedia across pages, but is still a repository for info that will be revisited for its different components for a long time.
People with experience spinning up or hosting trafficked wiki pages: what do I do?
Thanks gang.
If scale is a concern, then you want to be reducing drag as much as possible. Database lookups and edits can get expensive, so maybe just avoid them outright: DokuWiki only requires a web server with PHP. It keeps content as plain files, making it easy to serve up and to back up.
But it’s often said that perfection is when nothing can be taken away, so what if we remove PHP as well? In this case, we need the content to already be rendered HTML. And we can do that, since there’s a body of site generation packages that build from Markdown files. I found this one while randomly searching: https://codeberg.org/milofultz/swiki
At this point, it’s just a plain web server that dishes out static HTML files. Such simplicity will withstand most AI scrapers, because the cost per request is now absurdly low. And caching a static site is not particularly difficult. Indeed, if the sum total of the wiki content is small enough, it might even fit into something like Codeberg Pages, which means we’ve eliminated even the web server (though this would depart from c/selfhosted).
What seems to be missing from that swiki package – but is entirely feasible – would be to have all content in Markdown files and live in a Git repo, and the Git history itself is used as the Wiki history. This means your pages would retain the classic wiki change log, so that it can all be generated from a Git repo that’s small enough to keep on a floppy disk.
Simplify, then add lightness.
DokuWiki is great. Also very easy, straightforward and comes with a nice plugin ecosystem.
I guess the point about databases isn’t the entire story, though. The filesystem is a database as well. Doesn’t really matter whether you have a database system look up the article and return it, or have the filesystem find the file by it’s directories and name and return the data. That’s the same thing.
Only in the most reductionist sense would this be true. In practice, a relational database (ie has a schema and querying language) is no replacement for a hierarchical filesystem, and vice versa.
A filesystem stores binary blobs of data, and its up to each file format to define the semantics of the data contained. A database is a data structure coupled with accessors to present and cross-reference structured data. Trying to do a LEFT-JOIN on three binary files is a category error. And storing a PNG in a MariaDB table would be a sort of malpractice.
A closer comparison would be an SQL database versus a data serialization format like YAML. Both have structure and have data that can be acted upon in specific ways. Arithmetic can be performed on numeric values, and strings can be concatenated together. But neither will let you concatenate numeric values, such as 2 + 2 = “22”. You need a programming language like PHP to perform such shenanigans.
In the context of serving web content, the filesystem is the domain of the OS, meaning it has been honed by decades of experience to make it as performant as possible, built into the kernel to utilizing whatever caching tricks that make sense, and this is available irrespective of the specific userspace stack (eg LAMP) that is running. However, no mainline OS has a relational database built into the kernel and is available for a web application to use. Database engineers go through great pains to optimize a system to run as a database server.
Phrased another way, there are no mainline OS’s that omit the filesystem. So removing the need for a relational database is removing an attack surface, removing a dependency, removing another thing that can break. The point of a database is to look up pieces of data. But if a web server can just serve up a whole HTML file that includes all the data needed, then the database can be omitted and performance will be higher as a result.
Can a filesystem be used in lieu of a database? Sure but there are many things which can be done but shouldn’t in all normal circumstances. That is the crux of engineering: to select the right tool for the job.
Is this AI text? If this is AI text: Nevermind. If no: I guess we’re not really on the same page. I didn’t say relational database. I mean it’s obviously not. It’s a document-oriented one. Which is exactly the kind of database you need when processing …well… documents. If you do additional web application mumbo jumbo, you might need a relational database. But this wasn’t really about any of this. We were talking of how expensive document retrievals are…
Shorter version: instead of wiki software, use a static site generated from a git repo. It’s a valid approach. I think emacswiki may have once worked that way. Note that Gitit uses git for version management, so among other things you can copy your whole wiki with git clone. But it also has a built in web editor instead of just relying on git push.