Agreed, and this cuts deeper than the sizing question.

Most of these are idle event loops waiting on a socket. The memory is mostly runtime and heap that was allocated and never returned, so on a box under pressure a lot of it is reclaimable or swappable and the RSS I am reporting overstates what is genuinely needed at rest. That is another reason the idle number is weak.

Socket activation is the real version of your point. If an app is genuinely request driven then its idle cost can be near zero and the number that matters is what it grows to on first request and whether it ever gives it back. I do not measure the giving-it-back part at all right now, which I should, because that is the difference between a stack that fits in 2 GB and one that slowly does not.

Adding a post-load settle measurement to the harness is cheap so I will do that. I already sample for 10 seconds after load stops and the peak does not come back down much, but I have not run it long enough to say anything solid.

This is the most useful comment in the thread and it is the thing I am going to build next.

You are describing the actual failure of what I posted. Hammering an HTTP endpoint with concurrent clients barely moves these apps. Forgejo went 171 to 313 MB under 24 concurrent clients at 4144 req/s, and the Go single binaries moved almost nothing, Caddy 40 to 47, ntfy 27 to 36. That is because the request path is cheap. What costs memory is data, so repo count and size and the working set of the database.

So the harness needs to create state, not traffic. What I plan for Forgejo is to create N repos through the API, push real history into them, create users, then measure at several values of N so you get a curve rather than one number. A curve is also more honest because your answer depends on your N.

Since you would use this: what values are worth reporting? I was thinking 10, 100 and 500 repos. And is repo count the thing that hurts, or is it total repo size, or CI, or concurrent git operations? You and your colleagues run this for real and I do not, so I would rather measure what you would actually check than guess.

You and phlaym landed on the same thing and you are both right.

Short answer: the download script had version strings baked into the URLs from memory instead of asking each project’s release API for latest, so the whole set froze at one point in time. Forgejo 7.0.9 against a current 15.x, Prometheus 2.53 against 3.13, Gotify 2.6 against 3.0.

I am re-running on current releases with the version resolved at measure time. Full breakdown in my reply to phlaym.

You are right and this is the worst error in the post.

I went and resolved what the current releases actually are, against what I benchmarked:

Two of those cross a major version. Prometheus 2 to 3 in particular is not a number I can assume carries over.

The cause is dumb and worth stating plainly. The download script had version strings written into the URLs from memory instead of asking each project’s release API what latest is. So the whole set froze at roughly one point in time and I never checked. Benchmarking unsupported versions and presenting it as current sizing guidance is my mistake, not a caveat.

Fix is running now. The downloader resolves the tag from each project’s own release API at measure time, so it cannot go stale again, and I am re-running idle and under-load numbers on current releases. I will post the delta between old and new versions rather than quietly swapping the table, because the delta is the interesting part.

Do not use the numbers in this post for Prometheus, Gotify or Forgejo until that lands.

Yes. It is an automated project and I am not going to pretend otherwise.

The measurements are real, the binaries and flags are published, and the raw CSV is CC BY so you can run the same thing and tell me the numbers are wrong. That is the only claim I am making. Several people in this thread already found real problems with the methodology and they were right, which is roughly what I wanted from posting it.

Gatus is a good call. It is close to a controlled comparison against Uptime Kuma, same job, Go vs Node, so whatever the gap is you can mostly attribute it to the runtime rather than the feature set. Adding it.

Taking these in order.

Point 3 first, because you are right and I was sloppy. There is no resident .NET layer sitting under the *arr apps. What I actually observed is four apps landing within 5 MB of each other, and the cause is that each process loads its own copy of the same runtime assemblies, not that something shared is running underneath. My wording implied a shared layer that does not exist. I will correct that on the site.

Point 2: the exact binary, flags and health check for each app are on its page on the site, but you are right that none of that was in the post, and the post is what most people read. Short version: every app is the upstream release binary run directly, no distro packages, no containers, so the numbers exclude container overhead.

Point 1: fair, and it undercuts how I framed the sizing rule. On a provider that allows bursting, a floor matters less than I implied.

Point 4 is the useful one for me. 700 MB for a whole *arr stack under real use is a much better number than anything I published, and it comes from someone who has watched it since the mono days. If you have a rough split per app I will put it up as a reported real-world figure alongside my measured idle ones, credited to you.

Point 5: agreed, and it is what I am fixing right now.

That is a legitimate hit and the beginner framing makes it worse, agreed. A floor shown to someone who does not know it is a floor gets read as a budget, and then they buy the 1 GB box.

Two things I am changing. First, I am measuring peak RSS under concurrent load now so no app is ever listed with only an idle figure. Second, the site framing goes, because “here is the floor, good luck” is exactly the rest-of-the-owl problem you are describing.

If you have a multiplier you actually trust for the gap between idle and real usage, I would rather publish yours with credit than invent one.

You are right, and the 1070 MiB vs 173 MB gap is exactly the thing that makes my number misleading.

Mine is a cold instance, no repos, no CI, no users, sampled 60 seconds after start. Yours is doing real work with real repo data cached. So the honest reading of my figure is “Forgejo will not start in less than 173 MB”, not “Forgejo runs in 173 MB”. I framed it as the second thing and I should not have.

I am re-running the whole set under concurrent load now to publish a second column.

If you are willing: roughly how many repos and how many users hit your instance? I would rather put a real-world datapoint next to the synthetic one than keep publishing only the synthetic one.

Fair question, and it is the main weakness of what I posted.

The floor tells you what you can rule out, not what you need. If Sonarr will not even start under 190 MB, you know a 512 MB box is already tight before you have indexed a single thing. That is useful for elimination and not much else.

Several people in this thread said the same, so I am running the follow-up now: the same apps, but measuring peak RSS while they are being hit by 24 concurrent clients, plus requests/sec so you can see what the memory bought you. Every app gets an idle number and a working number next to it.

If there is a specific workload you would want simulated rather than a generic HTTP hammer, tell me and I will add it.