UFS dirhash has a limit on how much memory is used to prevent runaway
allocations. I believe the default limit (2MB) is rather conservative.
It was picked to avoid causing trouble back when machines were tiny and
dirhash was new.
For example, a maildir with 100K files takes about 3MB of memory. Because
this is over the current limit, we will fall back to linear search. This has a
detrimental impact on system wide performance.
I picked a new limit of 5MB. This allows for my one very large directory, a
few additional directories, and some spare room. It's also still reasonably
One can always change the default via sysctl, which is what I've done locally,
but I think increasing the default will reduce surprises in the future. And
for people with only small directories, there's no change. The increased limit
won't be used.
On Thu, Feb 21 2019, "Todd C. Miller" <[hidden email]> wrote:
> On Wed, 20 Feb 2019 17:00:23 -0500, "Ted Unangst" wrote:
>> I picked a new limit of 5MB. This allows for my one very large directory, a
>> few additional directories, and some spare room. It's also still reasonably
>> conservative imo.
> Yes please, this makes a huge difference with my email dirs ;-)
> From: "Ted Unangst" <[hidden email]>
> Date: Thu, 21 Feb 2019 17:25:09 -0500
> Jeremie Courreges-Anglas wrote:
> > I may be naive, but shouldn't dirhash memory be accounted with/like
> > buffer cache memory (freed when other sybsystems need it)?
> Probably. The limit exists in part because there is no backpressure.
A more generic backpressure mechanism would be welcome I guess. But
the current way it works for the page daemon and buffer cache is quite