While working on another bug involving DISM, I came across the following odd behavior:

If seg_pdisable is set to 1, it will cause AS_PAGELOCK on DISM pages to fail and thus fall back to the slower softlock fault which will end up calling segspt_dismfault with F_SOFTLOCK:
        case F_SOFTLOCK:

                mutex_enter(&freemem_lock);
                if (availrmem < tune.t_minarmem + npages) {
                        mutex_exit(&freemem_lock);
                        return (FC_MAKE_ERR(ENOMEM));
                } else {
                        availrmem -= npages;
                }
                mutex_exit(&freemem_lock);

So, anytime we come down this path, we decrement availrmem by npages.  npages represents how many pages to softlock.  If the page(s) that we want to softlock are part of a large page, we bump up npages to the size of that page and decrement availrmem by that amount.

Say we have 200 processes doing IO to the same 256M page, everytime they do IO via the softlock path (seg_pcache is disabled) we decrement availrmem by 256M.  So, 200 processes all decrementing availrmem by 256M would result in decrementing availrmem by 50G. The more likely scenario would be that we have a 4GB DISM segment, and 200 processes all doing IO somewhere in that DISM segment and all of the 4GB is mapped with 4M pages. This is not the desired behavior.  When the seg pcache is enabled, the availrmem accounting would only be done once and we would not have this double accounting going on.  In addition, DISM memory is usually mlocked, so the pcache should be caching a big chunk of it.  

This is similar to CR 6428347 but for DISM pages instead of seg_vn pages.  An approach similar to segvn_pp_lock_anonpages would likely be a good solution.

N/A

N/A