Just a short blog post: after merging OpenOffice.org on a Hardened Gentoo machine today, I was unable to boot OpenOffice.org Writer (or any of the other OOo programs). While the solution isn’t all that pretty, it is rather simple.
The problem has to do with OpenOffice.org throwing out the following error when trying to boot in Hardened Gentoo:
terminate called after throwing an instance of ‘std::bad_alloc’
what(): std::bad_alloc
Turns out this has to do with the way OpenOffice.org tries to work against the mprotect restrictions. You can lift those restrictions by using paxctl (emerge -av paxctl) in the following way:
# check for current PaX settings:
paxctl -v /usr/lib/openoffice/program/soffice.bin
# disable mprotect:
paxctl -m /usr/lib/openoffice/program/soffice.bin
Now OOo should finally launch. This enables you to write a polite letter to the OOo team asking them to allow us to run OOo with mprotect.

