Hi,
There's a Linux hardening project called PaX. Few years ago it
introduced some new interesting approaches to security, that over time
proved themselves to be of help when facing mostly unknown threats. The
idea behind this approach is that you design the security of the system
to prevent the exploit, and not the vulnerability.
The PaX project can be found on the web at
http://pax.grsecurity.net
And detailed documentation about the variety of features it contains,
including some performance benchmarks, can be found at
I decided to download the paxtest tool, used to check if the various
features of PaX are working or not. What this tool does is run a few
tests to see if techniques that are commonly used in exploits are
working or not. More on this inline below, in square brackets.
The PaX test-suite can be downloaded from
Extract it and type "gmake netbsd" to build the paxtest tool, then use
it as "./paxtest blackhat".
The attack techniques this test suite checks for are
1. If anonymous mappings, or the bss/data/stack/heap sections of a
binary, are executable by default. If they are, it means the most
basic buffer overflow exploits will work.
2. If it's possible to make previously executable mappings writable,
or previously writable mappings executable. While this is standard
operation, it's a technique used to bypass W^X and the likes, and
can be seen in several modern exploits.
3. Amount of randomization in mappings each time a program starts.
The more predictable the mappings are, the easier it is to perform
attacks that require jumping to a known address - like ret-to-libc
for example, used also to bypass W^X.
Note: While SSP may also help in preventing these ret-to-libc
attacks, this is another layer that should be considered, where the
two can provide more coverage against this type of attack.
4. Return-to-lib attacks via string manipulation routines. If shared
libraries are mapped where the address of functions in them have
a NULL byte it means that the attacker can't (easily) use a
return-to-lib attack to get to them, because the input used to
overflow must be ASCII armored.
5. Same as above, only this time the overflow is made in a routine
that isn't affected by a NULL byte. SSP should be able to stop
these attacks.
6. Checks if code in the data/bss segments of a shared library can be
executed.
7. Checks if it's possible to map text segments as writable. Another
form of the attack in #2.
The goal, of course, is to pass all tests in this suite.
Here's output from a (slightly modified version of) paxtest run on my
NetBSD machine:
phyre:paxtest-0.9.7-pre4 {319} ./paxtest blackhat
PaXtest - Copyright(c) 2003,2004 by Peter Busser <peter (AT) adamantix (DOT) org>
Released under the GNU Public Licence version 2 or later
Writing output to paxtest.log
It may take a while for the tests to complete
Test results:
PaXtest - Copyright(c) 2003,2004 by Peter Busser <peter (AT) adamantix (DOT) org>
Released under the GNU Public Licence version 2 or later
Mode: blackhat
NetBSD phyre.bsd.org.il 3.99.14 NetBSD 3.99.14 (GENERIC) #14: Sun Dec 11
23:00:39 IST 2005
elad@
amd64
Executable anonymous mapping : Killed
Executable bss : Killed
Executable data : Killed
Executable heap : Killed
Executable stack : Killed
[ #1. That's good. architectures where there's no NX bit,
these would be vulnerable too. i386, for example. ]
Executable anonymous mapping (mprotect) : Vulnerable
Executable bss (mprotect) : Vulnerable
Executable data (mprotect) : Vulnerable
Executable heap (mprotect) : Vulnerable
Executable shared library bss (mprotect) : Vulnerable
Executable shared library data (mprotect): Vulnerable
Executable stack (mprotect) : Vulnerable
[ #2. While to not be vulnerable to this attack we need to
change semantics of mprotect(), doing this based on a sysctl
knob might be a good idea.
The performance cost of adding this protection is ~zero,
because it's a matter of testing for flags. ]
Anonymous mapping randomisation test : No randomisation
Heap randomisation test (ET_EXEC) : No randomisation
Main executable randomisation (ET_EXEC) : No randomisation
Shared library randomisation test : No randomisation
Stack randomisation test (SEGMEXEC) : No randomisation
Stack randomisation test (PAGEEXEC) : No randomisation
[ #3. Randomization hooks could fix these. ]
Return to function (strcpy) : paxtest: return address
contains a NULL byte.
Return to function (strcpy, RANDEXEC) : paxtest: return address
contains a NULL byte.
[ #4. That's good. Is this on purpose? :) (no) ]
Return to function (memcpy) : Vulnerable
Return to function (memcpy, RANDEXEC) : Vulnerable
[ #5. SSP fixes this one, as well as #4. ]
Executable shared library bss : Killed
Executable shared library data : Killed
[ #6. That's good, but again, not on all architectures. ]
Writable text segments : Vulnerable
[ #7. Same note as #2. ]
phyre:paxtest-0.9.7-pre4 {320}
As you can see, NetBSD is not doing too well in these tests. While there
could certainly be a performance/functionality penalty in fixing some of
these -- namely, the randomization and mprotect() ones -- I feel we
should leave the choice to the users to decide (whether to enable or
not).
What I would like to discuss is the possibility of importing two
important features from PaX -- ASLR and MPRTECT.
I've CC'd this mail to the PaX author, who knows about this stuff a lot
more than I do (and is in general a very cool dude :) so he could give
his comments during the discussion; please CC him on replies as well.
Thanks,
-e.

Elad Efrat wrote:
The attack techniques this test suite checks for are

1. If anonymous mappings, or the bss/data/stack/heap sections of a
binary, are executable by default. If they are, it means the most
basic buffer overflow exploits will work.

Some architectures (powerpc) have ABIs that mandate executable bss/data.
There is no way to turn that off.

2. If it's possible to make previously executable mappings writable,
or previously writable mappings executable. While this is standard
operation, it's a technique used to bypass W^X and the likes, and
can be seen in several modern exploits.

If you have textrel relocations, you have to allow this. I think the
right way to have add a PRT_LCKED that mprotect can understand. If
this is supplied, the protection on such pages can't be downgraded.
munmap would still be allowed (it might be advisable to make munmap
change the region to PRT_NNE but leave it in place so it can't be
remapped.)

3. Amount of randomization in mappings each time a program starts.
The more predictable the mappings are, the easier it is to perform
attacks that require jumping to a known address - like ret-to-libc
for example, used also to bypass W^X.

The program itself is linked at a base address. the libraries and stacks
could be vary but on machines with virtual caches, that might be very painful.

Note: While SSP may also help in preventing these ret-to-libc
attacks, this is another layer that should be considered, where the
two can provide more coverage against this type of attack.

4. Return-to-lib attacks via string manipulation routines. If shared
libraries are mapped where the address of functions in them have
a NULL byte it means that the attacker can't (easily) use a
return-to-lib attack to get to them, because the input used to
overflow must be ASCII armored.

So you are saying that most libraries should be mapped at < 16MB? That's
just not practical for most architectures. I suppose that libraries with
<= 64KB of text can just be mapped on a 16MB boundary. LP64 you can map
libraries into the first 16MB of each 4GB region. course, you'll fragment
your memory space that way.

5. Same as above, only this time the overflow is made in a routine
that isn't affected by a NULL byte. SSP should be able to stop
these attacks.

6. Checks if code in the data/bss segments of a shared library can be
executed.

See above for ABI requirements.

7. Checks if it's possible to map text segments as writable. Another
form of the attack in #2.

The goal, of course, is to pass all tests in this suite.

I don't think that really practical for most platforms.

Matt Thomas wrote:

Some architectures (powerpc) have ABIs that mandate executable bss/data.
There is no way to turn that off.

True, but two things:

1. How many of our users run i386, amd64, sparc, sparc64? how many of
them run powerpc?
2. We are already providing this feature where we can, so it's not
this that is up for discussion. :)

If you have textrel relocations, you have to allow this.

True. However, PaX is aware of text relocations, and addresses them.

I think the
right way to have add a PRT_LCKED that mprotect can understand. If
this is supplied, the protection on such pages can't be downgraded.
munmap would still be allowed (it might be advisable to make munmap
change the region to PRT_NNE but leave it in place so it can't be
remapped.)

We should look at the two solutions and decide which fits best; or
maybe combine them.

The program itself is linked at a base address. the libraries and stacks
could be vary but on machines with virtual caches, that might be very painful.

Like I said, let's leave the choice to the user. I'm sure there are
plenty of people with 3ghz machines that will be happy to have this
feature at the given cost

So you are saying that most libraries should be mapped at < 16MB?

No, I'm saying that SSP (in gcc 4.1) should address both problems in
a more elegant manner than a NULL byte in the return address. This is
also why I didn't ask this feature to be discussed in the mail.

That's
just not practical for most architectures. I suppose that libraries with
<= 64KB of text can just be mapped on a 16MB boundary. LP64 you can map
libraries into the first 16MB of each 4GB region. course, you'll fragment
your memory space that way.

It's also a very ugly solution. It's not up for discussion. :)

I don't think that really practical for most platforms.

This mail has two intentions:

1. Present the paxtest tool and provide some explanantion about its
tests.
2. Ask to discuss the importing of two features from PaX: ASLR
(Address Space Layout Randomization) and MPRTECT (the mprotect()
restrictions).

ASLR is practical. It may add a performance hit, but like I already
said, let's leave that choice to the user. MPRTECT is also practical
but needs special care.
-e.

Hubert Feyrer wrote:

* under what license is the code to-be-imported (seeing GPL in the ouput
of that test), and

There is no licensing problem.

* what is the performance impact when this is enabled?
I guess some more benchmarks would be useful to establish that.

I haven't done any native benchmarks, but I can point you to (some)
benchmarks already done, available at
, although I'm not
sure these are fair wrt/to the features in question.

Some information that might help figuring what would be the costs of
ASLR and MPRTECT:

ASLR calculates 3 random values on execution and saves these as offsets
to be used when a random value is needed. How expensive are 3
arc4random() calls in the context of an entire sys_execve()?

MPRTECT, when not talking about text relocations, is very simple and
a matter of playing with flags. There's some more work involved when
handling textrels, but we have both the PaX way and Matt's way to look
into and decide what would be best for us.

For these two features, the last case of handling textrels *may* present
the only visible performance hit.

However, like I already mentioned these are security features, and we
should let the user do the trade-off if desired. We can always have
these turned off by default.
-e.

Hi,

Even though you ignored #2 that says we already done this

James Chacon wrote:

In general we don't count the number of users of a given arch before
looking at a feature and it's costs. Things need to fit into a cross
architecture framework that don't negatively impact other archs.

Some things, like the NX bit for example, are MD. Do you think we should
not provide a security feature because we can't provide it for all
architectures?

If you want that, there are other *BSD distributions aimed at "best
performance for the x86 arch" or other such goals

Again: it should be the end-user's choice what security features to
use, and it should be NetBSD's goal to take full advantage of the
features offered to us by the hardware.
-e.

Jason Thorpe wrote:

Dec 18, 2005, at 4:03 AM, Martin Husemann wrote:

>The mprotect changes are arch dependend, and as gimpy pointed out would
>violate ABI on some archs. I don't see how implementing them (which
>would
>be MD anyway) would cause any negative impact on other archs.

Yah, these would have to be implemented using new pmap hooks.

Why wouuld these require pmap changes? they (should) be pure
mmap/mprotect changes. Do you want to implement them in pmap because
they are MD, or is there a technical need?
-e.

Elad Efrat wrote:
Jason Thorpe wrote:


>Dec 18, 2005, at 4:03 AM, Martin Husemann wrote:

The mprotect changes are arch dependend, and as gimpy pointed out would
violate ABI on some archs. I don't see how implementing them (which
would
be MD anyway) would cause any negative impact on other archs.
>>
>>
>>Yah, these would have to be implemented using new pmap hooks.

Why wouuld these require pmap changes? they (should) be pure
mmap/mprotect changes. Do you want to implement them in pmap because
they are MD, or is there a technical need?

My proposal doesn't need pmap hooks. However, VM placement does (see
PMAP_PREFER() for instance) for virtual caches.

pageexec (AT) freemail (DOT) hu wrote:
18 Dec 2005 at 12:04, Elad Efrat wrote:

>>Matt Thomas wrote:
>>
>>
Some architectures (powerpc) have ABIs that mandate executable bss/data.
There is no way to turn that off.
>>
>>True, but two things:

actually, half-true. the ABI in question i guess is that of SysV, and
it has some sillyness in it (blrl in GT[-1] that's not even marked as
executable). the bigger problem is that it's runtime generated
(like many other archs), and that's the security problem. recognizing
this, some folks at Red Hat earlier this year had developed a better
mechanism they call secureplt, it's available for ppc and alpha already,
and similar can (and should) be implemented on other archs as well
(an item that's been also on my todo list for too long). see [1]-[4]
for more info.

I know about secureplt. But that doesn't help for backwards compatibility.

If you have textrel relocations, you have to allow this.
>>
>>True. However, PaX is aware of text relocations, and addresses them.

if you have textrels, it's best to fix them. see [5] why they're bad,
especially for security (basically, they need the privilege for runtime
code generation).

I know why they are bad. But they

I think the
right way to have add a PRT_LCKED that mprotect can understand. If
this is supplied, the protection on such pages can't be downgraded.
munmap would still be allowed (it might be advisable to make munmap
change the region to PRT_NNE but leave it in place so it can't be
remapped.)

a better (more generic) way would be to extend mmap/mprotect to be
able to specify what becomes maxprot, in addition to prot, so that
after textrels (or any kind of runtime code generation) are done,
userland can revoke maxprot bits, effectively sealing the protection
on those pages. this requires userland changes (in ld.so at least),
so i didn't implement it in linux this way, instead some heuristics
is used in the kernel to do the same (this is just FYI, you can do
it the proper way in NetBSD as you control userland as well).

That's an ABI change (if you are talking extra arguments).
it reduces down to the same thing as my proposal.

The program itself is linked at a base address.

FYI, it has always been possible to create executables in the ET_DYN
ELF format and have them load at random addresses, i've done this more
than 4 years ago under linux, and for something like 2 years now Red
Hat added official toolchain support for it. they call it PIE (as in,
Position Independent Executable), and it's available in gcc 3.3+ and
binutils 2.15+. of course your kernel's ELF loader has to support
ET_DYN executables as well, which afaik, it doesn't at the moment.

PIE? Ewww. :) PIE was primarily intended for small embedded systems.

the libraries and stacks
could be vary but on machines with virtual caches, that might be very painful.

since this virtually indexed cache problem came up more than once, i'd
like to ask what exactly you're referring to. i assume that it's something
to do with address space switches and avoiding cache flushes (someone also
mentioned TLB, but i don't see how that figures in here), but i need some
insights as to what you see as a problem. be as technical as you can be
(pointers to code are appreciated, i'm not too familiar with UVM and pmaps).

Some machines require all mappings of a physical page be to compatible virtual
addresses where the low N (16-24) bits have same value. That retricts where
you can map a page in the virtual address space.

Pavel Cahyna wrote:

Wouldn't it prevent future optimizations of the dynamic linker, which
might require constant and known addresses of dynamic libraries? I think
IRIX does that (don't know how RelCache was designed, maybe it applies
there too).

Let's leave this decision for the end-user to make.
-e.

Pavel Cahyna wrote:

Fine. If you implement this, can you please make the decision controllable
per-process, rather than per-system? E. g. with some proc.<pid>.xxx
sysctl. Because if any such optimizatoon appears, it will make sense to
enable randomization for processes where exec time is not a bottleneck and
are exposed to attacks (like sshd, bind, or setuid executables) but disable
it for other processes.

Sure. PaX already does something similar using its own ELF program
header to store related flags; I'll look into doing the same for
NetBSD.
-e.

pageexec (AT) freemail (DOT) hu wrote:
18 Dec 2005 at 13:49, Matt Thomas wrote:
>>PIE? Ewww. :) PIE was primarily intended for small embedded systems.

i think you're mixing it up with something else, PIE was explicitly
created to address the main executable randomization problem [1]:

"This option creates something between a shared library and normal
executable, which can be used for security exposed binaries so that their
base address can be randomized (either a constant address different on
each box through prelink -R (support for PIEs in prelink will be comming),
or totally random address)."

PIE also forces a portion of .text to be nonshared (any relative relocations
that could be fixed in a based image will no longer be shared among multiple
processes). It will increase the complexity of program loading which is
already very complex.

Are all programs built/linked at PIE, or just a subset?