Mon, 29 Jan 2007 20:31:40 -0600
<devkit1 (AT) thevortex (DOT) ccwrote:

I have gotten the impression this may be a sore subject for the list
based on searching through the archives, but I do not intend to work
anyone up. I have been trying to find a solution to the problem of
shared hosting with a dynamic language such as PHP. I found the old
perchild MPM and it appears it is not being maintained or there was
possibly a design problem. I would like to know two things.

1. Is there a mechanism (other than suexec) that allows functionality
similar to perchild, that will allow a uid to be assigned on a per
request basis?

There are several third-party solutions: google for metux, peruser,
mod_ruid, and fastcgi.

2. If there is, do the developers need help with it? I can write C
and I am willing to help out with this. If there is not, Would
anyone from the Apache team be interested in working with me so I may
write such functionality, maybe for a future version of Apache?

Patches welcome.

Bear in mind that perchild was threaded, and therefore never
likely to be suitable for php.

Hi Arnold,

You have obviously spent a great deal of time implementing your
solution. Personally I have always felt complete separation (e.g. what
is done with FastCGI) is a more robust approach. But I don't think the
issues surrounding the choices have been discussed enough in the
public. If you don't mind I would appreciate if you could share your
opinions and experiences.

In particular, I am wondering if and how are you handling the issue of
"leaked" Apache file descriptors (under quotes because they are not
really leaked - it's the same process)? These file descriptors can be
abused to, for example, log to the Apache log files and take over the
network sockets.

The other issue is the users loading custom shared libraries to gain
direct access to the process memory and then extract sensitive
information from it (e.g. SSL keys).

questions that come to mind:

1) Have you ever evaluated a FastCGI-based approach?

2) Have you ever measured the performance increase you gain with your
solution (as opposed to having a "pure" suExec-based approach)?

3) Do you require a secret of some kind to change users? For example,
can I change the user from custom CGI script or a binary executed from
PHP?

Thanks,
Ivan

1/31/07, Arnold Daniels <info (AT) adaniels (DOT) nlwrote:

Hi,

We run a shared hosting company as well and taken upon the route to patch
the linux kernel to allow switching of user of the current process. An
apache module allows you to switch the process based on the virtual host.
current module also implements mass virtual hosting, but any open source
module should also work with normal vhost files.

We've been discussing this in the PHP internals mailing list and are
preparing an open source solution, which can be tested by interested
parties. Please read the message below, where security concerns of the PHP
are addressed. I'll be sure to post a message on the apache list when the
patch is made ready for public viewing.

Best regards,

Arnold Daniels
Javeline (www.javeline.net)

Rik Arends schreef:
Hi Andi,

I'm Rik Arends, i co-wrote the kernel patch + apache module for in-process
user switching.
After reading your concerns i might shed some light on these issues.

First of all, i know that there are some possible security holes with this
system.
of the biggest problems i could see is triggering a bufferoverflow in
mod_php, so the user can get its own assembler code to run.
Then by knowing how to do the kernel calls, he could, theoretically switch
the user of the process back to www-data, after which he could switch to
any 'shared hosting user' (not just any user) in the system to access their
shared hosting files.
The complexity of this hack, plus that your apaches will be segfaulting
continously while a person is trying this might not make it too plausible to
happen.
Second, the main system would not be at risk, just some of the shared
hosting users-files that might be accessed. In 99.99% of the cases, there
really is not all that much to steal and the amount of effort to actually
hack this is pretty huge and requires exploitation of a hardcore hole in an
in-process scripting engine (mod_php for instance), plus the knowledge on
how to trigger the right kernel calls, and the userID's to switch to (which,
unless the user somehow gained shell access to his targets directory he has
no clue about)

The same way i think we can approach the, the 'resources that are still
open from other users' hole, if it might be there. I expect mod_php or other
modules to do proper cleanup of their handles or else they would be leaking
a lot in an apache process thats being reused. This is not a new problem,
and also a very very difficult one to succesfully exploit.
Say we have 256 apache processes with user switching. You are targetting
site X running on the 'same machine'. Then you'd have to poll continously
and hope you are served by an apache process that also served the other site
AND know which resources to access, and how to do that. Please note that you
can only use the resource leak bug when you are using an in-process
scripting engine so you cant just go poke around your memory. This same
'bug' if you will is also there in shared hosting structures with reused
apache processes that don't do user switching at all. Again the risk of this
exploit actually being used seems well, remote. Add to this the fact that in
shared hosting environments, nobody runs any security critical applications
such as full creditcard payment systems. For that people employ their own
server with SSL and certificates. That is just beyond the scale of shared
hosting.

I hope i might have lessened your concerns. I think the security risk our
patch poses is mostly theoretical, have a very difficult exploit route and
in almost all cases have a 0 to almost nil payoff. Its much much simpler to
try to hack the other persons site via bugs in forms or other installed
applications.

Regards,

Rik Arends
>
>
>
>
bericht
: RE: [PHP-DEV] Comments on PHP security
Datum: Thu, 18 Jan 2007 14:14:17 -0800
Van: Andi Gutmans <andi (AT) zend (DOT) com>
Aan: 'Arnold Daniels' <info (AT) adaniels (DOT) nl>, <internals (AT) lists (DOT) php.net>
>
>
>
I haven't seen the patch yet but my concern would be with resources which
have already been opened. Unless you guys clean that up in
between requests it can be very dangerous as I doubt Linux re-verify's
permissions when those are accessed. In any case, I'd be
happy to review and might be completely wrong
--

Nick Kew schreef:
Mon, 29 Jan 2007 20:31:40 -0600
<devkit1 (AT) thevortex (DOT) ccwrote:
>
>
>
I have gotten the impression this may be a sore subject for the list
based on searching through the archives, but I do not intend to work
anyone up. I have been trying to find a solution to the problem of
shared hosting with a dynamic language such as PHP. I found the old
perchild MPM and it appears it is not being maintained or there was
possibly a design problem. I would like to know two things.

1. Is there a mechanism (other than suexec) that allows functionality
similar to perchild, that will allow a uid to be assigned on a per
request basis?

There are several third-party solutions: google for metux, peruser,
mod_ruid, and fastcgi.
>
>
>
2. If there is, do the developers need help with it? I can write C
and I am willing to help out with this. If there is not, Would
anyone from the Apache team be interested in working with me so I may
write such functionality, maybe for a future version of Apache?

Patches welcome.

Bear in mind that perchild was threaded, and therefore never
likely to be suitable for php.
>
>
>

* Nick Kew <nick (AT) webthing (DOT) comwrote:
Mon, 29 Jan 2007 20:31:40 -0600
<devkit1 (AT) thevortex (DOT) ccwrote:

I have gotten the impression this may be a sore subject for the list
based on searching through the archives, but I do not intend to work
anyone up. I have been trying to find a solution to the problem of
shared hosting with a dynamic language such as PHP. I found the old
perchild MPM and it appears it is not being maintained or there was
possibly a design problem. I would like to know two things.

1. Is there a mechanism (other than suexec) that allows functionality
similar to perchild, that will allow a uid to be assigned on a per
request basis?

There are several third-party solutions: google for metux, peruser,
mod_ruid, and fastcgi.

Quite a bit outdated:

http://mpm.metux.de

cu

Comments below.

1/31/07, Arnold Daniels <info (AT) adaniels (DOT) nlwrote:
Hi,

I can't answer all your questions, since I'm not the developer of the
patch and module. I've forwarded this message to Rik Arends. But let me
answer the onces I can.

We've looked at running PHP as CGI, but we've noticed a performance drop
compared to running PHP as module, effectively being able to run 25% to
30% less accounts on a server, which is significant when you're running
discount hosting.

Besides this performance drop, running PHP as CGI doesn't really solve
the problem. We run about 300 customers per server. All of which have
their own system user, the 'shared hosting users'. We want all files,
PHP files as well as HTML files, images, etc, to be owned by the shared
hosting user without privileges for others (660). For Apache to handle a
request, it needs to run under that user instead of www-data.

That should be possible to do using suExec and some juggling of file
permissions. Although it does create a problem where the user scripts
run as also has write permissions over all files. I prefer to use two
accounts, one to manipulate the files with, the other to execute
scripts with. But this is difficult to pull off with mass hosting.

We tried
the perchild, but starting a new apache process for just about each
request, promoted serious performance issues (I don't have the figures
at hand, but it wasn't an option) as well as not being stable.

It's not common for a setup like this to offer SSL, since it requires an
unique IP address to be save. The module currently also implements mass
virtual hosting, making it even impossible to support SSL.

You do not need a secret, but the users who may changed are specified.
In our setup user 'www-data', which has no privileges on the server, may
change into any shared hosting user. Within the process where www-data
changed into the shared hosting user, a call can be made to change back
into 'www-data'. Since running a CGI script starts a new process, it is
not possible for a shared hosting user to change into 'www-data' using a
custom CGI script.

Good. Still, if you allow PHP users to load their own code from shared
libraries they'll be able to revert back to the www-data user.

We don't log to file, but use UDP, still the same goes. A hacker can't
really accomplish anything with writing data to the log, except maybe a
bit of vandalism.

I actually think that is a very serious problem. With the ability to
add to the access logs someone could frame a person for hacking.

Perhaps you give some examples on how the file
descriptors can be used to take over the network sockets.

I haven't tried to exploit the issue myself but, from memory, I
believe it is enough to duplicate the original descriptor, close the
original (thus preventing Apache from using it), and then serve your
own stuff using the duplicate.

The last time I looked PHP leaked file descriptors even on execution
of external binaries, making it trivial to exploit the issue provided
external command execution is allowed.

Disclaimer: I played with this two years ago. It might have been
silently fixed since then. But with a large number of different
versions of Apache and PHP in deployment the only way to be sure is to
test. I recommend env_audit ().

We are aware that the solution might not yet be solid, be we think the
approach is the right way to go. course, we're currently enjoying
security by obscurity, but with your help and other experts on Apache
internals, we should be able to make this into a good solution.

Best regards,
Arnold
--
Ivan Ristic schreef:
Hi Arnold,

You have obviously spent a great deal of time implementing your
solution. Personally I have always felt complete separation (e.g. what
is done with FastCGI) is a more robust approach. But I don't think the
issues surrounding the choices have been discussed enough in the
public. If you don't mind I would appreciate if you could share your
opinions and experiences.

In particular, I am wondering if and how are you handling the issue of
"leaked" Apache file descriptors (under quotes because they are not
really leaked - it's the same process)? These file descriptors can be
abused to, for example, log to the Apache log files and take over the
network sockets.

The other issue is the users loading custom shared libraries to gain
direct access to the process memory and then extract sensitive
information from it (e.g. SSL keys).

questions that come to mind:

1) Have you ever evaluated a FastCGI-based approach?

2) Have you ever measured the performance increase you gain with your
solution (as opposed to having a "pure" suExec-based approach)?

3) Do you require a secret of some kind to change users? For example,
can I change the user from custom CGI script or a binary executed from
PHP?

Thanks,
Ivan

1/31/07, Arnold Daniels <info (AT) adaniels (DOT) nlwrote:
>>
>Hi,
>>
>We run a shared hosting company as well and taken upon the route to
>patch
>the linux kernel to allow switching of user of the current process. An
>apache module allows you to switch the process based on the virtual
>host.
>current module also implements mass virtual hosting, but any open
>source
>module should also work with normal vhost files.
>>
>We've been discussing this in the PHP internals mailing list and are
>preparing an open source solution, which can be tested by interested
>parties. Please read the message below, where security concerns of
>the PHP
>are addressed. I'll be sure to post a message on the apache list when
>the
>patch is made ready for public viewing.
>>
>Best regards,
>>
>Arnold Daniels
>Javeline (www.javeline.net)
>>
>
>>
>Rik Arends schreef:
>Hi Andi,
>>
>I'm Rik Arends, i co-wrote the kernel patch + apache module for
>in-process
>user switching.
>After reading your concerns i might shed some light on these issues.
>>
>First of all, i know that there are some possible security holes
>with this
>system.
>of the biggest problems i could see is triggering a
>bufferoverflow in
>mod_php, so the user can get its own assembler code to run.
>Then by knowing how to do the kernel calls, he could, theoretically
>switch
>the user of the process back to www-data, after which he could switch to
>any 'shared hosting user' (not just any user) in the system to
>access their
>shared hosting files.
>The complexity of this hack, plus that your apaches will be segfaulting
>continously while a person is trying this might not make it too
>plausible to
>happen.
>Second, the main system would not be at risk, just some of the shared
>hosting users-files that might be accessed. In 99.99% of the cases,
>there
>really is not all that much to steal and the amount of effort to
>actually
>hack this is pretty huge and requires exploitation of a hardcore hole
>in an
>in-process scripting engine (mod_php for instance), plus the
>knowledge on
>how to trigger the right kernel calls, and the userID's to switch to
>(which,
>unless the user somehow gained shell access to his targets directory
>he has
>no clue about)
>>
>The same way i think we can approach the, the 'resources that are still
>open from other users' hole, if it might be there. I expect mod_php
>or other
>modules to do proper cleanup of their handles or else they would be
>leaking
>a lot in an apache process thats being reused. This is not a new
>problem,
>and also a very very difficult one to succesfully exploit.
>Say we have 256 apache processes with user switching. You are
>targetting
>site X running on the 'same machine'. Then you'd have to poll
>continously
>and hope you are served by an apache process that also served the
>other site
>AND know which resources to access, and how to do that. Please note
>that you
>can only use the resource leak bug when you are using an in-process
>scripting engine so you cant just go poke around your memory. This same
>'bug' if you will is also there in shared hosting structures with reused
>apache processes that don't do user switching at all. Again the risk
>of this
>exploit actually being used seems well, remote. Add to this the fact
>that in
>shared hosting environments, nobody runs any security critical
>applications
>such as full creditcard payment systems. For that people employ their
>own
>server with SSL and certificates. That is just beyond the scale of
>shared
>hosting.
>>
>I hope i might have lessened your concerns. I think the security
>risk our
>patch poses is mostly theoretical, have a very difficult exploit
>route and
>in almost all cases have a 0 to almost nil payoff. Its much much
>simpler to
>try to hack the other persons site via bugs in forms or other installed
>applications.
>>
>Regards,
>>
>Rik Arends
>>
>>
>>
>>
>bericht
>: RE: [PHP-DEV] Comments on PHP security
>Datum: Thu, 18 Jan 2007 14:14:17 -0800
>Van: Andi Gutmans <andi (AT) zend (DOT) com>
>Aan: 'Arnold Daniels' <info (AT) adaniels (DOT) nl>, <internals (AT) lists (DOT) php.net>
>>
>>
>>
>I haven't seen the patch yet but my concern would be with resources
>which
>have already been opened. Unless you guys clean that up in
>between requests it can be very dangerous as I doubt Linux re-verify's
>permissions when those are accessed. In any case, I'd be
>happy to review and might be completely wrong
>>
>>
>
>>
>Nick Kew schreef:
>Mon, 29 Jan 2007 20:31:40 -0600
><devkit1 (AT) thevortex (DOT) ccwrote:
>>
>>
>>
>I have gotten the impression this may be a sore subject for the list
>based on searching through the archives, but I do not intend to work
>anyone up. I have been trying to find a solution to the problem of
>shared hosting with a dynamic language such as PHP. I found the old
>perchild MPM and it appears it is not being maintained or there was
>possibly a design problem. I would like to know two things.
>>
>1. Is there a mechanism (other than suexec) that allows functionality
>similar to perchild, that will allow a uid to be assigned on a per
>request basis?
>>
>There are several third-party solutions: google for metux, peruser,
>mod_ruid, and fastcgi.
>>
>>
>>
>2. If there is, do the developers need help with it? I can write C
>and I am willing to help out with this. If there is not, Would
>anyone from the Apache team be interested in working with me so I may
>write such functionality, maybe for a future version of Apache?
>>
>Patches welcome.
>>
>Bear in mind that perchild was threaded, and therefore never
>likely to be suitable for php.
>>
>>
>>
>
>
>
>

[Let's continue the discussion privately from now on as it's becoming
less relevant for the httpd project]

2/2/07, Arnold Daniels <info (AT) adaniels (DOT) nlwrote:


>
>You do not need a secret, but the users who may changed are specified.
>In our setup user 'www-data', which has no privileges on the server, may
>change into any shared hosting user. Within the process where www-data
>changed into the shared hosting user, a call can be made to change back
>into 'www-data'. Since running a CGI script starts a new process, it is
>not possible for a shared hosting user to change into 'www-data' using a
>custom CGI script.
>
Good. Still, if you allow PHP users to load their own code from shared
libraries they'll be able to revert back to the www-data user.

Due to permissions it isn't possible for customers to place any files in
the extension directory and PHP won't load any binaries outside that
directory. Anyhow it is highly recommended that dl() is disabled. Which
is the case on our system.

Good. Still, you are relying on mod_php to enforce certain
restrictions. I prefer a mechanism where you can deploy securely with
any module and any module configuration.


>We don't log to file, but use UDP, still the same goes. A hacker can't
>really accomplish anything with writing data to the log, except maybe a
>bit of vandalism.
>
I actually think that is a very serious problem. With the ability to
add to the access logs someone could frame a person for hacking.

I'm not sure how the privileges of the access log are currently set on
our system. But the access log could of course be written by www-data,
so before the user is switched, which solves the problem.

If you are using syslog/UDP then, because syslog lacks authentication,
any user can fake an access log entry. As for the access log
permissions - root should be the only user allowed write access.

Either way, if a file descriptor is inherited then any user can write
to it (permissions of the user that opened the FD are used).