cap-talk,
I realize this topic is a bit off our capability focus. I hope
somebody will point me
to a more focused list if they know of one. Given the sorts of
topics we discuss on
this list, this one seems generally relevant to me.
I found this paper:
http://www.eecs.umich.edu/~pmchen/papers/king06.pdf
quite interesting reading. It contains a description of some work done
on Virtual Machine Based Rootkits (VMBRs).
Some have taken this work to suggest that the work that has recently
gone into processor design to make virtual machine support transparent
may backfire in some way to make rootkits like the above more difficult
to detect and therefore more prevalent.
I don't agree with this position. I would like to get a full discussion
happening lest such an understand (if indeed it is incorrect) gain
wide acceptance and another generation of virtual machine support
goes down the tubes.
Here are some comments I had on the paper:
It appears that all the work (the test
systems with VMWare and VirtualPC) were done on machines without the new
generation of virtual machine support processors:
"We expect future enhancements to the x86 platform
to reduce these perturbations. Upcoming virtualization
support from Intel [45] and AMD [7] will enable
more efficient virtualization."
so it's clear that one doesn't need such virtualization features to implement
such Virtual Machine Based Rootkits. Still, one could argue that such features
make their life easier. I don't believe the authors were suggesting
that, though
some have suggested that in email. I believe the authors appropriately placed
the issue in the context of an escalating battle. They pointed out some ways
in which virtualizability can enhance protection (e.g. run a secure VMM on the
base system and detect rootkits in VMs).
I believe such virtualization support makes it considerably easier to protect
against rootkits. I do think running relatively vulnerable systems (e.g.
systems where you run commercial or downloaded software, systems
that are network facing, etc.) under a VM makes protection easier. I'd
very much like to get a network monitor for my VMM. I know that in some
sense it's safer to monitor the network externally. Still doing so is less
convenient. I thing such monitoring from a base VMM could be quite
helpful.
I think there are a couple of points in the paper where the authors
got a bit enamored with their technology and seemed to suggest that it
could do things that I believe are difficult to impossible. For example,
when they were referring to efforts to detect a VMBR they said:
"even if the target system did see something amiss, the VMBR could
tamper with the execution of the detector and force it to report incorrect
results."
While I agree this is theoretically possible, I think in practice
it's very unlikely to ever be achieved.
From what I've read so far it seems that having better support for
virtual machine monitors supports efforts at protection and
detection more than it supports attacks. I will personally feel
more comfortable if I'm able to run my network facing Windows
system under a virtual machine monitor. E.g. one that can look
for boot record changes, can monitor network traffic, etc.
where I can take the system easily back to an initial state
(e.g. base build, base software installed) and easily add my
relevant files and some few new applications.
http://www.webstart.com/jed/
cap-talk mailing list
cap-talk (AT) mail (DOT) eros-os.org

At 09:31 AM 8/3/2006, Karp, Alan H wrote:
>Joanna Rutkowska is the name I've seen associated with this attack,
>which is frequently called Blue Pill, from the Matrix. She has
>demonstrated a running version on Vista x64 and is presenting at Black
>Hat today. According to reports, she was able to install the rootkit on
>a running system, no reboot required.
>,1895,1983037,00.asp is a news article on
>the subject.

<still going on with the VMBR discussion not focused on capabilities.
lists or suggestions to move or stay gratefully accepted>

I should have mentioned in my initial message that I've seen some of
the Blue Pill discussion. I'm a bit uncomfortable with what I've seen about
it so far, e.g. (from the article you mentioned Alan):

"Blue Pill is being developed exclusively for CSEINC Research and will
not be available for download. However, Rutkowska said the company is
planning to organize trainings about Blue Pill and other technologies
where the source code would be made available."

There seems to me to be a bit too much notoriety and profit motive in
what I've seen about this work so far. I'll take it more seriously when it
gets "out there" and generally worked over in the academic and
general open source software communities.

>The key point is that you're both right. You are safer if you use a
>virtual machine to run Windows.

any other guest S. Certainly true.

>However, if your base system gets
>infected, virtualizability assures that there is no mechanism by which
>the S can detect the attack.

I think the above is an overstatement. Even if there was no way for an S to
detect that it's running under a VMM (which there is - no VMM is completely
undetectable), there are other detection mechanisms. For example one can
power cycle the system, come up on clean media (e.g. a CD) and run an
analysis on the hard drive(s) - e.g. as discussed in the SubVirt paper. This
assumes that the BIS hasn't been corrupted - which gets into another set
of issues independent of VMMs. You seem to be referring to detection
mechanisms in the guest S. Even in the guest S there are possibilities
(e.g. see the King paper), but that is where the purity of the VM comes
into play.

Regarding:

At 10:11 AM 8/3/2006, David Hopwood wrote:
>It is true to say that a guest S cannot reliably detect a VMM in a
>way that is useful to prevent this kind of rootkit attack, in general.
>After all, we don't want guest Ses to refuse to run under any VMM;
>that would be more counterproductive than helpful.

I think this is what I was agreeing to above. However, regarding:

>Also, such a
>detection mechanism could be circumvented, if the attacker writes
>his/her code after the defender (and I believe this to be more practical
>than Jed does).

I don't recall saying anything about the above (the relative timing of
the attacker and defender writing code) and thinking about it just now
I don't have an opinion on it. Is there something I wrote that leads you
to think my view on this differs from yours? Are you referring to this:

Jed:

I think there are a couple of points in the paper where the authors
got a bit enamored with their technology and seemed to suggest that it
could do things that I believe are difficult to impossible. For example,
when they were referring to efforts to detect a VMBR they said:

"even if the target system did see something amiss, the VMBR could
tamper with the execution of the detector and force it to report incorrect
results."

While I agree this is theoretically possible, I think in practice
it's very unlikely to ever be achieved.

? That is, a situation where the VMBR detects the execution of
rootkit detection software running in the guest S and modifies
it's behavior in such a was as to render it ineffective in detecting
the VMBR or in reporting it's detection? If so then when you
consider the attacker writing their code after the defender you
must certainly consider the usual case that the "defender" code
will be regularly updated to detect new attackers. You might then
believe that this would be some sort of horse race, with the attacker
trying to keep updating the VMBR (presumably over the network -
which can of course be detected off the box, but never mind that) in
order to keep falsifying results for the defender. I believe even this
underestimates the difficulty for the attacker. It doesn't suffice for
the attacker to simply maintain the functioning of the virtual machine
and to reach into the guest S's VM to modify the behavior of the
defender so as to appear invisible, it must also either:

1. not be detectable to the defender. This is a little like
stealth technology - where shared resources must be
used invisibly. I don't believe it's possible. Just for example,
the guest S can assume it's running on the bare metal.
It can run randomized loops (e.g. with interrupts disabled)
where it should know exactly the number of cycles used and
be able to make counts of repeatable behavior. If the
behavior changes, it detects the VMBR.

or

2. even if the defender detects the VMBR, the VMBR will
reach into the guest S and modify the detectors behavior
in such a way as to make it appear that the VMBR wasn't
detected and more generally no problem was detected. I
believe this would be quite difficult to accomplish. thing
to keep in mind is that the detection software has the user
on it's side. It can set up some simple user interaction
that will help assure that it is operating unhindered.
The VMBR can't, for example, simply stop the detection
software from running and output what seems to it to be
nominally reassuring output.

I generally just feel that this is a situation that favors
the defense. I also don't think it's much impacted by
the effectiveness of the VM assist features in the hardware.
I know it's fashionable to be macho on the side of the
attacker in situations like this. I think there is ample
opportunity for a bit of macho on the side of the defender
in this situation.

Hey, I wonder if this is one of those sorts of situations where
a concrete enough bet could be set up to be meaningful?
For example, I could bet that the majority of commodity
processors sold in the year, say, 2011, will be virtualizable.
That is, all privileged operations will trap in user mode,
as with the current Intel VT and AMD Pacifica, so VMMs
can be run for unmodified Ss. Given the current state of
the art that seems a rather brash and, from my perspective,
hopeful position to take, but it might be fun to have a bet out
there to keep an eye on. Anybody have any thoughts on
such a bet, e.g. how to tighten it up or whether it seems a
reasonable bet? Certainly if virtualizability comes to be seen
as a negative feature because it makes systems more subject
to being invisibly rooted, then one would expect virtualizability
to die out or at least be compromised in some way.

Regarding:

>At 03:13 PM 8/3/2006, Karp, Alan H wrote:
>David Hopwood wrote:
>
That's not quite accurate; most VMMs are detectable, and
AFAIK all VMMs
that run on x86 hardware (VT, Pacifica or otherwise) are detectable.
>
>That's because x86 is not fully virtualizable. Rutkowska is working on
>architectures that are.

Huh? I thought the VT and Pacifica versions of "x86" are fully
virtualizable. Is that not true? My understanding is that all it
takes to be "fully virtualizable" is to have all privileged operations
trap in "user" mode. Perhaps I misread this, but I thought
Rutkowska was working with Pacifica. Not?

http://www.webstart.com/jed/

cap-talk mailing list
cap-talk (AT) mail (DOT) eros-os.org

At 12:58 PM 8/4/2006, David Hopwood wrote:
>Another is the VT/Pacifica-specific features themselves -- there was no
>attempt (and it would have been much more complex) to make these
>architectures *recursively* virtualizable.

That's interesting. That would seem to suggest that if you're running
a VMM and some cracker tried to install a VNBR by coming in through
a guest S, then they wouldn't be able to make use of the virtualizability
features of VT or Pacifica in any case. Then even if they were somehow
able to break through to the hardware level and install their VMBR, it
would seem that their doing so would mess up you're running your
own VMM - making their VMBR quite visible indeed.

I'm not sure where the truth lies here, but this thought that building
virtualizable processors will somehow make them more vulnerable
to rootkits seems vastly oversold to me at this point.

Perhaps Norm remembers this technical point. I seem to recall that
some of the IBM 370 computers came with virtual machine assist that
deliberately did provide for recursive virtualizability. Do you recall
that Norm? Does anyone know if there are still VM370 systems
running VMMs?

http://www.webstart.com/jed/

cap-talk mailing list
cap-talk (AT) mail (DOT) eros-os.org

8/4/06, Jed at Webstart <donnelley1 (AT) webstart (DOT) comwrote:
Perhaps Norm remembers this technical point. I seem to recall that
some of the IBM 370 computers came with virtual machine assist that
deliberately did provide for recursive virtualizability. Do you recall
that Norm? Does anyone know if there are still VM370 systems
running VMMs?

http://www.webstart.com/jed/

See page 14, upper right column of:
VM is still recursively hostable!
-David Mercer

cap-talk mailing list
cap-talk (AT) mail (DOT) eros-os.org

At 03:15 PM 8/4/2006, David Mercer wrote:
8/4/06, Jed at Webstart <donnelley1 (AT) webstart (DOT) comwrote:
Perhaps Norm remembers this technical point. I seem to recall that
some of the IBM 370 computers came with virtual machine assist that
deliberately did provide for recursive virtualizability. Do you recall
that Norm? Does anyone know if there are still VM370 systems
running VMMs?

http://www.webstart.com/jed/
>
>See page 14, upper right column of:
>VM is still recursively hostable!
>
>-David Mercer

Amazing! Thanks for sharing that David! I guess I should connect to my
buds working on IBM systems if I really want to do some work with VMs.
It seems I'm not likely to be able to afford one of those System z servers:

for my home though, even if it does run Linux ;-) Here's a note on Linux/390:

that points to this VM (like Klenex, no need to say "IBM") paper:

http://vm.marist.edu/~mvmuajs/vmoutlook/
(last updated 11/99)

This quote from the paper is somewhat telling:

IBM has always had an ambivalent attitude towards VM. of VM's
problems for IBM has been that it's just too efficient; IBM would
much rather that customers use a "strategic" platform that just
happens to require much more expensive iron to run. In later sections
of this paper I'll discuss IBM's attempts to move customers to
inappropriate, and often inferior technology.

As a result, IBM has always had an element that wished VM would
disappear. and predicted VM's imminent demise many times in VM's
25-year history. These predictions turned out to be foolish, and VM
has outlived many of its detractors within IBM. This has also had the
effect of alienating many IBM customers, and reducing IBM's credibility.

Incidents of IBM staff telling customers that "VM is dead" have been
reported since VM's early days. I don't think IBM realizes that
insulting parts of its product line damages its entire brand
reputation. This type of unprofessional behavior does not help
convince me to buy anything from IBM. Fortunately, this type of
self-inflicted wound seems to happen less frequently now, though a
lot of damage was done. If you see an IBM person badmouthing VM,
report it to the VM management team, and they'll see that it is dealt
with appropriately.

I guess I was one of those fooled into thinking VM was dead. This is
a good one, "new use of VM, most notably VM web serving". I
can see how it would make some sense.

I love this (from the above), "Some people say VM is dead.
people say MVS is dead, S/2 is dead, DS (the mainframe variety) is
dead, DS (the PC variety) is dead, Apple is dead, and even Unix is
dead. Some of these statements may be true. Well, S/2 is definitely dead."

Heh ;-) Also, "IBM's lack of marketing support for VM ensured
that only people who think for themselves would run VM. The result
was to 'select' VM customers for lack of docility."

Isn't amazing some of the techno/cultural niches we can get into as
computer people!?!

http://www.webstart.com/jed/

cap-talk mailing list
cap-talk (AT) mail (DOT) eros-os.org

8/4/06, Jed at Webstart <donnelley1 (AT) webstart (DOT) comwrote:
At 03:15 PM 8/4/2006, David Mercer wrote:
8/4/06, Jed at Webstart <donnelley1 (AT) webstart (DOT) comwrote:
Perhaps Norm remembers this technical point. I seem to recall that
some of the IBM 370 computers came with virtual machine assist that
deliberately did provide for recursive virtualizability. Do you recall
that Norm? Does anyone know if there are still VM370 systems
running VMMs?

http://www.webstart.com/jed/
>
>See page 14, upper right column of:
>VM is still recursively hostable!
>
>-David Mercer
>
Amazing! Thanks for sharing that David! I guess I should connect to my
buds working on IBM systems if I really want to do some work with VMs.
It seems I'm not likely to be able to afford one of those System z servers:

for my home though, even if it does run Linux ;-)

If you wanna play with LInux/s390, you can get a login to your very
own linux VM for free over at:

I've ported/validated some different open source packages on
Linux/s390 for the sheer hell of it before, it's pretty much like any
other linux. I have no idea if they have any VM/CMS sessions
available for use by developers (for love or money). They charge
(gasp!) $2000+/MNTH for use of a z/S virtual system (MVS by a new
name).
-David Mercer

cap-talk mailing list
cap-talk (AT) mail (DOT) eros-os.org

Aug 4, 2006, at 2:32 PM, Jed at Webstart wrote:

At 12:58 PM 8/4/2006, David Hopwood wrote:
>Another is the VT/Pacifica-specific features themselves --
>there was no
>attempt (and it would have been much more complex) to make these
>architectures *recursively* virtualizable.
>
That's interesting. That would seem to suggest that if you're running
a VMM and some cracker tried to install a VNBR by coming in through
a guest S, then they wouldn't be able to make use of the
virtualizability
features of VT or Pacifica in any case. Then even if they were
somehow
able to break through to the hardware level and install their VMBR, it
would seem that their doing so would mess up you're running your
own VMM - making their VMBR quite visible indeed.

I'm not sure where the truth lies here, but this thought that building
virtualizable processors will somehow make them more vulnerable
to rootkits seems vastly oversold to me at this point.

Seems right

Perhaps Norm remembers this technical point. I seem to recall that
some of the IBM 370 computers came with virtual machine assist that
deliberately did provide for recursive virtualizability. Do you
recall
that Norm? Does anyone know if there are still VM370 systems
running VMMs?

Long story. There were versions of VM/370 that could run several deep.
IBM first added VM-assist which was an option the CP could select that
caused the hardware to 'do the right thing' with certain simple frequent
privileged instructions. CP would select this option when running
virtually privileged code.
Different models of 370 could virtualize different subsets of the
privileged
functions and CP retained the ability to interpret all priv-ops.
I think that the virtual machines lacked VM-assist.
Later IBM introduced SIE (Start Interpretive Execution).
They released an incomplete specification of this instruction, but
changed
their mind and never released the complete specs.
Roughly SIE virtualized the entire privileged mode except for I/
The memory operand of the SIE pointed to a memory area where
images of virtual special registers were kept.
I think it was never documented completely for public consumption.
SIE was a privileged instruction for no reason that IBM would reveal.
There was no reason that CP could not have emulated SIE as it emulated
other priv-ops but I think that it did not.
We (Tymshare) stopped using VM/370 when IBM ceased delivering
the source code.
SIE was subsequent to that.

http://www.webstart.com/jed/

cap-talk mailing list
cap-talk (AT) mail (DOT) eros-os.org

cap-talk mailing list
cap-talk (AT) mail (DOT) eros-os.org

Aug 4, 2006, at 3:15 PM, David Mercer wrote:

8/4/06, Jed at Webstart <donnelley1 (AT) webstart (DOT) comwrote:
>Perhaps Norm remembers this technical point. I seem to recall that
>some of the IBM 370 computers came with virtual machine assist that
>deliberately did provide for recursive virtualizability. Do you
>recall
>that Norm? Does anyone know if there are still VM370 systems
>running VMMs?
>>
>http://www.webstart.com/jed/
>
See page 14, upper right column of: http://www.vm.ibm.com/library/
zvmref08.pdf
VM is still recursively hostable!

Thanks for finding that link.
As David Hopwood says, there is much variability in what features a
real system has and I suspect that higher level virtual zSeries
machines become simpler on this axis.
There is amazing smoke and mirrors under the cover in the zSeries.
Depending on the model it bottoms out in an x86 running code from
Fundamental software <http://www.funsoft.com/>,
A power PC with special features, or an honest to goodness 64 bit
implementation of the greatly enhanced 370 instruction set.

-David Mercer

cap-talk mailing list
cap-talk (AT) mail (DOT) eros-os.org

cap-talk mailing list
cap-talk (AT) mail (DOT) eros-os.org

At 10:48 PM 9/13/2006, Bill Frantz wrote:
>donnelley1 (AT) webstart (DOT) com (Jed at Webstart) on Thursday, August 3, 2006 wrote:
>
>My understanding is that all it
>takes to be "fully virtualizable" is to have all privileged operations
>trap in "user" mode.
>
>[Sorry to be so late replying. I've been traveling.]
>
>Having all privileged operations trap in "user" mode is necessary but
>not sufficient. some Intel architectures, there were instructions
>that executed differently in privileged mode and in user mode. If I
>remember correctly, some extra information was returned in privileged
>mode.

I was considering such an instruction "privileged" in the above sentence,
though I was of course being brief and informal.

>To be fully virtualizable, these instructions would also have to
>trap. I would say an additional criteria is, "All user mode
>instructions must have the same specification in both privileged and
>user mode."

Yep.

Regarding:

At 11:36 PM 9/13/2006, Mark S. Miller wrote:
>Section 10.4 of my thesis summarizes the classic paper on this topic:
>
>Popek and Goldberg's "Formal Requirements for Virtualizable
>Third Generation Architectures" [PG74] explains the
>conditions needed for a hardware architecture to be cleanly
>virtualizable.

I well remember that work by Popek. He was working with DEC
under an ARPA contract to use a virtual machine architecture to
develop a secure operating system. I remember it well because I
was working in that area on the RISS (Research Into the Security
of Systems) project and considered the development of
such a secure S difficult - VMM or no VMM. of the things I
remember is that every year for at least two and perhaps three years
Dr. Popek spoke at the Fall Joint Computer Conference
suggesting that by the next year he would have an operating
system proven secure.

As far as I know it never happened. I don't remember if the VMM for
the PDP-11/45 ever happened, though I assume something was
developed:

The PDP-11 virtual machine architecture: A case study :

Ah, I see from rereading the above that they learned some things:

"Architectural changes contain pitfalls for the unwary. Desires to
slide hardware changes "under" an existing architecture
arise in a number of other areas. When protection and security are
important, for example, capability and domain
architectures are often proposed. Proponents are advised however
that, despite considerable early effort to
foresee difficulties, not all problems were uncovered by the UCLA
project until large portions of the detailed design were
nearly complete. A few of the hardware peculiarities mentioned in
the appendix were not noticed, and the magnitude of
certain of the sources of performance overhead sere inaccurately estimated."

That suggests to me that their work stayed an academic one off, but
at least they got that
far and learned what the pitfalls were. He also later says:

"It has been argued that one of the most promising application areas
for program verification, at least with
respect to cost effectiveness, is in code that is a) frequently
executed for many users, and b) whose failure has
significant consequences: in other words, segments of operating
systems. However, verification methods first
require an axiomatic representation of the environment in which the
programs of interest run. system
code has hardware details, rather than high level programming
language constraints alone, as part of its relevant
complicate the verification task considerably, precisely at one of
the points where it could be so useful. Until hardware
architectures are simplified, this impediment is likely too limit the
utility of operating system verification."

which again suggests some learning, more than was evident in the talks I heard.

It's also interesting to me to hear him extol the virtues of the
UNIBUS I/ architecture in the
Formal Requirements paper, e.g.

" key restriction in the model is the exclusion of I/ devices and
instructions. While it is commonplace
now to provide users with an extended software machine without
explicit I/ devices or instructions, there is one
late third generation hardware machine that exhibits this appearance.
In the DEC PDP-11, I/ devices are
treated as memory cells and I/ operations are performed by doing the
proper memory transfer to the
appropriate cell."

(back when he was trying to get access to such machines and
cooperation from DEC) and later bemoan
that same architecture once the full implications of that
architecture for a virtual machine monitor
was better understood, e.g.

"Virtualization of the PDP-11/45 is practical, although with more effort than
might be expected." and "UNIBUS style I/ architectures are generally
inefficient
with respect to virtualization."

We also had a PDP-11/45 at LLNL, though without the VM support features
they added at UCLA. That was the system that the RATS S ran on,
later upgraded
to a PDP-11/70.

You may recall that during that time frame the VMM support for the
IBM 360 line was already pretty well established. I think by the
1973/1974 time frame the virtual machine concepts were already
pretty well developed. There was even a conference or two devoted
just to that topic. As with many of the formalization efforts
of the time, what Popek and Goldberg did was to try to formalize
the requirements for a VMM, as they say:

"Formal techniques are used to derive precise sufficient conditions
to test whether such an
architecture can support virtual machines."

That was an interesting time period in computer science. It still
amazes me that virtualization
went so far under ground for so many years, though perhaps I should
be surprised that it
arose again at all. In any case it gives me hope that capability
architectures (not necessarily
hardware) can arise again.

cap-talk mailing list
cap-talk (AT) mail (DOT) eros-os.org