tree a1f0a10458cc58d9a981e5a6cad55b3538ec815b
parent 2aa94575a5a7b809c593f73487f25ec4e2e90098
author Barret Rhoden <brho@cs.berkeley.edu> 1571871365 -0400
committer Barret Rhoden <brho@cs.berkeley.edu> 1571871989 -0400

vmm: reimplement the x86 instruction decoder

Our old decoder had a bunch of issues.  Whenever we get a new version of
Linux, we tend to have new instructions that need decoding.  The old
decoder was hard to extend, and it was also hiding a bunch of bugs.

Here are some of the problems the old decoder had:
- It assumed every operation was a load or store.  Including cmp (which
does not change registers/memory) and add (which does change them, but
only after adding).
- It did not set rflags
- It did not zero-extend 32-bit wide register results
- *word was busted.  At one point, we did word++ when we meant to
advance by a byte.
- etc.

The code was pretty 'swirly' too, where you'd have similar processing
repeated all over the place, like the REX checks.

To fix that, I added a 'decode' struct, to pass along values that we
determined, such as address size, operand size, rex bits, etc.

Best of all, we weren't computing the size correctly, since we didn't
really do the modrm handling right.  Here's the case:

	81 7d 00 5f 4d 50 5f    cmp    DWORD PTR [rbp+0x0],0x5f504d5f

That was being treated like it is only 4 bytes long, instead of 7.
Whoops!

However, it didn't crash, even though we set RIP to be part way (4
bytes) into the instruction!  Why?  well, those extra three bytes that
are just arbitrary numbers in the immediate32 part of the instruction
(which we end up running) decodes too!

	0:  4d 50                   rex.WRB push r8
	2:  5f                      pop    rdi

It pushes and pops, essentially clobbering rdi.  The Linux guest ends up
resetting rdi later, so no one noticed.

Had it been another value for the immed, we'd execute that too.  It
might blow up, and we'd notice.  But this one silently executed and
silently trashed a register.

To fix that, I needed better mod/rm+sib handling.  We still get away
with using GPA instead of decoding modrm+sib and translating through the
guest's page tables.  Ron's comment still applies.  =)

To handle the emulation of instructions, I had our callers pass us the
'access()' function.  So we can handle read-modify-write instructions,
like add.  Those didn't need to change too much, though I yanked out
destreg, which was just debug clutter.

I could have broken the commit up a little bit, but there wasn't a lot
of value in it, since the whole thing needed to be overhauled.

Note that the APIC_ACCESS and WRITE exits never happen.  That might have
been the case ever since we started using the x2APIC for the guest.

Signed-off-by: Barret Rhoden <brho@cs.berkeley.edu>