Windows 8 Instruction Cache

[Master674]

<something used to be here... its gone now>

[sitnspinlock]

what processor are you using? are you using the developer preview?

try something like this

for(int i=0, mask=1; i<logicalNumCpus; i++, mask << 1)
	{
		SetProcessAffinityMask((HANDLE)-1,mask);

		FlushInstructionCache(targetProcesswhatever);
	}

[Master674]

<something used to be here... its gone now>

[sitnspinlock]

just looked at the 8400 kernel and NtFlushInstructionCache is unimplemented. just returns zero.

they should really be returning something other then status_success.

[_Mike]

Try CLFLUSH--Cache Line Flush

[Master674]

<something used to be here... its gone now>

[Cypher]

From the Windows 8 RTM x64:

NTDLL x64:

0:017> uf ntdll!NtFlushInstructionCache
ntdll!NtFlushInstructionCache:
000007f9`24353901 4c8bd1          mov     r10,rcx
000007f9`24353904 b8d4000000      mov     eax,0D4h
000007f9`24353909 0f05            syscall
000007f9`2435390b c3              ret

NTDLL x86:

0:007> uf ntdll!NtFlushInstructionCache
ntdll!NtFlushInstructionCache:
770e1abc b8d4000c00      mov     eax,0C00D4h
770e1ac1 64ff15c0000000  call    dword ptr fs:[0C0h]
770e1ac8 c20c00          ret     0Ch

So it seems to be fixed now (assuming that the noop was actually in the usermode implementation rather than in the kernel, which I haven't checked).

[sitnspinlock]

From the Windows 8 RTM x64:

NTDLL x64:

0:017> uf ntdll!NtFlushInstructionCache
ntdll!NtFlushInstructionCache:
000007f9`24353901 4c8bd1          mov     r10,rcx
000007f9`24353904 b8d4000000      mov     eax,0D4h
000007f9`24353909 0f05            syscall
000007f9`2435390b c3              ret

NTDLL x86:

0:007> uf ntdll!NtFlushInstructionCache
ntdll!NtFlushInstructionCache:
770e1abc b8d4000c00      mov     eax,0C00D4h
770e1ac1 64ff15c0000000  call    dword ptr fs:[0C0h]
770e1ac8 c20c00          ret     0Ch

So it seems to be fixed now (assuming that the noop was actually in the usermode implementation rather than in the kernel, which I haven't checked).

it was the kernel implementation which returned status_success. ntdll stub was always there. it's probably updated by now.

[Greyman]

A kernel/user context switch may be sufficient on current CPU architectures to flush the instruction cache, explaining the simple return in the kernel implementation. As an example, in earlier architectures a <call> instruction was all that was required to flush the cache.

I certainly haven't had an issue in my Windows 8 install with hooks, and I'm running a pretty early beta. Is it possible you've munged something else and not realised it?

[Cypher]

A kernel/user context switch may be sufficient on current CPU architectures to flush the instruction cache, explaining the simple return in the kernel implementation. As an example, in earlier architectures a <call> instruction was all that was required to flush the cache.

I certainly haven't had an issue in my Windows 8 install with hooks, and I'm running a pretty early beta. Is it possible you've munged something else and not realised it?

I don't know which architectures exactly you're referring to, but it reminded me of this blog post by Raymond Chen:

http://blogs.msdn.com/b/oldnewthing/archive/2003/12/08/55954.aspx

However, the comments indicate that it may not have been the cache that was flushed by a call, but rather the prefetch queue. He seems equally 'hazy' on the matter though.

(Personally I have nfi. The Intel manuals are gigantic and I can't be bothered to even begin looking. :P).

[recuiuud]

Guess i would share this little

[sitnspinlock]

A kernel/user context switch may be sufficient on current CPU architectures to flush the instruction cache, explaining the simple return in the kernel implementation. As an example, in earlier architectures a <call> instruction was all that was required to flush the cache.

I certainly haven't had an issue in my Windows 8 install with hooks, and I'm running a pretty early beta. Is it possible you've munged something else and not realised it?

it's not sufficient. NtFlushInstructionCache makes use of KeStackAttachProcess which internally modifies the page directory for the current task. Current intel and amd cpu's store a pointer to the base of the page directory in cr3. A privileged 'mov cr3, rax' invalidates the translation lookaside buffer for instruction and data caches for whichever logical processor is currently in execution.

[Cypher]

Guess i would share this little

I don't know what you linked, but it's unavailable. Perhaps try hosting them elsewhere.

it's not sufficient. NtFlushInstructionCache makes use of KeStackAttachProcess which internally modifies the page directory for the current task. Current intel and amd cpu's store a pointer to the base of the page directory in cr3. A privileged 'mov cr3, rax' invalidates the translation lookaside buffer for instruction and data caches for whichever logical processor is currently in execution.

I just checked the system call table in ntoskrnl and the entry that is supposed to point to NtFlushInstructionCache points instead to xHalGetInterruptTranslator which is simply a nullsub:

PAGE:0000000140426FD0                 public xHalGetInterruptTranslatorPAGE:0000000140426FD0 xHalGetInterruptTranslator proc near    ; CODE XREF: AnFwpBackgroundUpdateTimer+Fp
PAGE:0000000140426FD0                                         ; AnFwDisplayBackgroundUpdate+5A68p ...
PAGE:0000000140426FD0                 xor     eax, eax
PAGE:0000000140426FD2                 retn
PAGE:0000000140426FD2 xHalGetInterruptTranslator endp

I'm not a kernel expert though (or even a kernel 'enthusiast'). Can somebody else take a look at this and confirm what I'm seeing? Perhaps we should take this to the OSR Mailing List or Connect or something?

EDIT:

Just for some added information, here is the W7 x64 implementation. It is just as everdox describes:

PAGE:00000001402EA4BC NtFlushInstructionCache proc near       ; DATA XREF: .text:0000000140080410oPAGE:00000001402EA4BC
PAGE:00000001402EA4BC var_58          = dword ptr -58h
PAGE:00000001402EA4BC var_50          = qword ptr -50h
PAGE:00000001402EA4BC var_48          = qword ptr -48h
PAGE:00000001402EA4BC var_38          = byte ptr -38h
PAGE:00000001402EA4BC arg_8           = qword ptr  10h
PAGE:00000001402EA4BC
PAGE:00000001402EA4BC                 push    rbx
PAGE:00000001402EA4BE                 sub     rsp, 70h
PAGE:00000001402EA4C2                 mov     r10, rcx
PAGE:00000001402EA4C5                 mov     rax, gs:188h
PAGE:00000001402EA4CE                 mov     r9b, [rax+1F6h]
PAGE:00000001402EA4D5                 test    rdx, rdx
PAGE:00000001402EA4D8                 jz      short loc_1402EA505
PAGE:00000001402EA4DA                 test    r8, r8
PAGE:00000001402EA4DD                 jz      short loc_1402EA512
PAGE:00000001402EA4DF                 test    r9b, r9b
PAGE:00000001402EA4E2                 jz      short loc_1402EA505
PAGE:00000001402EA4E4                 test    r8, r8
PAGE:00000001402EA4E7                 jz      short loc_1402EA501
PAGE:00000001402EA4E9                 lea     rcx, [rdx+r8]
PAGE:00000001402EA4ED                 mov     rax, cs:MmUserProbeAddress
PAGE:00000001402EA4F4                 cmp     rcx, rax
PAGE:00000001402EA4F7                 ja      short loc_1402EA4FE
PAGE:00000001402EA4F9                 cmp     rcx, rdx
PAGE:00000001402EA4FC                 jnb     short loc_1402EA501
PAGE:00000001402EA4FE
PAGE:00000001402EA4FE loc_1402EA4FE:                          ; CODE XREF: NtFlushInstructionCache+3Bj
PAGE:00000001402EA4FE                 mov     byte ptr [rax], 0
PAGE:00000001402EA501
PAGE:00000001402EA501 loc_1402EA501:                          ; CODE XREF: NtFlushInstructionCache+2Bj
PAGE:00000001402EA501                                         ; NtFlushInstructionCache+40j
PAGE:00000001402EA501                 jmp     short loc_1402EA505
PAGE:00000001402EA503 ; ---------------------------------------------------------------------------
PAGE:00000001402EA503                 jmp     short loc_1402EA514
PAGE:00000001402EA505 ; ---------------------------------------------------------------------------
PAGE:00000001402EA505
PAGE:00000001402EA505 loc_1402EA505:                          ; CODE XREF: NtFlushInstructionCache+1Cj
PAGE:00000001402EA505                                         ; NtFlushInstructionCache+26j ...
PAGE:00000001402EA505                 xor     ebx, ebx
PAGE:00000001402EA507                 cmp     r10, 0FFFFFFFFFFFFFFFFh
PAGE:00000001402EA50B                 jnz     short loc_1402EA51A
PAGE:00000001402EA50D
PAGE:00000001402EA50D loc_1402EA50D:                          ; CODE XREF: NtFlushInstructionCache+A5j
PAGE:00000001402EA50D                 test    rbx, rbx
PAGE:00000001402EA510                 jnz     short loc_1402EA563
PAGE:00000001402EA512
PAGE:00000001402EA512 loc_1402EA512:                          ; CODE XREF: NtFlushInstructionCache+21j
PAGE:00000001402EA512                                         ; NtFlushInstructionCache+BAj
PAGE:00000001402EA512                 xor     eax, eax
PAGE:00000001402EA514
PAGE:00000001402EA514 loc_1402EA514:                          ; CODE XREF: NtFlushInstructionCache+47j
PAGE:00000001402EA514                                         ; NtFlushInstructionCache+96j
PAGE:00000001402EA514                 add     rsp, 70h
PAGE:00000001402EA518                 pop     rbx
PAGE:00000001402EA519                 retn
PAGE:00000001402EA51A ; ---------------------------------------------------------------------------
PAGE:00000001402EA51A
PAGE:00000001402EA51A loc_1402EA51A:                          ; CODE XREF: NtFlushInstructionCache+4Fj
PAGE:00000001402EA51A                 and     [rsp+78h+var_48], rbx
PAGE:00000001402EA51F                 lea     rax, [rsp+78h+arg_8]
PAGE:00000001402EA527                 mov     [rsp+78h+var_50], rax
PAGE:00000001402EA52C                 mov     [rsp+78h+var_58], 746C6644h
PAGE:00000001402EA534                 mov     r8, cs:PsProcessType
PAGE:00000001402EA53B                 mov     edx, 20h
PAGE:00000001402EA540                 mov     rcx, r10
PAGE:00000001402EA543                 call    ObReferenceObjectByHandleWithTag
PAGE:00000001402EA548                 mov     rbx, [rsp+78h+arg_8]
PAGE:00000001402EA550                 test    eax, eax
PAGE:00000001402EA552                 js      short loc_1402EA514
PAGE:00000001402EA554                 lea     rdx, [rsp+78h+var_38]
PAGE:00000001402EA559                 mov     rcx, rbx
PAGE:00000001402EA55C                 call    KeStackAttachProcess
PAGE:00000001402EA561                 jmp     short loc_1402EA50D
PAGE:00000001402EA563 ; ---------------------------------------------------------------------------
PAGE:00000001402EA563
PAGE:00000001402EA563 loc_1402EA563:                          ; CODE XREF: NtFlushInstructionCache+54j
PAGE:00000001402EA563                 lea     rcx, [rsp+78h+var_38]
PAGE:00000001402EA568                 call    KeUnstackDetachProcess
PAGE:00000001402EA56D                 mov     rcx, rbx        ; Object
PAGE:00000001402EA570                 call    ObfDereferenceObject
PAGE:00000001402EA575                 nop
PAGE:00000001402EA576                 jmp     short loc_1402EA512
PAGE:00000001402EA576 NtFlushInstructionCache endp

EDIT:

More confirmation that the function has been removed:

http://j00ru.vexillium.org/dump/win7...skrnl.exe.html

[Master674]

<something used to be here... its gone now>

[_Mike]

I don't know what you linked, but it's unavailable. Perhaps try hosting them elsewhere.

Spam bot, look at his post history.

This also affects system calls that are supposed to force a flush. WriteProcessMemory for example.

I can write whatever I want to this function, it will never get flushed but breakpoint hits though.
Cencil has the same troubles with some other hook. Everything is perfectly fine in Windows 7 or below.

Did you try the clflush instruction? Unless I'm misunderstanding the intel developers manuals it should do what you want.