SSTIC 2009 Challenge vs Metasm

Fri 03 July 2009 by alex

Few weeks ago, preceding the 2009 SSTIC conference, the wonderful St├ęphane Duverger proposed the SSTIC 2009 Challenge. It has been one of the most intersting reverse challenge I've seen these last months and I will take advantage of it to describe few use cases of Metasm.

I strongly encourage you to check the reference solution proposed by Raphael Rigo and Simon Marechal cause I will not detail how the challenge works, but rather how we have used Metasm to solve it. This will be a good opportunity to introduce simple usages of its nice functionalities (backtracking, code binding, symbolism).

How to use backtracking?

The code throw a lots of exception using INT instruction and IO ports access, IN and OUT instructions. For these last two instructions, the number of the port is passed through edx register. How to find its value ? We will use the backtracking engine:

def io_port(addr)
        (Expression[@disasm16.backtrace(:edx, addr, :include_start => false)]).reduce_rec
end

For a given address (the parameter addr) the object disasm16 (16bits disassembler) will backtrace the register edx from the address addr, the instruction at address addr is not in the scope of the search (:include_start => false). The expression is then reduced so we have our port number: it means the last assignement of edx register before the IO instruction is executed.

How to use code binding?

Once an exception is triggered, it will be handler by a handler, there are 16 and 32 bits handlers but that is not our point. How can we simply understand the real behaviour of such a handler? We have used the code binding. Based on the backtracking engine, it computes the binding of a section of code (in our case a handler), and returns it as a hash of symbolic expressions (Expression objet).

Let's compute code binding for two of these handlers randomly choosen (a 16bits one and a 32bits one). We use the code_binding method from a Disassembler object.

binding = @disasm.code_binding(block.list.first.address, block.list.last.address)

A 16 bits handler:

Interruption vector 21h using IVT from virtual 8086 mode

2dbce8h mov edx, 1
2dbceeh mov ebx, dword ptr [esi+4*edx]
2dbcf3h dec edx
2dbcf5h mov eax, edx
2dbcf8h and eax, 3
2dbcfch xor eax, ebp
2dbcffh mov eax, dword ptr fs:[esi+4*eax]
2dbd05h xor eax, ecx
2dbd08h push edi
2dbd0ah push ecx
2dbd0ch push ebx
2dbd0eh xor edi, ebx
2dbd11h add eax, edi
2dbd14h shr ebx, 3
2dbd18h shl ecx, 4
2dbd1ch xor ebx, ecx
2dbd1fh mov edi, ebx
2dbd22h mov ebx, dword ptr [esp]
2dbd27h mov ecx, dword ptr [esp+4]
2dbd2dh shr ecx, 5
2dbd31h shl ebx, 2
2dbd35h xor ebx, ecx
2dbd38h add edi, ebx
2dbd3bh xor eax, edi
2dbd3eh pop ebx
2dbd40h pop ecx
2dbd42h pop edi
2dbd44h mov ecx, eax
2dbd47h mov eax, dword ptr [esi+4*edx]
2dbd4ch add ecx, eax
2dbd4fh mov dword ptr [esi+4*edx], ecx
2dbd54h iret

The computed binding:

ebx => dword ptr [esi+4]&0ffffffffh
dword ptr [esi] => (((dword ptr [segment_base_fs+esi+4*ebp]^ecx)+(edi^dword ptr [esi+4]))^((((dword ptr [esi+4]>>3)&1fffffffh)^(ecx<<4))+((dword ptr [esi+4]<<2)^((ecx>>5)&7ffffffh))))+dword ptr [esi]
eax => dword ptr [esi]&0ffffffffh
dword ptr [esp-4] => ecx
ecx => ((((dword ptr [segment_base_fs+esi+4*ebp]^ecx)+(edi^dword ptr [esi+4]))^((((dword ptr [esi+4]>>3)&1fffffffh)^(ecx<<4))+((dword ptr [esi+4]<<2)^((ecx>>5)&7ffffffh))))+dword ptr [esi])&0ffffffffh
edx => 0
dword ptr [esp-8] => dword ptr [esi+4]
dword ptr [esp] => edi

Quite simple !

Now, let's focus on the 32bits handler. Into the assembly code we found these kind of contruction using ebp and indirections like [eax+4Ch].

21BFD4h mov     eax, [eax+4Ch]
21BFD7h mov     edx, eax
21BFD9h shl     edx, 4
21BFDCh mov     eax, [ebp+8]
21BFDFh mov     eax, [eax+10h]
21BFE2h and     eax, 0FFFFh
21BFE7h lea     eax, [edx+eax]
21BFEAh add     eax, 2Eh
21BFEDh mov     [ebp-8], eax
21BFF0h mov     eax, [ebp+8]
21BFF3h mov     eax, [eax+54h]
21BFF6h mov     edx, eax

Of course, the computed binding also reveals the same phenomenon, and is more difficult to understand with lots of indirection through the stack:

ecx => ((dword ptr [dword ptr [esp+4]+4ch]<<4)+(dword ptr [dword ptr [esp+4]+10h]&0ffffh)+4)&0ffffffffh
dword ptr [esp-8] => (dword ptr [dword ptr [esp+4]+50h]<<4)+(dword ptr [dword ptr [esp+4]+10h]&0ffffh)

We will see in the next section how to clarify these expression.

How to use symbolism?

For now, let's focus on the 32bits handler. We are in an exception handler and these mysterious indirection are noting more than an indirection access to our process context through the exception context located onto the stack. How to take this point into account?

@sym_ctx = {
        Indirection[[ :esp, :+, 0x4], 4, nil] =>  Expression[ :context],
}

@sym_ctxreg ={}
25.times{|i| @symbolic_vm[Indirection[Expression[:context, :+, i*4], 4, nil]] = Expression["ctx#{i}".to_sym]}

@reg_map = {
        :ctx3 => Expression[:edi],
        :ctx4 => Expression[:esi],
        :ctx5 => Expression[:ebp],
        :ctx7 => Expression[:ebx],
        :ctx8 => Expression[:edx],
        :ctx9 => Expression[:ecx],
        :ctx10 => Expression[:eax],
}

Then for each of our expressions we only have to inject these information, the Ruby method to that is bind:

injected_expression = my_expression.bind(hash_symbols)

Expression before:

dword ptr [dword ptr [esp+4]+1ch] => dword ptr [(dword ptr [dword ptr [esp+4]+4ch]<<4)+(dword ptr [dword ptr [esp+4]+10h]&0ffffh)+8]

Expression after:

ebx => dword ptr [60008h]

Code is far most understandable using injected expression that raw binding.

Conclusion

These functionnalities are quite useful when time comes to analyze a binary. We have tried to present simple use cases which may encourage new users to cross the step and use Metasm. Feel free to ask question about the code. The script can be found in the archive (just run go.rb), once launched, for each handler it display the improved binding and few others information. I you are also interested in the decompiler, the archive also contains the example script (demo_decompilation.rb) that we have at SSTIC conference to present the decompiler (it also requires gtk).