VB6拾遗：调用C函数（__cdecl调用约定）

VB中，无论是自定义函数还用Declare声明的函数，都遵循__stdcall调用约定。

而在C语言中默认的调用约定是__cdecl，C标准库函数亦是如此，那么如何在VB中调用C函数呢？

这让我想起了很久以前，一位网友问我，如何让WriteConsole函数支持重定向？查了一下MSDN，上面是这么写的：

WriteConsole fails if it is used with a standard handle that is redirected to a file.

当标准（输出）句柄被重定向到文件时，WriteConsole函数会失败。于是我说，用printf函数。他说，VB用不了printf函数。当时不懂VB，最后是用WriteFile函数解决的。

扯远了，首先看一下__cdecl和__stdcall有何不同。前面说过，在__stdcall调用约定中，所有的参数按从右至左的顺序入栈，并且由被调用的函数来维持堆栈平衡；与__stdcall一样，在__cdecl调用约定中所有的参数也是按从右至左的顺序入栈，不同的是由函数的调用者来维持堆栈平衡。

int __stdcall stdcall_add(int a, int b)
{
    return a + b;
}

int __cdecl cdecl_add(int a, int b)
{
    return a + b;
}

int main()
{
    int x, y;
    x = stdcall_add(0x123, 0x456);
    y = cdecl_add(0x123, 0x456);
    return 0;
}

生成的汇编代码如下：

004010A8  |.  68 56040000   push    0x456
004010AD  |.  68 23010000   push    0x123
004010B2  |.  E8 4EFFFFFF   call    00401005                  ; __stdcall
                                                              ; 调用者不需要维持堆栈平衡
004010B7  |.  8945 FC       mov     dword ptr [ebp-0x4], eax
004010BA  |.  68 56040000   push    0x456
004010BF  |.  68 23010000   push    0x123
004010C4  |.  E8 41FFFFFF   call    0040100A                  ; __cdecl
004010C9  |.  83C4 08       add     esp, 0x8                  ; 由调用者维持堆栈平衡
004010CC  |.  8945 F8       mov     dword ptr [ebp-0x8], eax

可以看到，__cdecl函数在调用之后需要调用者为esp加上一个值来维持堆栈平衡，而__stdcall不需要。

所以在调用__stdcall的函数指针时，我们可以像《VB6拾遗：调用函数指针》中那样直接jmp到函数指针，让函数调用完后直接返回到上一层，而不必担心堆栈平衡的问题。但是如果想调用__cdecl函数指针，问题就没有那么简单了，在调用完函数之后必须返回到我们构造的汇编代码中，由我们来完成清理堆栈的任务（即维持堆栈平衡），然后才能返回上一层。

Matthew Curland大神也提供了一个CDECLFunctionDelegator对象，由于版权原因，完整的代码我不会贴上来，仅摘抄核心的汇编代码注释（这属于著作权法中的合理使用，不算侵权）：

'Here's the assembly code to translate a stdcall vtable call into
'a cdecl non-vtable call.  This code requires the stack size to be known.
'The whole point of this is to make a cdecl call, then clean the stack after
'the call so that it looks like a stdcall.  In order to do this, we need to
'store our data on the stack and then call the cdecl function.  This requires
'that we duplicate the parameters for the call above our stack data and push
'our own base pointer as a reference.  After the function returns, we use the
'base pointer to relocate our own values and remove the correct number of bytes
'from the stack.

'#define _PAGESIZE_ 0x1000
'push ebp                       // Run some prolog code
'mov ebp, esp                   // this = [ebp + 8], return = [ebp + 4], old ebp = [ebp]
'push esi
'push edi
'push ebx
'mov eax, [ebp + 8]             // Get this pointer
'mov ecx, [eax + 8]             // Get byte count into ecx
'mov ebx, ecx                   // Save the stacksize in ebx
'
'mov edi, esp                   // Make sure we have the stack safely loaded
'
'probepages:
'cmp ecx, _PAGESIZE_            // See if we need more than one page of stack
'jb short lastpage              // Note that this is very unlikely, but we must be safe.
'
'sub ecx, _PAGESIZE_            // yes, move down a page
'sub edi, _PAGESIZE_            // adjust request and...
'
'test DWORD PTR [edi], ecx      // ...probe it
'
'jmp short probepages           // Keep going
'
'lastpage:
'sub edi, ecx                   // Do a final probe
'test DWORD PTR [edi], ecx
'
'mov ecx, ebx                   // Reload ecx in case probing changed it
'
'mov esi, ebp                   // Establish the source pointer for the stack copy
'Add esi, 12
'
'mov esp, edi                   // Move the stack down before we lose edi
'
'shr ecx, 2                     // Change the byte stack size in ecx to a DWORD count
'cld                            // Copy ascending
'rep movsd                      // Do the stack copy (the DWORD count is in ecx)
'
'call DWORD PTR [eax + 4]       // Make the cdecl function call (the this pointer is still in eax)
'
'mov ecx, ebp                   // Move the return value to the correct position on the stack
'add ecx, 8                     // Add to move past this and function return values
'add ecx, ebx                   // Add extra stack size
'mov esi, [ebp + 4]             // Get return address.  Use esi since eax/edx hold the return value.
'mov [ecx], esi                 // Assign return address to correct position on stack.
'
'mov esp, ebp                   // Move the stack and restore the saved registers
'sub esp, 12
'pop ebx
'pop edi
'pop esi
'pop ebp
'mov esp, ecx                   // Move the stack pointer down
'ret                            // return to the calling function

可以看，跟调用__stdcall函数指针的代码相比，这个要复杂得多，理解起来也比较困难。其核心思想是构造自己的堆栈，把参数复制到堆栈中，call函数指针，然后清除堆栈，最后返回。

这样做未免太复杂了点，我想到一个更好的方法来实现，下面是我写的CFunctionDelegator对象：

Option Explicit

Private Declare Function CoTaskMemAlloc Lib "ole32.dll" (ByVal cb As Long) As Long
Private Declare Sub CoTaskMemFree Lib "ole32.dll" (ByVal pv As Long)
Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" ( _
    Destination As Any, Source As Any, ByVal Length As Long)
Private Declare Sub ZeroMemory Lib "kernel32" Alias "RtlZeroMemory" ( _
    dest As Any, ByVal numBytes As Long)

' __cdecl function delegator
' by Demon
' https://demon.tw

Private Type CFunctionDelegatorVTable
    VTable(3) As Long
End Type

Private Type CFunctionDelegator
    pVTable As Long
    pfn As Long
    StackSize As Long
    cRefs As Long
End Type

Private Type DelegateASM
    ASM(9) As Long
End Type

Private m_VTable As CFunctionDelegatorVTable
Private m_pVTable As Long
Private m_ReturnAddress As Long
Private m_DelegateASM As DelegateASM

Public Function CreateCFunctionDelegator( _
    ByVal pfn As Long, Optional ByVal StackSize As Long) As IUnknown

    Dim Struct As CFunctionDelegator
    Dim ThisPtr As Long
    
    If m_pVTable = 0 Then
        With m_VTable
            .VTable(0) = FuncAddr(AddressOf QueryInterface)
            .VTable(1) = FuncAddr(AddressOf AddRef)
            .VTable(2) = FuncAddr(AddressOf Release)
            
            With m_DelegateASM
                .ASM(0) = &H58FFF8B
                .ASM(1) = VarPtr(m_ReturnAddress)
                .ASM(2) = &H68FF8B58
                .ASM(3) = VarPtr(m_DelegateASM) + &H14
                .ASM(4) = &H900460FF
                .ASM(5) = &HC481FF8B
                .ASM(6) = StackSize
                .ASM(7) = &H35FFFF8B
                .ASM(8) = VarPtr(m_ReturnAddress)
                .ASM(9) = &HCCCCCCC3
            End With
            .VTable(3) = VarPtr(m_DelegateASM)
            
            m_pVTable = VarPtr(.VTable(0))
        End With
    End If
    
    ThisPtr = CoTaskMemAlloc(LenB(Struct))
    If ThisPtr = 0 Then Err.Raise 7
    
    With Struct
        .pVTable = m_pVTable
        .pfn = pfn
        .StackSize = StackSize
        .cRefs = 1
    End With
    
    CopyMemory ByVal ThisPtr, Struct.pVTable, LenB(Struct)
    ZeroMemory Struct.pVTable, LenB(Struct)
    
    CopyMemory CreateCFunctionDelegator, ThisPtr, 4
End Function

Private Function QueryInterface( _
    This As CFunctionDelegator, riid As Long, pvObj As Long) As Long

    With This
        pvObj = VarPtr(.pVTable)
        .cRefs = .cRefs + 1
    End With
End Function

Private Function AddRef(This As CFunctionDelegator) As Long
    With This
        .cRefs = .cRefs + 1
        AddRef = .cRefs
    End With
End Function

Private Function Release(This As CFunctionDelegator) As Long
    With This
        .cRefs = .cRefs - 1
        Release = .cRefs
        If .cRefs = 0 Then
            DeleteThis This
        End If
    End With
End Function

Private Function FuncAddr(ByVal pfn As Long) As Long
    FuncAddr = pfn
End Function

Private Sub DeleteThis(This As CFunctionDelegator)
    Dim tmp As CFunctionDelegator
    Dim pThis As Long
    
    pThis = VarPtr(This)
    CopyMemory ByVal VarPtr(tmp), ByVal pThis, LenB(This)
    CoTaskMemFree pThis
End Sub

核心部分的汇编代码如下：

8BFF            mov     edi, edi                ; 对齐
8F05 XXXXXXXX   pop     dword ptr [XXXXXXXX]    ; XXXXXXXX为VarPtr(m_ReturnAddress)
                                                ; 即把返回地址保存到m_ReturnAddress
58              pop     eax                     ; this指针
8BFF            mov     edi, edi                ; 对齐
68 XXXXXXXX     push    XXXXXXXX                ; XXXXXXXX为VarPtr(m_DelegateASM) + &H14
                                                ; 即nop指令后一条指令mov edi, edi的地址
                                                      ; 这样C函数调用完后返回到mov edi, edi
FF60 04         jmp     dword ptr [eax+0x4]     ; 调用C函数
90              nop                             ; 对齐
8BFF            mov     edi, edi                ; 对齐
81C4 XXXXXXXX   add     esp, XXXXXXXX           ; XXXXXXXX为StackSize
                                                ; 即调用C函数后需要平衡的堆栈大小
8BFF            mov     edi, edi                ; 对齐
FF35 XXXXXXXX   push    dword ptr [XXXXXXXX]    ; XXXXXXXX为VarPtr(m_ReturnAddress)
                                                ; 即把返回地址入栈
C3              retn                            ; 函数返回
CC              int3                            ; 对齐
CC              int3                            ; 对齐
CC              int3                            ; 对齐

是不是比Matthew Curland大神的简单一些？

代码已经注释得很详细了，但还是简单说一下。这里的对齐并不是为了提高效率的内存对齐，只是为了方便构造代码而已。本来想用ecx寄存器来保存返回地址的，没想到一些C标准库函数居然会修改ecx的值，只好保存到变量中去了。

一个简单的示例，调用之前说的VB不能用的printf函数：

Option Explicit

Private Declare Function LoadLibrary Lib "kernel32" Alias "LoadLibraryA" ( _
    ByVal lpLibFileName As String) As Long

Private Declare Function GetProcAddress Lib "kernel32" ( _
    ByVal hModule As Long, ByVal lpProcName As String) As Long

' call __cdecl function
' by Demon
' https://demon.tw

Sub Main()
    Dim hModule As Long
    Dim pFunc As Long
    Dim Iprintf As Iprintf
    
    hModule = LoadLibrary("msvcrt.dll")
    If hModule Then
        pFunc = GetProcAddress(hModule, "wprintf")
        If pFunc Then
            Set Iprintf = CreateCFunctionDelegator(pFunc, 4)
            Iprintf.Call "Hello World"
        End If
    End If
End Sub

Iprintf接口定义：

[
  uuid(82B585FA-99B0-4CC2-92B2-58FA7D909BF9),
  version(1.0)
]
library NewLib
{
    importlib("stdole2.tlb");

    [
      odl,
      uuid(898C0719-F770-478B-ACCA-B6CE4B62E570),
      nonextensible
    ]
    interface Iprintf : IUnknown {
        long _stdcall Call([in] BSTR format);
    };
};

为了能够看到控制台输出，还需要用PE工具（例如Load PE）把生成的EXE的子系统修改成Win32 Console。折腾了半天，终于看到了熟悉的Hello World。

不要高兴得太早，能够调用C函数指针只算成功了一半，一些C函数需要__cdecl函数指针作为参数，例如qsort，而VB的函数都是__stdcall函数。要想调用这类函数，又得费不少功夫，我是不想死脑细胞了，自己去找Matthew Curland的代码看看吧。我想除了他，没有人会变态到在VB中调用C标准库的qsort函数。

最后提醒一句，我写的CFunctionDelegator对象是有瑕疵的，不要不经思考地直接照抄。

赞赏

微信赞赏支付宝赞赏

随机文章:

1. 利用 WindowsInstaller.Installer 对象计算文件 MD5 hash 值
2. NDS上的GB模拟器Lameboy
3. 用VBS精确计算100的阶乘
4. VBS获取重定向的URL
5. 64位系统VBS调用32位COM组件

这篇文章发布于 2013年09月23日，星期一，11:04，归类于 程序设计。 您可以跟踪这篇文章的评论通过 RSS 2.0 feed。 您可以留下评论，或者从您的站点trackback。