Use C Library in EDK II UEFI Driver

When we use C library in EDK II UEFI Driver, build failure occurs. Please refer UEFI原理與編程 for the detail. This page describes my way to solve it.

We want to use sprintf() of C library in UEFI driver to provide the driver information (e.g., build-date). I like to write the code as below,

#include <stdio.h>

EFI_STATUS
EFIAPI
LibCDxeEntryPoint (
  IN EFI_HANDLE ImageHandle,
  IN EFI_SYSTEM_TABLE *SystemTable
  )
{
  CHAR8 Info [256];
            
  sprintf (Info, "Build Date: %s", __DATE__);

  return EFI_SUCCESS;
}  

, but build failure occurs. Please,

1. Add the following statements in your package DSC file.

These statements declare that we will build C library for UEFI driver.

[LibraryClasses.Common.UEFI_DRIVER]  
  ShellLib|ShellPkg/Library/UefiShellLib/UefiShellLib.inf
  FileHandleLib|MdePkg/Library/UefiFileHandleLib/UefiFileHandleLib.inf  
  SortLib|MdeModulePkg/Library/UefiSortLib/UefiSortLib.inf
  
  LibC|StdLib/LibC/LibC_ForDriver.inf
  #LibC|StdLib/LibC/LibC.inf
  LibCType|StdLib/LibC/Ctype/Ctype.inf  
  LibLocale|StdLib/LibC/Locale/Locale.inf  
  LibSignal|StdLib/LibC/Signal/Signal.inf  
  LibStdio|StdLib/LibC/Stdio/Stdio.inf  
  LibStdLib|StdLib/LibC/StdLib/StdLib.inf
  LibString|StdLib/LibC/String/String.inf  
  LibTime|StdLib/LibC/Time/Time.inf  
  LibUefi|StdLib/LibC/Uefi/Uefi.inf  
  LibWchar|StdLib/LibC/Wchar/Wchar.inf  
  
  LibGen|StdLib/PosixLib/Gen/LibGen.inf  
  LibIIO|StdLib/LibC/Uefi/InteractiveIO/IIO.inf

  LibContainer|StdLib/LibC/Containers/ContainerLib.inf   
  
  LibGdtoa|StdLib/LibC/gdtoa/gdtoa.inf  
  DevConsole|StdLib/LibC/Uefi/Devices/daConsole.inf
  DevUtility|StdLib/LibC/Uefi/Devices/daUtility.inf

2. Use LibC_ForDriver.inf instead of LibC.inf.

We removes ShellCEntryLib from LibC_ForDriver.inf. If we don't do it, the error occurs.

error 1001: Module type [UEFI_DRIVER] is not supported by library instance [...\MdePkg\Library\UefiApplicationEntryPoint\UefiApplicationEntryPoint.inf]

LibC_ForDriver.inf

[LibraryClasses]
#  ShellCEntryLib
  UefiLib
  BaseLib
  BaseMemoryLib
  MemoryAllocationLib
  LibStdLib
  LibStdio
  LibString
  DevConsole

3. Add dummy main().

After we follow the steps, the link error occurs, the main symbol is not defined. Please add dummy main() in your driver.

int
main (
  IN int Argc,
  IN char **Argv
  )
{
  return 0;
}

Now we can build the UEFI driver with C standard library.

-Count

Make Your Code Portable

The following are principles to make your code portable.

1) Use C instead of C++.

If you want your library more portable, please use C instead of C++.

2) Use C standard library.

Because EDK II supports C standard library, we just use it.

Please refer Use C Library in EDK II UEFI Driver. 

3) As far as possible, allocate memory space for a returned parameter before calling a public function.

For example,

EFI_STATUS
CompressData (
  IN UINT8 *Buffer,
  IN UINTN BufferSize,
  OUT UINT8 **Output,
  OUT UINTN *OutputSize
  );

This Compress function allocates memory space for the returned Output. The problem is that the memory management mechanisms are different in environments. In UEFI DXE environment, gBS->AllocatePool() supports the memory space allocation. In SMM mode, UEFI provides SmmAllocatePool(). In Windows KMDF, we should use WdfMemoryCreate to allocate memory space.

Therefore please rework the function as below otherwise we need to create many stubs to wrap the memory management functions for different environments.

EFI_STATUS
GetCompressedDataSize (
  IN UINT8 *Buffer,
  IN UINTN BufferSize,
  OUT UINTN *OutputSize
  );

EFI_STATUS
CompressData (
  IN UINT8 *Buffer,
  IN UINTN BufferSize,
  OUT UINT8 *Output,
  IN UINTN OutputSize
  );

4) The sizes of fields in C struct must be determined.

Don't use UINTN or ulong, the sizes are undetermined, in your struct. If you define struct in UEFI environment, please use UINT8, UINT16, UINT32, CHAR8, CHAR16 of which sizes are always same in IA32 and X64.

-Count

Use C++ in EDK II

There are two ways to write C++ code in EDK II.
1. Make your whole files as CPP
2. Low level files are CPP, High level files are C.

I prefer the second way because we always use C to develop UEFI driver and sometimes we want to reuse C++ components.

Please refer the UEFI原理与编程 for the detail of C++ in EDK II, where my idea comes from.

Below is my source code of UEFI application that uses C++ directly.

CppTest.inf -

The INF file defines the UEFI application, CppTest.c calls C functions of Wrap.cpp that wraps C++ of MyObject.cpp.

[Sources]
  CppTest.c
  MyObject.cpp
  MyObject.h
  Wrap.cpp
  Wrap.h

MyObject.h -

Declare the class MY_OBJECT.

class MY_OBJECT
{
  private:
    UINT8 Value1;
    UINT8 Value2;

  public:
    MY_OBJECT (UINT8 Value1, UINT8 Value2);
    UINT8 GetValue1 ();
    UINT8 GetValue2 ();    
    ~MY_OBJECT ();
};

MyObject.cpp - 

Implement the class MY_OBJECT.

MY_OBJECT::MY_OBJECT (
  IN UINT8 Value1, 
  IN UINT8 Value2
  )
{
  this->Value1 = Value1;
  this->Value2 = Value2;
}

UINT8 
MY_OBJECT::GetValue1 ()
{
  return this->Value1;
}

UINT8 
MY_OBJECT::GetValue2 ()
{
  return this->Value2;
}

MY_OBJECT::~MY_OBJECT ()
{
  this->Value1 = 0;
  this->Value2 = 0;
}

Wrap.h - 

Declare the C functions that wraps the class MY_OBJECT. The term, extern "C", declares that the following functions are name mangling in C. The value of __cplusplus is 1 when CPP file includes the H file so that the term enables. That is why we use #ifdef __cplusplus to wrap the term.

#ifdef __cplusplus
extern "C" {
#endif

VOID 
EFIAPI
CreateMyObject (VOID);

EFI_STATUS
EFIAPI
GetValuesOfMyObject (
  OUT CHAR8 *Buffer,
  IN UINTN BufferSize
  );
  
VOID 
EFIAPI
DestroyMyObj (VOID);  

#ifdef __cplusplus
}
#endif

Wrap.cpp -

Implement the C wrapping functions. We use the term, extern "C", previous to each function.It is interesting that we implement new and delete operators. If we don't do it, link errors about new and delete occurs.

MY_OBJECT *mMyObject;

void * operator new (size_t Size)
{
  VOID *RetVal;
  RetVal = AllocatePool (Size);
  return RetVal;
}

void operator delete (void *p)
{
  FreePool (p);
}

extern "C"
VOID 
EFIAPI
CreateMyObject (VOID)
{
  mMyObject = new MY_OBJECT (10, 20);
}

extern "C"
EFI_STATUS
EFIAPI
GetValuesOfMyObject (
  OUT CHAR8 *Buffer,
  IN UINTN BufferSize
  )
{
  UINTN i;
  i = 0;

  if (i == BufferSize) {
    return EFI_BUFFER_TOO_SMALL;
  }
  Buffer[i++] = mMyObject->GetValue1 ();

  if (i == BufferSize) {
    return EFI_BUFFER_TOO_SMALL;
  }
  Buffer[i++] = mMyObject->GetValue2 ();
  
  return EFI_SUCCESS;
}  

extern "C"
VOID 
EFIAPI
DestroyMyObj (VOID)
{
  delete mMyObject;
}

CppTest.c -

The C main program that tests the C wrapping function.

int
main (
  IN int Argc,
  IN char **Argv
  )
{
  UINT8 Buffer [10];
  MY_STRUCT *MyStruct;
  
  CreateMyObject ();
  
  GetValuesOfMyObject (Buffer, 10);
  printf ("Buffer [0] = %d\n", Buffer [0]);
  printf ("Buffer [1] = %d\n", Buffer [1]);
  
  DestroyMyObj ();
  
  return 0;
}


-Count

Link C++ Library in EDK II Module

It is hard to write C++ code in cpp and put it in EDK II INF file to build a driver with EDK2 build system because build failures occur. There are two ways to solve it.

1) We use native C++ compiler (e.g., VC++) to pack C++ code in a library with wrapped pure C function first, copy the library in your UEFI driver source code, and put the lib file in the EDK II INF file.

2) We write C++ code in a cpp file and to put the cpp file in the driver INF file.

This page describes the first way. I will describe the second way in another page, Use C++ in EDK II.

When we select the first way, build failure occurs on link errors,

error LNK2001: unresolved external symbol "void __cdecl `eh vector constructor iterator' ...

error LNK2001: unresolved external symbol "void __cdecl `eh vector destructor iterator' ...

The root cause is, we use new/delete operator to create/destroy an object array. VC++ needs the both symbols to handle the new-delete-object-array operations. The errors don't occur when building a Windows program because Windows environment provides symbols in a Windows specific library. But the link error occurs in UEFI building environment because it lacks the library.

I cannot find a regular and simple solution. My workaround is to modify C++ code to avoid to use new operator in class array.

MY_OBJ *ObjArray = new MY_OBJ [100];
delete[] ObjArray;                  

I use malloc/free instead of new/delete as below for an object array;

MY_OBJ **ObjArray = (MY_OBJ **) malloc (sizeof (MY_OBJ *) * 100);
for (int i = 0; i < 100; i++) {                                  
  ObjArray [i] = new MY_OBJ;                                     
}                                                                
                                                                 
for (int i = 0; i < 100; i++) {                                  
  delete ObjArray [i];                                           
}                                                                
free (ObjArray);                                                 

The regular solution is to use C struct instead of C++ class for platform portability.

Another link error occurs as below.

error LNK2001: unresolved external symbol __CxxFrameHandler3

The symbol __CxxFrameHandler3 is to check buffer overflow. The UEFI building environment lacks the symbol so that the link error occurs.

My solution is to create a dummy __CxxFrameHandler3 function in UEFI source code to avoid the link error.

void __CxxFrameHandler3 (void)
{
  return;
}

-Count

Observe Reclaiming Process of EDKII Variable Services

This page provides a way to observe reclaiming process of EDKII Variable Services with UEFI Fault Tolerant Write Protocol.

EDKII Variable Services provides reclaim process with the recovery mechanism. When the platform restarts after power-off happens on NV reclaiming process, the driver checks the flag NV_FTW_WORKING that the NV reclaiming is not completed, it restores variables from NV_FTW_SPARE into NV_VARIABLE_STORE. Below files which are generated in build-time provide the regions information.

FlashMap.h
#define FLASH_REGION_NVSTORAGE_SUBREGION_NV_VARIABLE_STORE_OFFSET 0x590000
#define FLASH_REGION_NVSTORAGE_SUBREGION_NV_FTW_WORKING_OFFSET    0x5ae000
#define FLASH_REGION_NVSTORAGE_SUBREGION_NV_FTW_SPARE_OFFSET      0x5b0000

AutoGenFlashMap.h
#define _PCD_VALUE_PcdFlashNvStorageVariableBase    0xFFF90000U
#define _PCD_VALUE_PcdFlashNvStorageVariableSize    0x0001D000U
#define _PCD_VALUE_PcdFlashNvStorageFtwWorkingBase  0xFFFAE000U
#define _PCD_VALUE_PcdFlashNvStorageFtwWorkingSize  0x00002000U
#define _PCD_VALUE_PcdFlashNvStorageFtwSpareBase    0xFFFB0000U
#define _PCD_VALUE_PcdFlashNvStorageFtwSpareSize    0x00020000U

Please follow the steps to observe the reclaiming process.

Step 1. Burn BIOS in ROM and dump it with DediProg to observe the region of NV_FTW_WORKING.

We see the header of NV_FTW_WORKING. The header is used to check if data is successfully written by FTW protocol. If the writing data is completed, FTW driver set a flag in NV_FTW_WORKING to indicator the completion.

Step 2. Boot UEFI shell and observe NV_FTW_WORKING.

We use a tool to observe the region of NV_FTW_WORKING.

> ToolA.efi -read -id 3 -lba 1 -offset e000 -size 70

We see that the region only has header.

Step 3. Make variable reclaiming.

We use a tool to make reclaiming process happen.

> ToolB.efi -tc1:rt=1,size=1000

Step 4. Observe NV_FTW_WORKING and NV_FTW_SPARE

We use a tool to observe the region of NV_FTW_WORKING.

> ToolA.efi -read -id 3 -lba 1 -offset e000 -size 70

We a tool to observe the region of NV_FTW_SPARE.

> ToolA.efi -read -id 3 -lba 2 -offset 0000 -size 70

We find that the NV_FTW_SPARE is empty because FTW erased it after the large data were written successfully in NV variable store. If we disable the following code in FaultTolerantWrite.c,

#if 0
  Status  = FtwEraseSpareBlock (FtwDevice);
  Ptr     = SpareBuffer;
  for (Index = 0; Index < FtwDevice->NumberOfSpareBlock; Index += 1) {
    MyLength = FtwDevice->BlockSize;
    Status = FtwDevice->FtwBackupFvb->Write (
                                        FtwDevice->FtwBackupFvb,
                                        FtwDevice->FtwSpareLba + Index,
                                        0,
                                        &MyLength,
                                        Ptr
                                        );
    if (EFI_ERROR (Status)) {
      FreePool (SpareBuffer);
      DPRINTF_INST ("}..19\n");
      return EFI_ABORTED;
    }

    Ptr += MyLength;
  }
#endif

and repeat the step 1, 2, and 3 again, we can find that the the NV_FTW_SPARE stores the backup data.

> ToolA.efi -read -id 3 -lba 2 -offset 0000 -size 70

It proofs that the region of NV_FTW_SPARE is built by UEFI Fault Tolerant Write Protocol. The region is a backup of NV_VARIABLE_STORE that is reclaimed.

How to Tweak EDKII BaseTools

Sometimes BIOS engineers need to tweak EDKII BaseTools to add some enhancements. Most of EDKII build tools are developed in Python scripts that are frozen into executables (EXE files) by cx_freeze. The BaseTools directory contains Python source code. The path is, for example.

D:\EDKII\BaseTools\Source\Python

We can modify them and rebuilt them into executables. This page teach us how to do it. If you have modified Python scripts files in the BaseTools directory, please follow the following steps to build executables.

Step 1. Install Python 2.7.

Please make sure that if the Python 2.7 is installed. If not, please go to python website, download it, and install it. The installed path is, for example.
C:\Python27

Step 2. Install cx_freeze.

Please check if cx_freeze installs in Python 2.7. How do we check it? Just run dir.
C:\Python27\Scripts> dir *freeze*.*

If the cx_freeze has been installed. The dir command displays as follows.

    68 cxfreeze
 1,256 cxfreeze-postinstall
    78 cxfreeze-quickstart
    78 cxfreeze-quickstart.bat
    67 cxfreeze.bat

If the cx_freeze has not been installed, please go to the page. http://cx-freeze.sourceforge.net

Please click the PyPI link in the page,  download cx_Freeze-4.3.3.win32-py2.7.exe, and install it.

Step 3. Check if the cxfreeze.bat exists.

Sometimes the cxfreeze.bat is missed after we install cx_freeze. If the file doesn't exist in python script path, (e.g., C:\Python27\Scripts), please manually create it as follows.

cxfreeze.bat
@echo off
C:\Python27\python.exe C:\Python27\Scripts\cxfreeze %*

Step 4. Go to BaseTools directory. 

For example,
D:\EDKII\BaseTools\Source\Python

Step 5. Check the Makefile file and change it if necessary.

Sometimes we use old BaseTools where Makefile uses FreezePython.exe not cxfreeze.bat. Please modify it as follows.

Makefile
# FREEZE=$(PYTHON_FREEZER_PATH)\FreezePython.exe
FREEZE = $(PYTHON_FREEZER_PATH)\cxfreeze.bat

Step 6. Set environment variables before building.

> set PYTHON_FREEZER_PATH=C:\Python27\Scripts
> set BASE_TOOLS_PATH=D:\EDKII\BaseTools
> set EDK_TOOLS_PATH=D:\EDKII\BaseTools

Step 7. Run "nmake all"

D:\EDKII\BaseTools\Source\Python> nmake all

You can find that the executables are generated in BaseTools Win32 path. For example,
D:\EDKII\BaseTools\Bin\Win32

The Design Pattern of UEFI Report Status Code

Observer is a kind of design pattern that is described in the book, Design Pattern - Elements of Reusable Object-Oriented Software, 1994. I recommend the book as a bible.

This blog page describes that the design patterns are not only used in the implementation with Object Oriented Program (e.g., C++, JAVA), but also used in the implementation (e.g., UEFI) with non OOP (e.g., C). Mostly EDKII are implemented in C language. We can find there are many Observer patterns in it. ReportStatusCode is an example.

If you have the book on your hand, you can compare the package diagram of the blog page with the structure diagram in Observer chapter of the book.

• The ReportStatusCodeRouterRuntimeDxe.inf  is corresponding to the Subject object.
• The FirmwarePerformanceDxe.inf and StatusCodeHandlerRuntimeDxe.inf are corresponding to the Observer objects.
• The Register () is corresponding to the Subject.Attach()
• The Unregister () is corresponding to the Subject.Detach()
• The ReportStatusCode() is corresponding to the Subject.Notify()

In OOP implementation, the Notify() calls all Observer->Update() to update all registered Observer objects.
Notify ()
{
  for o in Observers {
    o->Update()
  }
}

But for non-OOP implementation, ReportStatusCode(), how does it work? Below is the protocol definition of EFI_RSC_HANDLER_PROTOCOL.

typedef struct {
  EFI_RSC_HANDLER_REGISTER Register;
  EFI_RSC_HANDLER_UNREGISTER Unregister;
} EFI_RSC_HANDLER_PROTOCOL;

Let's focus on Register() in which we only be interested. For Unregister(), it seems that it is seldom used.

typedef
EFI_STATUS
(EFIAPI *EFI_RSC_HANDLER_REGISTER)(
  IN EFI_RSC_HANDLER_CALLBACK   Callback,
  IN EFI_TPL                    Tpl
  );

The most important parameter of the Register() is Callback. Below is the prototype of the Callback, EFI_RSC_HANDLER_CALLBACK.

typedef
EFI_STATUS
(EFIAPI *EFI_RSC_HANDLER_CALLBACK)(
  IN EFI_STATUS_CODE_TYPE   CodeType,
  IN EFI_STATUS_CODE_VALUE  Value,
  IN UINT32                 Instance,
  IN EFI_GUID               *CallerId,
  IN EFI_STATUS_CODE_DATA   *Data
  );

Below is the prototype of the EFI_STATUS_CODE_PROTOCOL.ReportStatusCode().

typedef
EFI_STATUS 
(EFIAPI *EFI_REPORT_STATUS_CODE) (
  IN EFI_STATUS_CODE_TYPE     Type,
  IN EFI_STATUS_CODE_VALUE    Value,
  IN UINT32                   Instance,
  IN EFI_GUID                 *CallerId  OPTIONAL,
  IN EFI_STATUS_CODE_DATA     *Data      OPTIONAL
  )

We found that the prototype of Register() and ReportStatusCode() are same. Therefore I guess that the implementation of the ReportStatusCode() should be as follows. Forgive me that I use python-style pseudo code to explain it.

ReportStatusCode (params):
  for CallBack in mCallBackList:
    CallBack (params)  

Is pseudo code really true? You can trace the EDKII source code by yourself to verify if my assumption is true. This page just describes that there are many design patterns in EDK and EDKII. Furthermore, the ReportStatusCodeRouterRuntimeDxe.inf should be a design pattern of Singleton. What is Singleton? Please read book or GOOGLE it.

How does UEFI Fault Tolerant Write protocol work on Variable reclaiming process?

The FTW (Fault Tolerant Write) protocol is called when the variable reclaiming happens. Why does Variable driver use FTW during reclaiming? How does FTW work?

We know that variables are stored in ROM. When a variable is updated by SetVariable() calling, Variable driver appends a new variable in the tail and marks the old variable as invalid. Why not to update variable directly? The reason is the hardware characters of flash part. In Write operation, the bit in flash part can be changed from 1 to 0, but cannot be changed from 0 to 1. Erase operation is to erase a block in flash part with FF value. That is why the Variable driver needs to reclaim when the variable store is about full. Reclaiming means deleting invalid variables and combined all valid variables together. The simple reclaiming process is:

1. Put all valid variables into memory.
2. Erase variable store in ROM.
3. Write all variables from memory to ROM.

There is a risk that if the platform is shut down unexpectedly in the step 3, the variables reclaiming are not completed to make platform crashed. To avoid the risk, the reclaiming process is updated as follows, I guess.

1. Put all valid variables into memory.
2. Set a flag to announce that the reclaiming start.
3. Write all valid variables from memory to backup region in ROM
4. Erase variable store in ROM.
5. Write all variables from memory to variable store in ROM.
6. Update the flag to indicate that the reclaiming is completed.

So if the platform is shut down unexpectedly in step 5 and the platform restarts, Don't worry. The platform will recover variables from backup region because it detects the reclaiming has not been completed. The recovery process is:

1. Detect the flag that the reclaiming has not be completed yet.
2. Restore variables from backup region to variable store.
3. Update the flag to indicate that the reclaiming is completed.

Again, if the platform is shut down unexpectedly in step 2, the recovery steps are run again when the platform starts.

How does FTW support this idea? Variable driver uses FTW->Write() to write large data to implement step 2 - step 6 in reclaiming process.

• The flag is updated in NV_FTW_WORKING.
• The backup region is in NV_FTW_SPARE.

We can use a tool to set different variables to trigger variable reclaiming, and use another tool to dump NV_FTW_WORKING and NV_FTW_SPARE. How do we find the two regions? There are many ways to find them.

Method 1 - Dump ROM and search the signature of NV_FTW_WORKING

EFI_FAULT_TOLERANT_WORKING_BLOCK_HEADER->Signature

#define EDKII_WORKING_BLOCK_SIGNATURE_GUID \ 

{0x9e58292b, 0x7c68, 0x497d, {0xa0, 0xce, 0x65, 0x0, 0xfd, 0x9f, 0x1b, 0x95} }

Method 2 - Find the values of NV_FTW_WORKING and NV_FTW_SPARE in AutoGenFlashMap.h

The AutoGenFlashMap.h is generated by EDKII build system. We can find the values that we are interested.

#define _PCD_VALUE_PcdFlashNvStorageVariableBase    0xFFF90000U
#define _PCD_VALUE_PcdFlashNvStorageVariableSize    0x0001D000U
#define _PCD_VALUE_PcdFlashNvStorageFtwWorkingBase  0xFFFAE000U
#define _PCD_VALUE_PcdFlashNvStorageFtwWorkingSize  0x00002000U
#define _PCD_VALUE_PcdFlashNvStorageFtwSpareBase    0xFFFB0000U
#define _PCD_VALUE_PcdFlashNvStorageFtwSpareSize    0x00020000U

From the definitions, we know that the offset of FtwWorkingBase (NV_FTW_WORKING) is  FFFAE000h - FFF90000 = 1E000h

Furthermore, we can use a tool to dump the content in UEFI Shell. For example,
ToolA.efi -read -id 3 -lba 1 offset e000 -size 70

The tool consumes Firmware Volume Block protocols to dump data from ROM. I will describe it in another page.

Because the page is too long, please refer the page to observe the detail of reclaiming process.
http://countchu.blogspot.tw/2014/10/observe-reclaiming-process-of-edkii.html

I hope this page and the referred page provide you enough background knowledge of FTW and Variable reclaiming process.