Overview

XCENA SDK Emulator provides a comprehensive software environment for running and testing computation CXL memory, enabling developers to work without the need for dedicated hardware devices. This powerful emulator bridges the gap between concept and deployment, offering a flexible and efficient development platform.

  • Hardware Independence: Accurately emulates the computational capabilities of CXL memory, eliminating the need for physical hardware and reducing development costs.
  • Developer-Friendly Interface: Provides a seamless and intuitive interface that mirrors the behavior of real CXL memory devices, ensuring smooth integration and ease of use.
  • Accelerated Prototyping: Facilitates the design, testing, and validation of prototypes in a controlled environment, allowing developers to iterate faster before deploying to actual hardware.
  • Scalability: Supports a wide range of scenarios, from small-scale testing to complex, large-scale applications, making it suitable for diverse use cases.
graph TD
    subgraph QEMU
        CXLType3Device[CXL Type3 Device]
    end

    subgraph GuestOS[Guest OS]
        Applications
        RootFileSystem[Root File System]
        Kernel
    end

    DeviceSimulator[Device Simulator]
    
    GuestOS <--> QEMU
    QEMU <--> |CXL packet| DeviceSimulator

image

  1. QEMU
    In this modular architecture, QEMU plays a central role in the emulator by bridging the gap between the Guest OS and the Device Simulator. As a client of the socket interface, QEMU utilizes CXL packets to standardize communication with device simulator. The native backend of QEMU’s memory region is overridden with a software-defined CXL packet mechanism, where memory access operations are intercepted and processed as custom CXL packets.

  2. Device Simulator
    We provide a standalone device simulator that interfaces with QEMU using the previously mentioned CXL packets. Device simulator processes requests from QEMU, simulates device behavior through firmware emulation, and communicates via a socket interface.

Installation

Prerequisite

Ubuntu 22.04 environment is recommended.

sudo apt-get install pkg-config libglib2.0-dev mkosi libncurses-dev gawk flex bison openssl libssl-dev dkms libelf-dev libudev-dev libpci-dev libiberty-dev autoconf llvm

Get repository from GitHub

  1. Get repository 
     git clone git@github.com:metisx-dev/cxl_emulator.git --recurse-submodules
  2. Build all components 
     ./setup.sh

    For detailed setup configurations, check setup.conf.

    Note This repository is currently private and will be made public in future.

Get repository from docker

You can find emulator repository in /work/emulator in docker container.

Run Emulator

  1. Configure CXL device emulation
    Refer to sample configs in cfg/cxl_socket. The basic configuration method follows the same approach as the QEMU reference (socket= is mandatory).
    Example: single device emulation

     -object memory-backend-ram,id=vmem0,share=on,size=32G
 -object memory-backend-file,id=cxl-lsa0,share=on,mem-path=lsa.raw,size=256M
 -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1
 -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2
 -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,lsa=cxl-lsa0,id=cxl-vmem0,socket=/tmp/pxl_device_socket0_$USER
 -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=64G

    Note Current version only supports single device emulation.

  2. Run with config
    Execute run.sh with config file.
    --debug option increases verbosity, providing more in-depth information about CXL driver operations, which can be particularly useful for development and troubleshooting purposes.

     # use default QEMU config without debug option
 ./run.sh

     # explicit QEMU config with debug option
 ./run.sh --cxl cfg/cxl_socket/vmem_32g --debug

    QEMU will boot with all required kernel modules (cxl, dax, SDK driver) preloaded and ready to use.
    If you want to check whether the dma driver was probed properly, check the ctx of mx_engine_init in the kernel log.

     root@localhost:~# dmesg | grep mx_dma
 [    2.181566] mx_dma:mxdma_init: MXDMA driver is loaded
 [    2.242072] mx_dma:map_bars: HMBOX BAR at 0xfd200000 is mapped at 0xffffc90000600000, length=2097152
 [    2.242230] mx_dma:set_dma_addressing: use 64-bit DMA
 [    2.242300] mx_dma:create_mx_cdev: mx_dma0_data (243:0) is created
 [    2.246682] mx_dma:create_mx_cdev: mx_dma0_context (243:1) is created
 [    2.249291] mx_dma:create_mx_cdev: mx_dma0_sq (243:2) is created
 [    2.249881] mx_dma:create_mx_cdev: mx_dma0_cq (243:3) is created
 [    2.249952] mx_dma:mx_engine_init: submit: ctx_addr=0xffffc90000600180, data_addr=0xffffc90000705600, ctx=0x350ac00a11c1
 [    2.250027] mx_dma:mx_engine_init: complete: ctx_addr=0xffffc90000601180, data_addr=0xffffc90000606400, ctx=0x350c800011
 [    2.250043] mx_dma:mxdma_driver_probe: pci device is probed (vendor=0xbabe device=0xface)

Debug QEMU

  1. GDB by CLI
    Run with --gdb option. 
     ./run.sh <args> --gdb

    Send a SIGINT to interrupt GDB process.

     $ ps -ef|grep qemu
 yrsong     83248   83238  3 20:58 pts/17   00:00:00 gdb -x gdb.qemu
 $ kill -2 83248

    This will allow you to start debugging QEMU with GDB.

  2. GDB by VSCODE
    Will be updated.