Running Arbitrary Workloads in a nano VM

This post will show you how you can run an arbitrary piece of code in your nano VM. This post assumes that you already have a nano VM running ( ). A full version of the program can also be found here ( ). here gitlab At this time we will write our application using assembly language and compile using nasm compiler ( ). here Our program is will be very simple - it will take two numbers in two registers and output a result via serial port. Following is our program written in assembly (Intel syntax): start , , , , global start: mov dx 0x3f8 add eax ebx add eax '0' out dx al hlt Firstly we load into register. This address is commonly used as serial port. Take a note that 0x3f8 is in 1MiB of memory. Next we add numbers in and registers (and store result in ). 0x3f8 dx eax ebx eax Following we add ascii value of zero to our result in register. Finally we write first byte (least significant byte, aka LSB) from the register (abbreviated as ) to the address stored in - serial port. eax eax al dx instruction will cause our VM to exit (this is know as ). VM exit will be handled by our host OS and then control will be returned to the VM. instruction will then cause another VM exit which will be again handled by our host VM which will terminate the VM. out VM exit hlt Before we can run the program we need to compile it. To do that we will use nasm compiler. Assuming the program is stored in `program.asm` you can run the following command: nasm program.asm -o program.bin Next we need to load the program into memory that is will be mapped into the VM. To do that we'll use two helper functions: and . get_file_size read_into_buffer get_file_size( fd) { (fstat(fd, &buf) < ) { ; } buf.st_size; } { temp[BUFFER_SIZE]; r = read(fd, &temp, BUFFER_SIZE); total_bytes_copied = ; (r != ) { (buf + total_bytes_copied, temp, r); total_bytes_copied += r; r = read(fd, &temp, BUFFER_SIZE); } total_bytes_copied; } # BUFFER_SIZE 32 define off_t int ; struct stat buf if 0 return -1 return int read_into_buffer ( fd, * buf) int char char int int 0 while 0 memcpy return uses fstat function to get some information about file including file size. get_file_size takes a file descriptor and a pointer to a chunk of memory into which the program will be placed. It then uses function to read the file and copy it into the buffer. read_into_buffer read 1. Using the above two functions we can now load a file that's passed as an argument to our program as follows: { payload_fd = open(argv[ ], O_RDONLY); fsize = get_file_size(payload_fd); *buf = (fsize); (read_into_buffer(payload_fd, buf) != fsize) { ; } .... } int main ( argc, **argv) int char int 1 off_t void malloc if return -1 2. The next part of the puzzle is to copy contents of the buffer into memory area that will be used by VM as physical memory. To do that we use function: buf memcpy *mem = mmap( , , PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, , ); (mem, buf, fsize); .slot = , .guest_phys_addr = , .memory_size = , .userspace_addr = ( )mem, }; ioctl(vmfd, KVM_SET_USER_MEMORY_REGION, &region); void NULL 0x1000 -1 0 memcpy = { struct kvm_userspace_memory_region region 0 0x1000 0x1000 uint64_t 3. Compile the main program and run the workload in a VM: ./main program.bin # Run the following command as sudo Conclusion In this part you saw how to write a small program in assembly, compile it and run it in a VM. This is not as fancy as running an arbitrary C++/golang/rustlang/xyz program but it is something. In order for us to be able to run more complex programs we need to work a bit more on configuring our VM and VCPU. Stay tuned for the next post when I will outline next steps that we will take in order to get us closer to running more complex programs produced by compilers such as C, C++ or even rust. Full program is available at: https://gitlab.com/mvuksano/kvm-playground/-/blob/master/02b-load-payload/main.c