Skip to main content

Lab 12 - Application Interaction

Task: Time server​

Navigate to chapters/app-interact/time-server/drills/tasks/time-server/support. Try to figure out the protocol used by the server and the client. You can do this by reading the source code, corroborated with information obtained at runtime.

Run the server again (the version in C), but instead of running the client, let's run netcat and pipe the output to hexdump:

nc -d 127.0.0.1 2000 | hexdump -C

Quiz

Quiz

If you're having difficulties solving this exercise, go through this reading material.

Task: Password cracker​

Navigate to chapters/app-interact/password-cracker/drills/tasks/password-cracker/support. Creating 26 processes is not very realistic, since it's unlikely that a usual machine has that many cores.

Modify the program so that it only creates 4 workers. Each worker will receive 2 characters instead of one, defining an interval to search. For example, the first worker will receive a and f, meaning it will brute-force passwords starting with a, b, c, d, e, or f, the second g - l, and so on.

Check that the worker() function is indeed called from different worker processes. One simple way to do this is to print out the current process ID at the beginning of the function. To get the current process ID, use the getpid() function from the os module.

If you're having difficulties solving this exercise, go through this reading material.

Task: D-Bus - Battery level​

Navigate to chapters/app-interact/dbus/drills/tasks/dbus/support. Use D-Bus to find out the computer's battery level. There is the org.freedesktop.UPower interface on the system bus that can provide this information.

The method you need to call is org.freedesktop.DBus.Properties.Get from the /org/freedesktop/UPower/devices/DisplayDevice object.

This method needs 2 arguments: an interface name and a property name.

Those should be org.freedesktop.UPower.Device and Percentage respectively.

Then input all of the above into a gdbus call, which, if everything is correct, should output the battery percentage level as a number between 0 and 100.

Note: if you are running on a desktop computer or inside a virtual machine, you will get the value 0.0, because those systems don't have a battery.

If you're having difficulties solving this exercise, go through this reading material.

The X Window System​

Unix-like systems that support a Graphical User Interface usually do this through the X Window System. This system is implemented with a client-server model: the X Server is the component that controls the screen, keyboard, mouse, and other parts related to the GUI, while the X clients are the applications that want to use the graphical interface (like, for example, an internet browser).

The clients and the server communicate using a standardized protocol, and the system does not necessarily require the client and server to be on the same machine. Although not so common nowadays, the X client can run on a different machine than the server, with the communication happening over the network. But in the more usual case, when both the client and the server are on the same machine, modern implementations of the X Window System use a faster communication channel, like a Unix socket.

X Client and Server on the Same Machine​

Let's investigate the case when both the X client and X server run on the same machine. First we'll take a look at the Unix sockets that are in listening mode.

student@os:~$ sudo netstat -xnlp | grep X11
unix  2      [ ACC ]     STREAM     LISTENING     29120    3472/Xorg            @/tmp/.X11-unix/X0
unix  2      [ ACC ]     STREAM     LISTENING     29121    3472/Xorg            /tmp/.X11-unix/X0

We observe the Xorg process (the X server) listening on a Unix socket with the path /tmp/.X11-unix/X0.

Now let's run an X client (that is, a GUI application) and check that it will indeed connect to this Unix socket. A very simple example is the xeyes application:

student@os:~$ strace -e trace=socket,connect xeyes
socket(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC, 0) = 3
connect(3, {sa_family=AF_UNIX, sun_path=@"/tmp/.X11-unix/X0"}, 20) = 0

As expected, the application creates a Unix socket, then connects to the path @"/tmp/.X11-unix/X0".

Furthermore, let's confirm that there is actual communication taking place between xeyes and the X server. We'll run xeyes again, and then we'll keep moving the mouse cursor around. When the mouse is moved, the following events are taking place:

  • The X server captures the mouse movements (since the server is the one that controls the mouse)
  • The X server will pass these "mouse moved" events to the clients (including xeyes)
  • The client (xeyes) uses these events to update its window (changing the position of the pupils inside the eyes)

So, if we run xeyes under strace, we expect to see some communication on the Unix socket that is created at the beginning:

strace -e 'trace=!poll' -e trace='socket,connect,recvmsg' xeyes |& grep -v '\-1 EAGAIN'

strace-xeyes

Oneko​

An alternative to xeyes which allows us to observe Unix sockets is oneko. Going through the same steps, we see that the application also create a Unix socket, then connects to the path @"/tmp/.X11-unix/X0".

student@os:~$ strace -e trace=socket,connect oneko
socket(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC, 0) = 3
connect(3, {sa_family=AF_UNIX, sun_path=@"/tmp/.X11-unix/X1"}, 20) = 0
--- SIGALRM {si_signo=SIGALRM, si_code=SI_KERNEL} ---

When running oneko, what differs from xeyes is the SIGALRM signal. This means that oneko uses a timer, which is periodically set, and then it expires only to be reset again. The purpose of this timer is to slow down the cat.

Verifying the communication between the X server and oneko is easy. We see that the cat follows our mouse cursor, behaving similarly to xeyes. After running oneko under strace, we see the communication uses the UNIX socket created at the beginning:

strace -e 'trace=!poll' -e trace='socket,connect,recvmsg' oneko |& grep -v '\-1 EAGAIN'

Quiz

Time Server​

Check out the code in chapters/app-interact/time-server/support/server.c and chapters/app-interact/time-server/support/client.c.

This is a simple program consisting of a server and a client. The server uses a tcp socket to wait for connections. Once a client has connected, the server will send the current time to it. The client will then print the received time to the console.

Let's build and run this example:

student@os:~/.../support$ make
student@os:~/.../support$ ./server

Then, in another terminal:

student@os:~/.../support$ ./client 127.0.0.1 2000
The time is Thu Sep  1 11:48:03 2022

Python Version​

In chapters/app-interact/time-server/support/python we have the equivalent python implementation for both the server and client:

student@os:~/.../support/python$ python3 server.py
student@os:~/.../support/python$ python3 client.py 127.0.0.1 2000
The time is Thu Sep  1 11:58:01 2022

Password Cracker​

In this example, we will solve the following problem: given the sha512 hash of a password, we want to obtain the password that generated the hash.

Since a hash function is not reversible, one way to solve this problem is by brute-force: generate all possible word combinations, compute the hash for each word, and compare it with our desired hash value. This is not feasible for long passwords, so for our example we will consider only passwords containing lowercase letters and having the length of 4.

In order to speed up the entire process, we want to parallelize the solution. Instead of one process checking all combinations, we'll split the work among multiple processes or threads.

Multiprocess Version​

The code for this version is in chapters/app-interact/password-cracker/support/password-cracker-multiprocess.c.

The idea is the following: we create 26 worker processes, where each process will consider passwords that start with one particular letter (the first process will brute-force passwords starting with a, the second with b, and so on).

Since we are using processes, which are naturally isolated, we need a method of communication. The main process should be able to send data to the workers and read back results from them. For this purpose we will use pipes: a pair of 2 pipes between the main process and each worker, one pipe for each direction of communication.

In summary, the flow will look like this:

  • main process

    • create worker processes, along with 2 pipes for each worker (one pipe for requests, one for results)

    • send the 'a' character to the first process request pipe, 'b' to the second, etc.

    • read the results from each result pipe

  • worker process

    • read one character from the request pipe

    • generate all words of length 4 that begin with that character

    • for each generated word, compute the sha512 hash and compare it with the desired hash

    • if there is a match, write it to the result pipe

Let's build and run the program:

student@os:~/.../support/password-cracker$ make
gcc -Wall -o password-cracker-multiprocess password-cracker-multiprocess.c -lcrypto
gcc -Wall -Wno-int-to-pointer-cast -Wno-pointer-to-int-cast -o password-cracker-multithread password-cracker-multithread.c -lcrypto -lpthread

student@os:~/.../support/password-cracker$ ./password-cracker-multiprocess
worker 7 found haxx

Multithreaded Version​

Check out the code in chapters/app-interact/password-cracker/support/password-cracker-multithread.c.

The core idea of the program is the same, but now we're using threads instead of processes.

This makes the communication easier: we'll use the thread function argument to send the first character of the password to each thread. As for the result, each thread will return it as the return value of the thread function.

student@os:~/.../support$ ./password-cracker-multithread
worker 7 found haxx

Multiprocess Version in Python (1)​

Code in chapters/app-interact/password-cracker/support/python/password-cracker-multiprocess-1.py.

This is the Python equivalent of the previous multiprocess version. The program structure is the same, but Python has a few nice features that make our life easier:

  • there is a Process object that takes a function argument and spawns a new process that begins execution from that function. No need to call fork manually.

  • the Pipe object in Python is already bidirectional, unlike the OS pipes, which are unidirectional. So we don't need to create 2 pipes for each direction.

  • we don't have to write the code that generates all the password combinations, itertools.product will do it for us

student@os:~/.../support$ python3 python/password-cracker-multiprocess-1.py
worker 7 found haxx

Multiprocess Version in Python (2)​

Code in chapters/app-interact/password-cracker/support/python/password-cracker-multiprocess-2.py.

In this case, the code looks different than in the previous examples. Now we are taking advantage of some Python constructs, namely process pools, which are a collection of worker processes.

A Pool object has, among others, a function called map. map takes a function, together with an array of values, and applies this function on each value from the array. At first glance, it might look like the usual map function, but with the key difference that the function application is done by the processes from the pool.

In other words, the work is distributed to the worker processes from the pool, and all the communication that we had to handle in the previous examples is done behind the scenes, greatly simplifying the code.

student@os:~/.../support$ python3 python/password-cracker-multiprocess-2.py
worker 7 found haxx

Multithreaded Version in Python​

Code in chapters/app-interact/password-cracker/support/python/password-cracker-multithread.py.

The Python equivalent of the previous multithreaded version.

student@os:~/.../support$ python3 python/password-cracker-multithread.py
worker 7 found haxx

This example is given only to provide an idea of how a multithreaded program is written. Remember that CPU-bound threads in python don't actually run in parallel, due to the Global Interpreter Lock.

Guide: Containers vs VMs​

Containers are a lightweight virtualization technology that allows multiple isolated user-space instances to run on a single host operating system. They are often compared to chroot because they both provide isolated environments for running applications.

Cgroups limit, account for, and isolate the resource usage (CPU, memory, disk I/O, network, etc.) of a collection of processes. They can be used to enforce resource limits, prioritization, accounting, and control. Namespaces isolate processes from each other by creating independent views of system resources. There are different types of namespaces, such as user, PID, network, mount, IPC, and UTS. You can read more about them here, here and a particularly good read about namespaces can be found here

Quiz

However, containers take this isolation a step further by using kernel features such as namespaces and cgroups to provide a more complete and secure isolation of resources.

Virtual machines, on the other hand, are a heavier form of virtualization that involves running a complete guest operating system on top of a host operating system using a hypervisor. This allows multiple guest operating systems to run on a single physical machine, each with its own set of virtualized hardware resources.

VMs vs Containers

One key difference between containers and VMs is the level of abstraction. Containers virtualize the operating system, allowing multiple containers to share the same kernel while providing the illusion of running on separate machines. VMs virtualize the hardware, allowing multiple guest operating systems to run on the same physical machine while providing the illusion of running on separate physical hardware.

Another difference is the resource overhead. Containers are generally more lightweight than VMs because they share the host kernel and do not require a separate guest operating system to be installed. This means that containers can start up faster and use less memory than VMs.

Quiz

Containers​

Our app will make use of docker containers. A container is an OS-level virtualization method in which a group of userspace processes are isolated from the rest of the system.

Take for example a database server. Instead of running it directly on the host system, we'll run it in its own container. This way, the server process will be isolated from other processes on the system. It will also have its own filesystem.

Besides isolation, containers are also useful for portability. Since a container comes with its own filesystem image, we can pack it together will all the dependencies, so that the app will run correctly no matter what packages are installed on the host system.

Finally, since our application will consist of more than 1 container, we'll also use docker-compose, which is a tool that helps us with running multi-container applications.

OS Cloud​

In this section, we are going to build a "toy cloud" called OS Cloud. Similar to a real cloud (like aws), OS Cloud will allow us to create and manage virtual machines, through an http API.

Prerequisites​

Make sure the following packages are installed:

sudo apt-get -y update; sudo apt-get -y install docker-compose jq

Also, make sure your user can run docker commands. If not, maybe you need to add it to the docker group:

sudo usermod -aG docker student

Then, after re-login:

student@os:~$ docker ps
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES

If you are running inside a virtual machine, you need to enable nested virtualization. Example for vmware:

nested-virt-vmware

For virtualbox:

nested-virt-vbox

If the button is greyed out, try from the command line:

student@os:~$ VBoxManage  list vms
"USO 2022-2023" {042a5725-bfb7-4a46-9743-1164d3acac23}

student@os:~$ VBoxManage modifyvm {042a5725-bfb7-4a46-9743-1164d3acac23} --nested-hw-virt on

Initial Liftoff​

First, we need to do some initial setup:

student@os:~/.../support/os-cloud$ ./initial_setup.sh

Then go to support/os-cloud and run:

student@os:~/.../support/os-cloud$ ./setup_db.sh
Setting up db
Starting db server
Waiting for db server to start
...
Stopping db server
Restarting db server
Waiting for db server to start
Creating tables
Stopping db server

student@os:~/.../support/os-cloud$ docker-compose up --build

Now the http API will listen on port localhost:5000. Let's try:

student@os:~/.../support/os-cloud$ curl localhost:5000
Welcome to OS Cloud!

Let's check the running virtual machines:

student@os:~/.../support/os-cloud$ curl localhost:5000/vm_list
[]

We got an empty list, since there are no virtual machines yet. Let's create one (the command will take about 1 minute to complete):

student@os:~/.../support/os-cloud$ curl -H "Content-Type: application/json" \
 -d '{ "name": "my_vm", "image": "ubuntu_22.04", "network": "default", "mem_size": "2G", "disk_size": "10G"}' \
 localhost:5000/vm_create
{"id":1,"status":"ok"}

Quiz

Check the virtual machine list again:

student@os:~/.../support/os-cloud$ curl localhost:5000/vm_list
[{"id":1,"name":"my_vm"}]

We can also use the jq tool to pretty print the json outputs:

student@os:~/.../support/os-cloud$ curl -s localhost:5000/vm_list | jq .
[
  {
    "id": 1,
    "name": "my_vm"
  }
]

We see our newly created virtual machine. Let's get some information about it:

student@os:~/.../support/os-cloud$ curl -s -H "Content-Type: application/json" -d '{ "id": 1 }' localhost:5000/vm_info | jq .
{
  "disk_size": 10737418240,
  "id": 1,
  "ip": "192.168.0.2",
  "mem_size": 2147483648,
  "name": "my_vm",
  "network": "default",
  "os": "ubuntu_22.04",
  "state": "RUNNING"
}

We recognize some parameters that we specified at creation time, like mem_size and disk_size. Also, the IP address 192.168.0.2 has been allocated for our machine.

Virtual Machine Creation​

Take a look at the vm_create function in support/os-cloud/os-cloud/vm.py. The steps undertaken are roughly:

  1. some initial allocations: the virtual machine IP address, network interface, qemu ports, etc

  2. the virtual machine disk is created, based on the template specified by the user (like ubuntu_22.04)

  3. the virtual machine is started with this new disk, in order to do some more customizations (the ubuntu_22_04_vm_prepare function)

  4. the virtual machine is restarted again with the final disk in place

Disk Creation​

All the disk templates are in chapters/app-interact/os-cloud/support/disk-templates. This directory will be mounted in /disk-templates inside the container.

The first step of disk creation is to create a qcow2 disk file based on the template specified by the user (step 2 from the explanation above).

This is done in the create_disk_from_template function in chapters/app-interact/os-cloud/support/os-cloud/disk.py. The function will first create a disk object in the database, then it will call 2 shell scripts: create_disk_from_template.sh and setup_root_password.sh.

The second step is to start the virtual machine with this disk and do some customizations (step 3 from above).

This is done in the ubuntu_22_04_vm_prepare function in chapters/app-interact/os-cloud/support/os-cloud/vm.py. The code will connect to the vm's qemu serial console using pexpect. Then it will use a series of expect_exact + sendline pairs to interact with the virtual machine, as if those commands were typed in the command-line.

OS-Cloud: More Disk Customization​

You might have probably noticed that there are 2 types of disk customizations:

  • One type is for things that can be done without running the virtual machine. If we only want to modify some files inside the disk filesystem, we can do so by mounting the disk. This is done, for example, in the disk-templates/ubuntu_22.04/setup_root_password.sh script. There we use nbd_connect_qcow2 + mount to mount the disk, then we modify the /etc/shadow file to change the root password.

  • The second case is for operations that must be done with the virtual machine running. These are handled in the ubuntu_22_04_vm_prepare function: the virtual machine is first started (start_qemu_for_vm), then pexpect is used to interact with the virtual machine via the qemu serial console. Here we do things like running ssh-keygen - a binary that is part of the disk filesystem, which depends on other parts of the operating system from the disk to be running. Note that in ubuntu_22_04_vm_prepare, for convenience, we also do some customizations that fall into the first category (like modifying /etc/ssh/sshd_config).

Copy Additional Files to the Newly Created Disk​

This is a customization from the first category. In disk-templates/ubuntu_22.04/files there is a file called 99-os-cloud-welcome (a script that prints a greeting message). We want to copy this file to /etc/update-motd.d in our newly created disk, so that it will run whenever a user logs in.

To do this, you will create a script called copy_files.sh in disk-templates/ubuntu_22.04. This script will receive a path to a qcow2 disk file as an argument, it will mount the disk, and then copy the file to the necessary location. Then, in the create_disk_from_template function in disk.py you will call this script, similar with how the other scripts are called.

You can use disk-templates/ubuntu_22.04/setup_root_password.sh as an example.

SSH Key Setup​

We want to be able to log into the virtual machine using an ssh key, instead of the password 123456. Notice that the vm_create API also accepts an ssh_key parameter. Here, the user can provide an ssh public key, which the system will install in /root/.ssh/authorized_keys in the newly created virtual machine.

Your task is to implement this feature, as a customization from the second category (that is, implemented in the ubuntu_22_04_vm_prepare function). The key will be accessible to the function as the ssh_pub_key parameter. Then it's only a matter of writing the key to the appropriate place, using a command like echo key > /root/.ssh/authorized_keys. Note that the /root/.ssh directory might not exist, so you need to create it as well.

After the feature is complete, you can test it using the keys in the support/os-cloud/keys directory. This directory contains a pair of public-private keys. The directory will also be mounted inside the os-cloud container in /keys.

You will create another virtual machine, passing the public key to vm_create:

student@os:~/.../support/os-cloud$ curl -H "Content-Type: application/json" \
 -d '{ "name": "my_vm2", "image": "ubuntu_22.04", "network": "default", "mem_size": "2G", "disk_size": "10G", "ssh_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC8CDHgeE4NIIIih3wSz58GDkfPLUk2m9gmbZB1f6o8Lzawzb3HVFpslAUWK0f/Ymw9cloInpMo50gWMYFSyJ7ZrOWWak54BedpHDkFAxxy+JCE9b+pkKsrAT7wiir7gn2LHlhj55FLZkC9PpM9cBcrMfzlcP9Bf+2cnpDdINybSLmOUmrI23ANteM4lEVaa2yEbCaJk6dFB8+atz5zPjvVI0Hd+kJK7yJ0xV6Zc2ADle7TKW3dyiXOE9qFKe9933Rj7ocqNXCAO1cxUoJCVuVS7lh+1pSSPXLWLTOhVp/XiLGWVP6KRYmmn710MWKm9Kj1tPiGUphUraL20SJiRT6/ os-cloud-user"}' \
 localhost:5000/vm_create
{"id":2,"status":"ok"}

Obtain the IP address that was allocated to the new vm:

student@os:~/.../support/os-cloud$ curl -s -H "Content-Type: application/json" -d '{ "id": 2 }' localhost:5000/vm_info | jq .
{
  "disk_size": 10737418240,
  "id": 2,
  "ip": "192.168.0.3",
  "mem_size": 2147483648,
  "name": "my_vm2",
  "network": "default",
  "os": "ubuntu_22.04",
  "state": "RUNNING"
}

Then go inside the os-cloud container and ssh to the vm using the private key in /keys. It should work without prompting for the password:

student@os:~/.../support/os-cloud$ docker-compose exec os-cloud bash
root@ac93d3d6cab2:/app# ssh -i /keys/ssh_key root@192.168.0.3
Welcome to Ubuntu 22.04.1 LTS (GNU/Linux 5.15.0-56-generic x86_64)
[...]
Powered by OS Cloud
Last login: Mon Jan  2 19:34:53 2023 from 192.168.0.1
root@ubuntu:~#

OS-Cloud: Internet Access​

Notice that our virtual machines don't have Internet access:

Powered by OS Cloud
Last login: Mon Jan  2 19:52:47 UTC 2023 on ttyS0
root@ubuntu:~# curl google.com
curl: (6) Could not resolve host: google.com

In this task, we want to fix this problem. To do this, we must first understand how the networking for the virtual machines is done.

First, there is the concept of a network, which you saw in the previous section. There is a network called default, with the address of 192.168.0.0/24. All virtual machines are part of this network, that's why they were allocated ip addresses like 192.168.0.2.

Let's go inside the os-cloud container and take a look at the network interfaces:

$ docker-compose exec os-cloud bash
root@8333e5cefb0d:/app# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 8a:68:b7:5b:6b:45 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.1/16 scope global br0
       valid_lft forever preferred_lft forever
3: tap0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP group default qlen 1000
    link/ether 8a:68:b7:5b:6b:45 brd ff:ff:ff:ff:ff:ff
4: tap1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP group default qlen 1000
    link/ether fa:f8:7f:83:50:8f brd ff:ff:ff:ff:ff:ff
77: eth0@if78: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:ac:16:00:03 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 172.22.0.3/16 brd 172.22.255.255 scope global eth0
       valid_lft forever preferred_lft forever

root@8333e5cefb0d:/app# ps -ef | grep qemu
root          19       8 29 09:15 ?        00:01:26 qemu-system-x86_64 -m 2048 -hda /vm-disks/1/disk.qcow2 -net nic,macaddr=52:54:00:12:34:00 -net tap,ifname=tap0,script=no -monitor telnet::10001,server,nowait -serial telnet::10002,server,nowait -nographic -enable-kvm
root          29       8 28 09:15 ?        00:01:24 qemu-system-x86_64 -m 2048 -hda /vm-disks/2/disk.qcow2 -net nic,macaddr=52:54:00:12:34:01 -net tap,ifname=tap1,script=no -monitor telnet::10003,server,nowait -serial telnet::10004,server,nowait -nographic -enable-kvm

Here we have 2 virtual machines running. Each virtual machine uses a tap interface (the -net tap,ifname=tap0,script=no parameter for qemu). This means that the ens0 interface inside the virtual machine corresponds to the tap0 interface outside the virtual machine. All the tap interfaces are bridged together into the br0 bridge, which has the ip address 192.168.0.1. Also, each virtual machine has the default gateway configured to be 192.168.0.1.

In summary, it looks something like this:

os-cloud

All the traffic coming from the virtual machines passes through the br0 interface. So, in order to make the Internet work, all we have to do is a simple NAT, with a command like:

root@8333e5cefb0d:/app# iptables -t nat -A POSTROUTING -s 192.168.0.0/24 -j MASQUERADE

Now, the virtual machines should have Internet access:

root@8333e5cefb0d:/app# ssh root@192.168.0.2
[...]
root@ubuntu:~# curl google.com
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>

Now your task is to run the iptables command above automatically when the system starts, so that it's not necessary to run it manually like we did in the above example.

A good place to do this is in the create_one_network function in network.py. There you can add another subprocess.run call to run iptables. The 192.168.0.0/24 value should not be hardcoded, but you can take it from the ip_with_prefixlen member of the Net object.

Task: Create a New Disk by Hand​

Navigate to chapters/app-interact/os-cloud/drills/tasks/os-cloud/support. Let's replicate the above-mentioned steps and create a new disk ourselves.

First, we have to call the 2 scripts from the create_disk_from_template function:

student@os:~/.../support$ ./disk-templates/ubuntu_22.04/create_disk_from_template.sh ./disk-templates/ubuntu_22.04/ubuntu_22.04.qcow2 my-disk.qcow2 10737418240
Image resized.

student@os:~/.../support$ ls -lh my-disk.qcow2
-rw-r--r-- 1 student student 619M Nov 20 15:41 my-disk.qcow2

student@os:~/.../support$ sudo ./disk-templates/ubuntu_22.04/setup_root_password.sh my-disk.qcow2 123456

Now we can start a qemu instance using this disk:

student@os:~/.../support$ qemu-system-x86_64 -enable-kvm -m 2G -hda my-disk.qcow2 -nographic
...
Ubuntu 22.04 LTS ubuntu ttyS0

ubuntu login: root
Password:
...
root@ubuntu:~#

Here we can further run customization commands, like the ones in the ubuntu_22_04_vm_prepare function, or any other things that we want.

When we're done, we run the halt command:

root@ubuntu:~# halt
root@ubuntu:~#          Stopping Session 1 of User root...
[  OK  ] Removed slice /system/modprobe.
[  OK  ] Stopped target Graphical Interface.
...
         Starting System Halt...
[   86.431398] reboot: System halted

When the System halted message is printed, press CTRL+A X to exit qemu (that is, press CTRL+A, release CTRL and A, press X).

Task: Implement vm_stop​

The vm_stop command will stop a particular virtual machine, meaning it will stop the qemu process for that vm. The implementation starts in api_vm_stop in app.py, which is the function that handles the http request for the stop operation. Here you need to do the following:

  • extract the virtual machine id from the request

  • use the vm.vm_get function to convert this ID into a VM structure

  • call vm.vm_stop and pass the VM object to it

In vm.vm_stop:

  • call stop_qemu_for_vm

  • change the vm pid in the database to -1

  • change the vm state in the database to VM_STATE_STOPPED

After modifying the code, you should run docker-compose up --build again. Also, if your database became inconsistent, you can clean it up by re-running the setup_db.sh script. Then delete all vm disks with sudo rm -rf vm-disks/*.

With vm_stop implemented, the system should work like this:

student@os:~/.../support$ curl -s localhost:5000/vm_list | jq .
[
  {
    "id": 1,
    "name": "my_vm"
  }
]
student@os:~/.../support$ curl -H "Content-Type: application/json" -d '{ "id": 1}' localhost:5000/vm_scurl -s -H "Content-Type: application/json" -d '{ "id": 1 }' localhost:5000/vm_info | jq .
{
  "disk_size": 10737418240,
  "id": 1,
  "ip": "192.168.0.2",
  "mem_size": 2147483648,
  "name": "my_vm",
  "network": "default",
  "os": "ubuntu_22.04",
  "state": "RUNNING"
}

The vm is in the RUNNING state. Now let's stop it:

student@os:~/.../support$ curl -H "Content-Type: application/json" -d '{ "id": 1}' localhost:5000/vm_stop
{"status":"ok"}
student@os:~/.../support$ curl -s -H "Content-Type: application/json" -d '{ "id": 1 }' localhost:5000/vm_info | jq .
{
  "disk_size": 10737418240,
  "id": 1,
  "ip": "192.168.0.2",
  "mem_size": 2147483648,
  "name": "my_vm",
  "network": "default",
  "os": "ubuntu_22.04",
  "state": "STOPPED"
}

Now the state is STOPPED. Inside the container, the qemu process should be gone as well:

student@os:~/.../support$ docker-compose exec os-cloud bash
root@b0600eff8903:/app# ps -ef
UID          PID    PPID  C STIME TTY          TIME CMD
root           1       0  0 10:00 ?        00:00:00 /sbin/docker-init -- python3 -u app.py
root           7       1  0 10:00 ?        00:00:00 python3 -u app.py
root          33       0  0 10:00 pts/3    00:00:00 bash
root          41      33  0 10:00 pts/3    00:00:00 ps -ef

Finally, the vm can be started again using vm_start:

student@os:~/.../support$ curl -H "Content-Type: application/json" -d '{ "id": 1}' localhost:5000/vm_start
{"status":"ok"}

If you're having difficulties solving this exercise, go through this reading material.

More Implementation Details​

The application consists of 2 containers:

  • db, which runs a MySQL database

  • os-cloud, which runs the web application and the virtual machines

Let's check them. After running docker-compose up, in another terminal run docker-compose ps:

student@os:~/.../support/os-cloud$ docker-compose ps
       Name                      Command              State                    Ports
------------------------------------------------------------------------------------------------------
os-cloud_db_1         docker-entrypoint.sh mariadbd   Up      3306/tcp
os-cloud_os-cloud_1   python3 -u app.py               Up      0.0.0.0:5000->5000/tcp,:::5000->5000/tcp

Now let's move inside the os-cloud container:

student@os:~/.../support/os-cloud$ docker-compose exec os-cloud bash
root@89a986d2526e:/app#

Since the virtual machines run inside this container, we should expect to see the one that we created in the previous step.

root@89a986d2526e:/app# ps -ef | cat
UID          PID    PPID  C STIME TTY          TIME CMD
root           1       0  0 09:02 ?        00:00:00 /sbin/docker-init -- python3 -u app.py
root           7       1  0 09:02 ?        00:00:00 python3 -u app.py
root          12       7  6 09:02 ?        00:00:41 qemu-system-x86_64 -enable-kvm -m 2048 -hda /vm-disks/1/disk.qcow2 -net nic,macaddr=52:54:00:12:34:00 -net tap,ifname=tap0,script=no -monitor telnet::10001,server,nowait -serial telnet::10002,server,nowait -nographic
root          27       0  0 09:11 pts/3    00:00:00 bash
root          35      27  0 09:13 pts/3    00:00:00 ps -ef

Indeed, a qemu-system-x86_64 process is there. The vm should be accessible via ssh on the IP 192.168.0.2 with password 123456 (if you get connection refused here, you need to wait a bit more for the machine to boot):

root@adf6e0bf4e6e:/app# ssh root@192.168.0.2
The authenticity of host '192.168.0.2 (192.168.0.2)' can't be established.
ED25519 key fingerprint is SHA256:3Mfa1fB9y4knUDJWEmEOTz9dWOE7SVhnH/kCBJ15Y0E.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '192.168.0.2' (ED25519) to the list of known hosts.
root@192.168.0.2's password:
Welcome to Ubuntu 22.04 LTS (GNU/Linux 5.15.0-40-generic x86_64)

...

Last login: Thu Nov 17 07:49:55 2022
root@ubuntu:~#

The vm is also accessible on the serial console (notice the -serial telnet::10002,server,nowait argument to qemu). If we start a telnet connection on port 10002, qemu will show us the virtual machine's serial console (basically the output that we normally see when running a virtual machine in text mode)

root@adf6e0bf4e6e:/app# telnet localhost 10002
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

ubuntu login: root
Password:
Welcome to Ubuntu 22.04 LTS (GNU/Linux 5.15.0-40-generic x86_64)

...

Last login: Thu Nov 17 07:50:11 UTC 2022 from 192.168.0.1 on pts/0
root@ubuntu:~#

To exit the serial console, press CTRL+], then type quit:

root@ubuntu:~#
telnet> quit
Connection closed.
root@adf6e0bf4e6e:/app#

(Even) More Implementation Details​

The architecture of the system can be summarized in the following diagram:

os-cloud

The os-cloud container is the core of the entire system. It consists of a web application written in python using flask. This web application exposes a virtual machine API that the user can interact with (like vm_create).

So, when we're calling curl like in the example above:

curl -H "Content-Type: application/json" \
 -d '{ "name": "my_vm", "image": "ubuntu_22.04", "network": "default", "mem_size": "2G", "disk_size": "10G"}' \
 localhost:5000/vm_create

It will do an HTTP POST request (because of the -d parameter) to /vm_create. The request will be handled by the api_vm_create function in app.py (because of the @app.route("/vm_create", methods=["POST"]) line).

Inside this function, we also have access to the request payload (the string that comes after -d in our curl call). More specifically, request.json will parse this payload as a JSON object and hand it back to us as a python dictionary. In this dictionary we'll find the parameters for our request, like name, image, network, and so on.

The function will then take the actions required to create the virtual machine: create the disk, start qemu, interact with the database, etc. Finally, whatever is returned by the api_vm_create function will be received by the curl request as the HTTP response. Here we also return JSON objects, like {"id":1,"status":"ok"}.

There are 3 objects used by the system:

  • vm - the actual virtual machine

  • disk - holds information about virtual machine disks

  • network - holds information about a network

Each of these objects are stored in a table in the database.

Let's check the database contents (take the password from the setup_db.sh file):

student@os:~/.../support/os-cloud$ docker-compose exec db mysql -u os-cloud -p os-cloud
Enter password:
...
MariaDB [os-cloud]> select * from vm;
+----+-------+---------+------------+------------+-------------------+------------+----------+-------------------+------------------+-------+
| id | name  | disk_id | mem_size   | network_id | tap_interface_idx | ip         | qemu_pid | qemu_monitor_port | qemu_serial_port | state |
+----+-------+---------+------------+------------+-------------------+------------+----------+-------------------+------------------+-------+
|  1 | my_vm |       1 | 2147483648 |          1 |                 0 | 3232235522 |       18 |             10001 |            10002 |     0 |
+----+-------+---------+------------+------------+-------------------+------------+----------+-------------------+------------------+-------+
1 row in set (0.001 sec)

MariaDB [os-cloud]> select * from disk;
+----+-------------+---------------+
| id | size        | template_name |
+----+-------------+---------------+
|  1 | 10737418240 | ubuntu_22.04  |
+----+-------------+---------------+
1 row in set (0.000 sec)

MariaDB [os-cloud]> select * from network;
+----+---------+----------------------+------------+------------+
| id | name    | bridge_interface_idx | ip         | mask       |
+----+---------+----------------------+------------+------------+
|  1 | default |                    0 | 3232235520 | 4294901760 |
+----+---------+----------------------+------------+------------+
1 row in set (0.000 sec)

Note: in real life, DON'T store passwords in text files inside a repository.

Some observations:

  • There is a default network already created. That is why we specified "network": "default" in the vm creation parameters, and we see that the vm is assigned to this network (network_id is 1).

  • This network's ip address is 3232235520, which in hex is 0xC0A80000, that is, 192.168.0.0. The netmask is 0xFFFF0000, or /16. This explains why our vm received the ip address 192.168.0.2.

  • There is a disk with the size of 10GB, based on the ubuntu_22.04 template, exactly like we requested. This disk is assigned to our vm (disk_id is 1). The disk file will reside in support/os-cloud/vm-disks/1/disk.qcow2, or /vm-disks/1/disk.qcow2 inside the container.

D-Bus​

D-Bus is an Inter-Process Communication (IPC) mechanism that is commonly present on Linux. It is particularly used by various components of the desktop environment (like GNOME) to communicate between one another, although the system itself is general-purpose and can be used in any other situations.

As the name suggests, the communication model is that of a bus: processes connect to the bus, then exchange messages with other processes through the bus. The bus itself is implemented by the dbus-daemon, and there are in fact multiple buses: one system bus, accessible system-wide, and one or more session buses, each one corresponding to one user login session.

Every process that connects to D-Bus receives a unique connection name. This name can be something human-readable, like org.freedesktop.Notifications, or some generated ID, like :1.63. Once a process is connected, it can expose one or multiple objects. An object has a path-like name, consisting of strings separated by a slash character (for example, /org/freedesktop/Notifications). Each object contains one or more interfaces, which have the methods that can be called on that object.

Guide: D-Bus Inspection with D-Feet​

In order to better understand these concepts, we'll use a graphical tool (D-Feet) to inspect all the available D-Bus objects on our system.

Run D-Feet and select Session Bus from the top button:

dfeet-session-bus

On the left panel, we can see all the processes connected to D-Bus with their associated connection names. Scroll down and find org.freedesktop.Notifications. On the right side, expand /org/freedesktop/Notifications and then expand the org.freedesktop.Notifications interface. The window should look like this:

dfeet-notifications

Some observations:

  • The bus communication happens over a Unix socket, with the path /run/user/1000/bus.

  • org.freedesktop.Notifications on the left panel is the connection name.

  • The process that has connected with this name is /usr/bin/gjs /usr/share/gnome-shell/org.gnome.Shell.Notifications and has the pid of 4373.

  • This process exposes one object: /org/freedesktop/Notifications. Note that the object name is the same as the connection name, where the dots have been replaced with slashes. This is not a requirement, as the objects exposed by a process can have any name.

  • The object has 4 interfaces: org.freedesktop.DBus.Introspectable, org.freedesktop.DBus.Peer, org.freedesktop.DBus.Properties and org.freedesktop.Notifications. Note that the last one (org.freedesktop.Notifications) is the same as the connection name, but this again is just a coincidence, not a requirement.

  • The interface org.freedesktop.Notifications has some methods that can be called, such as Notify.

Guide: Calling D-Bus Methods​

The application behind org.freedesktop.Notifications is responsible with desktop notifications (the small bubbles of text that appear at the top of the screen when some event happens). When an application wants to send a notification it needs to connect to D-Bus and call the Notify method from the org.freedesktop.Notifications interface.

In this example, we want to call the Notify method ourselves. To do this, we must first understand the signature of this method:

Notify (String arg_0, UInt32 arg_1, String arg_2, String arg_3, String arg_4, Array of [String] arg_5, Dict of {String, Variant} arg_6, Int32 arg_7) ↦ (UInt32 arg_8)

This doesn't tell us much, but we can find more documentation here, since freedesktop is an open standard.

We'll set the arguments to the following (for our simple case, most of them will be unused):

  • app_name: ""

  • replaces_id: 0

  • app_icon: ""

  • summary: "This is the title"

  • body: "This is the content"

  • actions: []

  • hints: {}

  • expire_timeout: -1

Now the question is how to actually call the method. Normally, we would have to write an application that connects to D-Bus and executes the call. But for demonstrative purposes there are easier ways.

One way is directly from d-feet. If we double-click on the Notify method in the right-side pane of d-feet, a window will open that allows us to call the method with any arguments that we want:

dfeet-execute-dialog

Then we click the Execute button and the notification will appear:

dfeet-execute-

Another way is from the command-line. There's the gdbus tool that can do this:

Guide: Inspecting the Low-level Communication​

Let's run gdbus under strace to see what's happening behind the scenes. Run the script in support/dbus/send_notification_strace.sh:

strace: Process 61888 attached
[pid 61887] socket(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC, 0) = 5
[pid 61887] connect(5, {sa_family=AF_UNIX, sun_path="/run/user/1000/bus"}, 110) = 0
[pid 61887] sendmsg(5, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="\0", iov_len=1}], msg_iovlen=1,
msg_control=[{cmsg_len=28, cmsg_level=SOL_SOCKET, cmsg_type=SCM_CREDENTIALS, cmsg_data={pid=61887,
uid=1000, gid=1000}}],
msg_controllen=32, msg_flags=0}, MSG_NOSIGNAL) = 1
strace: Process 61889 attached

[...]

[pid 61889] sendmsg(5, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="l\1\0\1T\0\0\0\3\0\0\0\237\0\0\0\1
\1o\0\36\0\0\0/org/freedesktop/Notifications\0\0\2\1s\0\35\0\0\0org.freedesktop.Notifications\0\0\0\6\1s\0\35
\0\0\0org.freedesktop.Notifications\0\0\0\10\1g\0\rsusssasa{sv}i\0\0\0\0\0\0\3\1s\0\6\0\0\0Notify\0\0\0\0\0\0
\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\21\0\0\0This is the title\0\0\0\23\0\0\0This is the content\0\0\0\0\0\0\0\0\0
\0\0\0\0\377\377\377\377", iov_len=260}], msg_iovlen=1, msg_controllen=0,
 msg_flags=0}, MSG_NOSIGNAL) = 260
[pid 61889] recvmsg(5, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="l\2\1\1\4\0\0\0\312\0\0\0.\0\0\0", iov_len=16}],
msg_iovlen=1, msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, MSG_CMSG_CLOEXEC) = 16
[pid 61889] recvmsg(5, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="\6\1s\0\6\0\0\0:1.497\0\0\10\1g\0\1u\0\0\5\1u\0
\3\0\0\0\7\1s\0\5\0\0\0:1.49\0\0\0\36\0\0\0", iov_len=52}], msg_iovlen=1, msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, MSG_CMSG_CLOEXEC) = 52
(uint32 30,)
[pid 61889] +++ exited with 0 +++
[pid 61888] +++ exited with 0 +++
+++ exited with 0 +++

We see a Unix socket being created and a connection made to /run/user/1000/bus, as expected. Then a series of messages are exchanged on the socket, which are part of the D-Bus protocol. On a closer look, we can even identify some strings from our notification, like This is the title or This is the content:

[pid 61889] sendmsg(5, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="l\1\0\1T\0\0\0\3\0\0\0\237\0\0\0\1
\1o\0\36\0\0\0/org/freedesktop/Notifications\0\0\2\1s\0\35\0\0\0org.freedesktop.Notifications\0\0\0\6\1s\0\35
\0\0\0org.freedesktop.Notifications\0\0\0\10\1g\0\rsusssasa{sv}i\0\0\0\0\0\0\3\1s\0\6\0\0\0Notify\0\0\0\0\0\0
\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\21\0\0\0This is the title\0\0\0\23\0\0\0This is the content\0\0\0\0\0\0\0\0\0
\0\0\0\0\377\377\377\377", iov_len=260}], msg_iovlen=1, msg_controllen=0,

Guide: D-Bus usage in Python​

Use the dbus python bindings to get the computer's battery level using a python script. You can start from the documentation here. You need to read the sections Connecting to the Bus, Proxy objects, and Interfaces and methods.

There's also a skeleton you can use in chapters/app-interact/arena/support/dbus/get_battery_level.py.

In summary, your script will start by connecting to the System Bus. Then you'll use the get_object method to obtain a proxy object. On this proxy object, you can actually do the method call as explained here:

To call a method, call the method of the same name on the proxy object, passing in the interface name via the dbus_interface keyword argument

So, if you want to call the method this.is.an.interface.method with the arguments A and B you can do it like this:

result = proxy.method(A, B, dbus_interface = "this.is.an.interface")

Guide: Firefox​

Let's do the following experiment:

  • Open the Firefox browser

  • From a terminal run firefox www.google.com

firefox-url-open

Notice that the URL we passed in the command-line was opened in the existing Firefox window as a new tab. Even though we started a separate Firefox process, which should have created a separate new window, this didn't actually happen. Instead, the process that we started from the command-line exited immediately and the site was opened in the already running Firefox instance.

Without any precise knowledge about Firefox internals, we can guess that something like this happened:

  • The newly started Firefox process detected that another instance of Firefox is already running

  • The newly started Firefox process sent a message to the existing running process, requesting it to open a URL in a new tab

Since we're talking about message passing between 2 processes, there's a chance that maybe D-Bus was involved. Let's check: we'll use a tool called dbus-monitor that will print all messages passed through D-Bus.

student@os:~$ dbus-monitor

Then, in another terminal, we'll run firefox www.google.com again.

Going back to the dbus-monitor output, we find the following:

...
method call time=1655809062.813923 sender=:1.757 -> destination=org.mozilla.firefox.ZGVmYXVsdC1yZWxlYXNl serial=2 path=/org/mozilla/firefox/Remote; interface=org.mozilla.firefox; member=OpenURL
   array of bytes [
      02 00 00 00 1a 00 00 00 2f 00 00 00 2f 68 6f 6d 65 2f 61 64 72 69 61 6e
      73 00 2f 6f 70 74 2f 66 69 72 65 66 6f 78 2f 66 69 72 65 66 6f 78 00 77
      77 77 2e 67 6f 6f 67 6c 65 2e 63 6f 6d 00
   ]

There was a D-Bus call to org.mozilla.firefox.ZGVmYXVsdC1yZWxlYXNl, on the object /org/mozilla/firefox/Remote, method OpenURL from the org.mozilla.firefox interface. Indeed, we see that this object exists in d-feet as well:

dfeet-firefox

We can try to call the OpenURL method ourselves, directly from d-feet. The method has only one argument of the type Array of [Byte]. Although there's no documentation for it, we can use the same byte array that we saw in dbus-monitor:

   array of bytes [
      02 00 00 00 1a 00 00 00 2f 00 00 00 2f 68 6f 6d 65 2f 61 64 72 69 61 6e
      73 00 2f 6f 70 74 2f 66 69 72 65 66 6f 78 2f 66 69 72 65 66 6f 78 00 77
      77 77 2e 67 6f 6f 67 6c 65 2e 63 6f 6d 00
   ]

(Note that 77 77 77 2e 67 6f 6f 67 6c 65 2e 63 6f 6d at the end is the string www.google.com, so that's another confirmation that we're on the right track).

dfeet-url-open