Lab 4 - Investigate Memory

Task: Allocating and Deallocating Memory

Navigate to the chapters/data/process-memory/drills/tasks/alloc-size/support directory.

Use pmap to analyze the process address space for ALLOC_SIZE_KB initialized to 256. Notice the new memory areas and the difference between the use of mmap syscall and brk syscall.
Use valgrind on the resulting executable, and notice there are memory leaks. They are quite obvious due to the lack of proper freeing. Solve the leaks.
Use valgrind on different executables in the system (in /bin/, /usr/bin/) and see if they have memory leaks.

If you're having difficulties solving this exercise, go through this reading material.

Task: Memory Mapping

Navigate to the chapters/data/process-memory/drills/tasks/copy/ directory, run make skels and open the support/src directory. Here you will find these files:

read_write_copy.c in which you will implement copying with read / write syscalls
mmap_copy.c in which you will implement copying using mmap
benchmark_cp.sh script which runs the two executables mmap_copy and read_write_copy

Complete the implementations for the read_write_copy.c and mmap_copy.c files. The goal is to copy the in.dat file to out.dat using the read / write syscalls and mmap respectively. Check your work by running the checker.sh script in the support/tests/ directory.
Once you have a working implementation, run the benchmark_cp.sh script to compare the performance of the two approaches. The output should look like this:
```
student@os:~/.../drills/tasks/copy/support$ ./benchmark_cp.sh
Benchmarking mmap_copy on in.dat
time passed 54015 microseconds

Benchmarking read_write_copy on in.dat
time passed 42011 microseconds
```
Run the script a few more times. As you can see, there isn't much of a difference between the two approaches. Although we would have expected the use of multiple system calls to cause overhead, it's too little compared to the memory copying overhead.
Add a sleep() call to the mmap_copy.c file after the files were mapped. Rebuild the program and run it. On a different console, use pmap to view the two new memory regions that were added to the process, by mapping the in.dat and out.dat files.

If you're having difficulties solving this exercise, go through this reading material.

Task: Memory Regions

Enter the chapters/data/process-memory/drills/tasks/memory-areas/support directory. We investigate other programs.

The hello.c program prints out a message and then sleeps. Build it:

student@os:~/.../drills/tasks/memory-areas/support$ make

then run it (it will block):

student@os:~/.../drills/tasks/memory-areas/support$ ./hello
Hello, world!

In another terminal, use the command below to show the memory areas of the process:

student@os:~/.../drills/tasks/memory-areas/support$ pmap $(pidof hello)
8220:   ./hello
000055c0bef4b000      8K r-x-- hello
000055c0bf14c000      4K r---- hello
000055c0bf14d000      4K rw--- hello
000055c0bf454000    132K rw---   [ anon ]
00007f2a9e4a5000   1948K r-x-- libc-2.27.so
00007f2a9e68c000   2048K ----- libc-2.27.so
00007f2a9e88c000     16K r---- libc-2.27.so
00007f2a9e890000      8K rw--- libc-2.27.so
00007f2a9e892000     16K rw---   [ anon ]
00007f2a9e896000    164K r-x-- ld-2.27.so
00007f2a9ea8c000      8K rw---   [ anon ]
00007f2a9eabf000      4K r---- ld-2.27.so
00007f2a9eac0000      4K rw--- ld-2.27.so
00007f2a9eac1000      4K rw---   [ anon ]
00007ffee6471000    132K rw---   [ stack ]
00007ffee6596000     12K r----   [ anon ]
00007ffee6599000      4K r-x--   [ anon ]
ffffffffff600000      4K --x--   [ anon ]
 total             4520K

The output is similar, but with fewer dynamic libraries than bash, since they are not used by the program.

Make a program in another language of your choice that prints Hello, world! and sleeps and investigate it with pmap. Note that in the case of interpreted languages (Python, Lua, Perl, Ruby, PHP, JavaScript etc.) you have to investigate the interpreter process.

If you're having difficulties solving this exercise, go through this reading material.

Task: Modifying Memory Region Size

Navigate to the chapters/data/process-memory/drills/tasks/modify-areas/support directory.

Comment out different parts of the hello.c program to notice differences in only specific areas (text, data, bss, heap, stack).
Use a different argument (order) for the call to the alloc_stack() function. See how it affects the stack size during runtime (investigate with pmap).
Do a static build of hello.c and check the size of the memory areas, both statically and dynamically.
The extend_mem_area.py Python script allocates a new string at each step by merging the two previous versions. Start the program and investigate the resulting process at each allocation step. Notice which memory area is updated and explain why.

If you're having difficulties solving this exercise, go through this reading material.

Task: Memory Layout of Statically-Linked and Dynamically-Linked Executables

Let's investigate another static executable / process.
If not already installed, install the busybox-static package on your system. On Debian/Ubuntu systems, use:
```
student@os:~$ sudo apt install busybox-static
```
Start a process using:
```
student@os:~$ busybox sleep 1000
```
Investigate the process using pmap and the executable using size.

If you're having difficulties solving this exercise, go through this reading material.

Task: Page Mapper

Navigate to the chapters/data/process-memory/drills/tasks/page-mapper/support directory.

Your goal is to update the src/page_mapper.c source code file to reserve virtual pages in the address space of the current process. Use mmap() to reserve virtual pages. Use anonymous mapping (i.e. the MAP_ANONYMOUS) flag. Use any permissions required.

Inside the src/ directory, use make to do a quick check of the implementation. To test it, enter the tests/ directory and run:

make check

In case of a correct solution, you will get an output such as:

./run_all_tests.sh
test_res_1                       ........................ passed ...  25
test_res_2                       ........................ passed ...  25
test_res_10                      ........................ passed ...  25
test_res_10_res_20               ........................ passed ...  25

Total:                                                           100/100

If you're having difficulties solving this exercise, go through this reading material.

Task: Operator overloading

Navigate to the chapters/data/process-memory/drills/tasks/reference-counting/support directory. Analyze the operators.d file. A struct is defined that also implements 4 special functions: a constructor, a copy constructor, an assignment operator and a destructor. Each of these special functions may be called automatically by the compiler:

the constructor is called automatically whenever an object is initialized with a field of a type that corresponds to the constructor parameter type.
the copy constructor is called automatically when an object is initialized from an object of the same type.
the assignment operator is called automatically when an object is assigned an object of the same type.
the destructor is called automatically whenever an object goes out of scope.

Note: the difference between initialization and assignment is that the initialization occurs when an object is being declared and occurs a single time (Obj o1 = 1), whereas assignment is decoupled from the declaration site and may occur multiple times (provided that the variable is mutable).

Compile and run the program in operators.d. Notice how the different special functions are automatically called. Considering the definition of Obj from the file operators.d, answer the following.

Analyze the refcount_skel.d. A reference counted int array is implemented, however, some bits are missing. Run the code, try to understand what happens.

The constructor allocates memory for the array, whereas the destructor deallocates it. Compile and run the code. Notice how the array's memory is automatically managed.

Uncomment the following line in the main function (//test1()). Run the code. What happens? Why?
The reference counted array does not implement the copy constructor. Comment the version(none) annotation for the copy constructor and implement the logic so that the reference counted array is correct. When an object is initialized from another object, we need to appropriately set the fields and then increment the reference count. Once you have completed this exercise, make sure the output is correct and that the reference counted array is not freed too early.
Uncomment the following line in the main function (//test2()). Run the code. What happens? Why? Use GDB to find out.
The reference counted array does not implement the assignment operator. Comment the version(none) annotation for the assignment operator and implement the logic so that the reference counted array is correct. When an object is assigned to another object, we need to first decrement the count for the object that is being assigned to, then fill the fields similarly to the copy constructor case and lastly increment the count for the assigned object. After completing the exercise, make sure that the memory is properly managed.
Play with your reference counted array and create different scenarios to test its limits.

If you're having difficulties solving this exercise, go through this reading material.

Investigate Memory Actions

Memory actions generally mean:

memory access: read, write or execute
memory allocation
memory deallocation

By far, the most important actions are allocation and deallocation. Because, if not done right, these can get to memory loss and poor memory use.

Memory loss generally happens in the form of memory leaks.

`malloc()` in Musl

Each libc (or memory allocator such as jemalloc) uses their own implementation of malloc(), free() and other functions. Musl libc is a lightweight standard C library that provides compatible features with the more heavyweights GNU libc.

Take a look through implementation of malloc() and free() in Musl libc. See all three implementations for malloc():

the one in lite_malloc.c
the one in mallocng/malloc.c
the one in oldmalloc/malloc

You needn't spend too much time browsing the implementation of these functions, just having a broad understanding of how they work.

Guide: App Investigation: Deluge

Deluge is a Bittorrent client written in Python.

We want to locate places that allocate memory in Deluge (in Python). This generally means locating instantiation of classes.

Let's clone the source code:

student@os:~/.../app-investigation$ git clone https://github.com/deluge-torrent/deluge
Cloning into 'deluge'...
[...]

student@os:~/.../app-investigation$ cd deluge/

student@os:~/.../app-investigation/deluge$ ls
AUTHORS       deluge      docs             gen_web_gettext.py  MANIFEST.in       msgfmt.py  pyproject.toml  requirements-dev.txt    requirements.txt  setup.py  version.py
CHANGELOG.md  DEPENDS.md  generate_pot.py  LICENSE             minify_web_js.py  packaging  README.md       requirements-tests.txt  setup.cfg         tox.ini

And enter the deluge/core/ subdirectory:

student@os:~/.../app-investigation/deluge$ cd deluge/core/

student@os:~/.../deluge/deluge/core$ ls
alertmanager.py  core.py          daemon.py        filtermanager.py  pluginmanager.py       rpcserver.py       torrent.py
authmanager.py   daemon_entry.py  eventmanager.py  __init__.py       preferencesmanager.py  torrentmanager.py

Most files in the subdirectory have a class defined. We can search for instantiations of that class using grep:

student@os:~/.../deluge/deluge/core$ grep -rn 'Torrent('
torrentmanager.py:644:        torrent = Torrent(handle, options, state, filename, magne

student@os:~/.../deluge/deluge/core$ grep -rn 'TorrentManager('
core.py:139:        self.torrentmanager = TorrentManager()
torrentmanager.py:135:class TorrentManager(component.Component):

This gives us an overview of when memory is allocated in Deluge / Python.

Practice

Investigate the lines shown to contain instantiations of classes. Explore the source code and understand their placements in the source code.
Find out other classes and search for their instantiation in the source code.

Guide: App Investigation: Servo

Servo is a browser engine written in Rust that provides reusable components to implement web standards.

We do not clone the repository, since it's very large.

We find information about allocator used, by accessing the components/allocator/ in its source code. In Cargo.toml we see that it requires jemalloc for non-Windows implementations and the standard Windows API (called heapapi) for Windows:

[...]
[lib]
path = "lib.rs"

[target.'cfg(not(windows))'.dependencies]
jemalloc-sys = { version = "0.3.2" }

[target.'cfg(windows)'.dependencies]
winapi = { version = "0.3", features = ["heapapi"] }
ust: https://github.com/servo/servo

In lib.rs, in GlobalAlloc:alloc() we see it is using the mallocx custom function from jemalloc(). See the initialization of ffi.

See the use of the allocator in the Cargo.toml file in the net component. Search for the alloc string.

Practice

Look for uses of the allocator in other components of Servo.

Guide: Investigation: Allocator in the D Programming Language

Phobos is the standard library that comes with the D programming language compiler.

Let's clone the source code:

student@os:~/.../d-allocator$ git clone https://github.com/dlang/phobos
[...]

student@os:~/.../d-allocator$ cd phobos/

student@os:~/.../d-allocator/phobos$ ls
azure-pipelines.yml  changelog  CODEOWNERS  CONTRIBUTING.md  dub.sdl  etc  index.dd  LICENSE_1_0.txt  posix.mak  project.ddoc  README.md  std  test  unittest.d  win32.mak  win64.mak

And enter std/experimental/allocator/ to browse information about the allocator:

student@os:~/.../d-allocator/phobos$ cd std/experimental/allocator/

student@os:~/.../std/experimental/allocator$ ls
building_blocks  common.d  gc_allocator.d  mallocator.d  mmap_allocator.d  package.d  showcase.d  typed.d

We then do a search of the allocate( string to find instances of allocation calls:

student@os:~/.../std/experimental/allocator$ grep -r 'allocate('
[...]

We see that there are definitions of the function (as expected) as part of ...allocator files: mallocator.d, gc_allocator.d, mmap_allocator.d. Browse the functions and look for implementations of the allocate() function.

Practice

Do a similar search and then source code browsing for the deallocate() function.

Guide: App Investigation: Git

Git is among the most used source code management system, powering development infrastructures such as GitHub, GitLab and Bitbucket.

Navigate to the chapters/data/investigate-memory/guides/git directory.

Let's clone the repository. Note that it is about 200MB large:

student@os:~/.../git$ git clone https://github.com/git/git
[...]

student@os:~/.../git$ cd git/

We look of uses of malloc():

student@os:~/.../git/git$ grep -r 'malloc(' .

We see there are multiple calls to the xmalloc() function, which is likely a wrapper for malloc(). We search for the definition of xmalloc():

student@os:~/.../git/git$ grep -rn 'xmalloc(' . | grep -v ';'
./commit.c:188:                 graft = xmalloc(st_add(sizeof(*graft),
./add-interactive.c:157:        list->sorted.items = xmalloc(st_mult(sizeof(*list->sorted.items),
./Documentation/RelNotes/2.24.0.txt:91: * xmalloc() used to have a mechanism to ditch memory and address
./Documentation/RelNotes/2.24.0.txt:210:   xmalloc() wrapper, as the rest of the system, for consistency.
./Documentation/RelNotes/2.34.0.txt:230: * mmap() imitation used to call xmalloc() that dies upon malloc()
./Documentation/RelNotes/2.33.1.txt:44: * mmap() imitation used to call xmalloc() that dies upon malloc()
./diffcore-delta.c:56:  new_spanhash = xmalloc(st_add(sizeof(*orig),
./diffcore-delta.c:135: hash = xmalloc(st_add(sizeof(*hash),
./kwset.c:41:/* adapter for `xmalloc()`, which takes `size_t`, not `long` */
./builtin/fast-import.c:461:    b = xmalloc(sizeof(struct object_entry_pool)
./hashmap.h:311: * your structure was allocated with xmalloc(), you can just free(3) it,
./xdiff/xdiff.h:122:#define xdl_malloc(x) xmalloc(x)
./wrapper.c:45:static void *do_xmalloc(size_t size, int gentle)
./wrapper.c:70:void *xmalloc(size_t size)
./contrib/credential/wincred/git-credential-wincred.c:26:static void *xmalloc(size_t size)
Binary file ./.git/objects/pack/pack-c587b9f11a82bc4d49848d74132e60ea4dbeb177.pack matches
./git-compat-util.h:1046:# define xalloca(size)      (xmalloc(size))
./git-compat-util.h:1086:#define ALLOC_ARRAY(x, alloc) (x) = xmalloc(st_mult(sizeof(*(x)), (alloc)))
./read-cache.c:3768:    ieot = xmalloc(sizeof(struct index_entry_offset_table)

Line ./wrapper.c:70 is the one with the definition of the xmalloc() function. It makes a call of the do_xmalloc() function, that makes extra checks. Also, if the XMALLOC_POISON macro is defined, all the allocated data is overwritten with a "poison" value (0xA5). This is useful for early detection of memory-related issues, although, evidently, it adds overhead.

We can look for parts of the source code with the largest number of uses of xmalloc():

student@os:~/.../git/git$ grep -rc 'xmalloc(' . | grep -v ':0' | sort -n -t ':' -k 20
[...]
./compat/mingw.c:6
./submodule-config.c:6
./merge-recursive.c:7

We can look into the merge-recursive.c file for uses of the xmalloc() function.

Practice

Do the same actions as above for the mmap() and xmmap() function calls.

Note that these are not memory allocation calls, since a valid fd file argument is passed. These are file mapping calls, that we will talk more as part of the I/O chapter.

Guide: `jemalloc`

jemalloc is a featureful allocator that is intended to replace the standard allocator in the standard C library (libc). jemalloc provides replacements for the general malloc() and free() functions, and also provides a custom API targeted for performance tuning.

As documented, there are multiple ways to use jemalloc, the easiest of which is to use the LD_PRELOAD environment variable and preload the library and hook into malloc() and free() function calls.

Navigate to the chapters/data/investigate-memory/guides/jemalloc directory.

First install jemalloc on our system. On your typical Ubuntu / Debian-based system, use apt:

student@os:~/.../jemalloc$ sudo apt -y install libjemalloc-dev

Note that this installs the distribution package, not the latest one (that may provide more features).

With this in place, we can use jemalloc against our pre-built executables or system executables (such as ls, ps). We can test it against the executable files from support/memory-leak/:

student@os:~/.../jemalloc$ LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so ./memory_leak_malloc
Andrei Popescu is 22 years old and likes Linux
Ioana David is 23 years old and likes macOS

jemalloc can use the MALLOC_CONF environment variable for a diverse set of configurations. For example, by using stats_print:true we print out information regarding the use of the library functions:

student@os:~/.../jemalloc$ LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so MALLOC_CONF="stats_print:true" ./memory_leak_malloc
Andrei Popescu is 22 years old and likes Linux
Ioana David is 23 years old and likes macOS
___ Begin jemalloc statistics ___
Version: 3.6.0-11
Assertions disabled
Run-time option settings:
  opt.abort: false
  opt.lg_chunk: 22
  opt.dss: "secondary"
  opt.narenas: 32
  opt.lg_dirty_mult: 3
  opt.stats_print: true
  opt.junk: false
  opt.quarantine: 0
  opt.redzone: false
[...]
dirty pages: 26:0 active:dirty, 0 sweeps, 0 madvises, 0 purged
            allocated      nmalloc      ndalloc    nrequests
small:          72672          114            0            3
large:          32768            1            0            1
total:         105440          115            0            4
active:        106496
mapped:       4194304
[...]

jemalloc doesn't work against system executables using preloading, likely because of security options disabling the use of the library:

student@os:~/.../jemalloc$ LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so MALLOC_CONF="stats_print:true" /bin/ls
Makefile  memory_leak  memory_leak.cpp  memory_leak_malloc  memory_leak_malloc.c  memory_leak_malloc.o  memory_leak.o

student@os:~/.../jemalloc$ LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so MALLOC_CONF="stats_print:true" /bin/ps
  PID TTY          TIME CMD
 1581 pts/22   00:00:00 ps
26732 pts/22   00:00:01 bash

Guide: Memory Actions (and Leaks) in Existing Programs

We can use Valgrind to investigate existing programs in the system. This tells us whether they possess memory leaks:

student@os:~/.../memory-actions$ valgrind ls
==24669== Memcheck, a memory error detector
==24669== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==24669== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==24669== Command: ls
==24669==
Makefile  memory_leak  memory_leak.cpp  memory_leak_malloc  memory_leak_malloc.c  memory_leak_malloc.o  memory_leak.o
==24669==
==24669== HEAP SUMMARY:
==24669==     in use at exit: 21,696 bytes in 14 blocks
==24669==   total heap usage: 51 allocs, 37 frees, 61,331 bytes allocated
==24669==
==24669== LEAK SUMMARY:
==24669==    definitely lost: 0 bytes in 0 blocks
==24669==    indirectly lost: 0 bytes in 0 blocks
==24669==      possibly lost: 0 bytes in 0 blocks
==24669==    still reachable: 21,696 bytes in 14 blocks
==24669==         suppressed: 0 bytes in 0 blocks
==24669== Rerun with --leak-check=full to see details of leaked memory
==24669==
==24669== For counts of detected and suppressed errors, rerun with: -v
==24669== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

student@os:~/.../memory-actions$ valgrind ps
==24671== Memcheck, a memory error detector
==24671== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==24671== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==24671== Command: ps
==24671==
  PID TTY          TIME CMD
24671 pts/22   00:00:00 memcheck-amd64-
26732 pts/22   00:00:01 bash
==24671==
==24671== HEAP SUMMARY:
==24671==     in use at exit: 264,929 bytes in 25 blocks
==24671==   total heap usage: 692 allocs, 667 frees, 334,268 bytes allocated
==24671==
==24671== LEAK SUMMARY:
==24671==    definitely lost: 0 bytes in 0 blocks
==24671==    indirectly lost: 0 bytes in 0 blocks
==24671==      possibly lost: 0 bytes in 0 blocks
==24671==    still reachable: 264,929 bytes in 25 blocks
==24671==         suppressed: 0 bytes in 0 blocks
==24671== Rerun with --leak-check=full to see details of leaked memory
==24671==
==24671== For counts of detected and suppressed errors, rerun with: -v
==24671== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

student@os:~/.../memory-actions$ valgrind bash -c 'echo "ha"'
==24675== Memcheck, a memory error detector
==24675== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==24675== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==24675== Command: bash -c echo\ "ha"
==24675==
ha
==24675==
==24675== HEAP SUMMARY:
==24675==     in use at exit: 43,056 bytes in 672 blocks
==24675==   total heap usage: 774 allocs, 102 frees, 51,405 bytes allocated
==24675==
==24675== LEAK SUMMARY:
==24675==    definitely lost: 12 bytes in 1 blocks
==24675==    indirectly lost: 0 bytes in 0 blocks
==24675==      possibly lost: 0 bytes in 0 blocks
==24675==    still reachable: 43,044 bytes in 671 blocks
==24675==         suppressed: 0 bytes in 0 blocks
==24675== Rerun with --leak-check=full to see details of leaked memory
==24675==
==24675== For counts of detected and suppressed errors, rerun with: -v
==24675== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

We can see that ls and ps don't have memory leaks. However, the shell (Bash) shows a memory leak of 12 bytes (on the test system). This may be a false positive or the subject of an actual investigation.

Note that the still reachable section of the output refers to memory that wasn't freed, but still has pointers referring to it. A true memory leak occurs when no pointers refer any memory area.

Practice

Investigate 2-3 other executables in the system using Valgrind.
Use ltrace to list malloc() and free() calls made by the investigated system executables.

Note that, as explained in the Software Stack lab, on some systems, ltrace does not accurately show the output, due to now binding. Fear not, you can always check the library calls with a more verbose and harder to parse ltrace command:

student@os:~$ ltrace -x "*"

Guide: Memory Leaks

A memory leak happens when a memory region is allocated but no longer accessible. This typically occurs when a pointer that referenced a memory area is redirected to a new memory location, leaving the original memory area unreachable and unable to be freed.

Enter the chapters/data/investigate-memory/guides/memory-leak folder, run make skels. Enter the chapters/data/investigate-memory/guides/memory-leak/support/src directory. It stores two files showing memory leaks:

one in C++: memory_leak.cpp
one in C: memory_leak_malloc.c

Let's build and run the two executables:

student@os:~/.../memory-leak/support/src$ make
g++    -c -o memory_leak.o memory_leak.cpp
cc   memory_leak.o  -lstdc++ -o memory_leak
cc    -c -o memory_leak_malloc.o memory_leak_malloc.c
cc   memory_leak_malloc.o  -lstdc++ -o memory_leak_malloc

Running them yields similar output:

student@os:~/.../memory-leak/support/src$ ./memory_leak
Linus Torvalds is 22 years old and likes Linux
Steve Jobs is 23 years old and likes macOS
student@os:~/.../memory-leak/support/src$ ./memory_leak_malloc
Linus Torvalds is 22 years old and likes Linux
Steve Jobs is 23 years old and likes macOS

We investigate the memory leaks of the two programs by using Valgrind:

student@os:~/.../memory-leak/support/src$ valgrind ./memory_leak
==22362== Memcheck, a memory error detector
==22362== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==22362== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==22362== Command: ./memory_leak
==22362==
Linus Torvalds is 22 years old and likes Linux
Steve Jobs is 23 years old and likes macOS
==22362==
==22362== HEAP SUMMARY:
==22362==     in use at exit: 72 bytes in 1 blocks
==22362==   total heap usage: 4 allocs, 3 frees, 73,872 bytes allocated
==22362==
==22362== LEAK SUMMARY:
==22362==    definitely lost: 72 bytes in 1 blocks
==22362==    indirectly lost: 0 bytes in 0 blocks
==22362==      possibly lost: 0 bytes in 0 blocks
==22362==    still reachable: 0 bytes in 0 blocks
==22362==         suppressed: 0 bytes in 0 blocks
==22362== Rerun with --leak-check=full to see details of leaked memory
==22362==
==22362== For counts of detected and suppressed errors, rerun with: -v
==22362== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

student@os:~/.../memory-leak/support/src$ valgrind ./memory_leak_malloc
==22369== Memcheck, a memory error detector
==22369== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==22369== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==22369== Command: ./memory_leak_malloc
==22369==
Linus Torvalds is 22 years old and likes Linux
Steve Jobs is 23 years old and likes macOS
==22369==
==22369== HEAP SUMMARY:
==22369==     in use at exit: 148 bytes in 1 blocks
==22369==   total heap usage: 3 allocs, 2 frees, 1,320 bytes allocated
==22369==
==22369== LEAK SUMMARY:
==22369==    definitely lost: 148 bytes in 1 blocks
==22369==    indirectly lost: 0 bytes in 0 blocks
==22369==      possibly lost: 0 bytes in 0 blocks
==22369==    still reachable: 0 bytes in 0 blocks
==22369==         suppressed: 0 bytes in 0 blocks
==22369== Rerun with --leak-check=full to see details of leaked memory
==22369==
==22369== For counts of detected and suppressed errors, rerun with: -v
==22369== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

As we are doing allocations that are not freed, this results in memory leaks.

For malloc()-based programs, we can use mtrace() feature and mtrace command to verify proper allocations with malloc() and deallocations with free(). We call mtrace() in the program (in memory_leak_malloc.c) to enable malloc() and free() checking.

To use mtrace() we define the MALLOC_TRACE environment variable. We probably also require to preload the libc malloc debugging library, so we use LD_PRELOAD for that. Note that the file path used for LD_PRELOAD may need to be updated, depending on your distribution:

student@os:~/.../memory-leak/support/src$ LD_PRELOAD=/lib/x86_64-linux-gnu/libc_malloc_debug.so.0 MALLOC_TRACE=mem.trace ./memory_leak_malloc
Linus Torvalds is 22 years old and likes Linux
Steve Jobs is 23 years old and likes macOS

Subsequently, we use the mtrace tool to show information about the leaked data:

student@os:~/.../memory-leak/support/src$ mtrace ./memory_leak_malloc mem.trace

Memory not freed:
-----------------
           Address     Size     Caller
0x000056506d8be6a0     0x94  at 0x56506c3777ec

The size (0x94) is the same value shown by Valgrind (148).

mtrace provides an outcome similar to Valgrind. Valgrind is however more powerful: it works on different types of memory (not only those allocated with malloc()) and it doesn't require access to the source code (and the compiler phase).

Practice

Print the size of the Student class and the struct student structure to see if it equates to the leak shown by Valgrind.
Solve the memory leaks in both programs. Run the checker (./checker.sh in the memory-leak/support/tests/ folder) to check your results.
Sample run:

student@so:~/.../support/tests/$ ./checker.sh
make: Entering directory '/home/student/operating-systems/chapters/data/investigate-memory/guides/memory-leak/support/src'
g++    -c -o memory_leak.o memory_leak.cpp
cc   memory_leak.o  -lstdc++ -o memory_leak
cc    -c -o memory_leak_malloc.o memory_leak_malloc.c
cc   memory_leak_malloc.o  -lstdc++ -o memory_leak_malloc
make: Leaving directory '/home/student/operating-systems/chapters/data/investigate-memory/guides/memory-leak/support/src'
-------------------------------------------------
Checking memory leaks for C executable: ../src/memory_leak
C executable is leak-free!

Points for ../src/memory_leak: 50/50
-------------------------------------------------
Checking memory leaks for C++ executable: ../src/memory_leak_malloc
C++ executable is leak-free!

Points for ../src/memory_leak_malloc: 50/50
-------------------------------------------------
Total Points: 100/100

Lab 4 - Investigate Memory

Task: Allocating and Deallocating Memory​

Task: Memory Mapping​

Task: Memory Regions​

Task: Modifying Memory Region Size​

Task: Memory Layout of Statically-Linked and Dynamically-Linked Executables​

Task: Page Mapper​

Task: Operator overloading​

Investigate Memory Actions​

malloc() in Musl​

Guide: App Investigation: Deluge​

Practice​

Guide: App Investigation: Servo​

Practice​

Guide: Investigation: Allocator in the D Programming Language​

Practice​

Guide: App Investigation: Git​

Practice​

Guide: jemalloc​

Guide: Memory Actions (and Leaks) in Existing Programs​

Practice​

Guide: Memory Leaks​

Practice​

Task: Allocating and Deallocating Memory

Task: Memory Mapping

Task: Memory Regions

Task: Modifying Memory Region Size

Task: Memory Layout of Statically-Linked and Dynamically-Linked Executables

Task: Page Mapper

Task: Operator overloading

Investigate Memory Actions

`malloc()` in Musl

Guide: App Investigation: Deluge

Practice

Guide: App Investigation: Servo

Practice

Guide: Investigation: Allocator in the D Programming Language

Practice

Guide: App Investigation: Git

Practice

Guide: `jemalloc`

Guide: Memory Actions (and Leaks) in Existing Programs

Practice

Guide: Memory Leaks

Practice