Buddy allocation algorithm finished and documented!

Author: adam-sawicki-a

docs/gfx/Buddy_allocator.png

@@ -141,7 +141,23 @@ <h2><a class="anchor" id="linear_algorithm_ring_buffer"></a>
 <div class="image">
 <img src="../gfx/Linear_allocator_6_ring_buffer_lost.png" alt="Ring buffer with lost allocations"/>
 </div>
-<p>Ring buffer is available only in pools with one memory block - <a class="el" href="struct_vma_pool_create_info.html#ae41142f2834fcdc82baa4883c187b75c" title="Maximum number of blocks that can be allocated in this pool. Optional. ">VmaPoolCreateInfo::maxBlockCount</a> must be 1. Otherwise behavior is undefined. </p>
+<p>Ring buffer is available only in pools with one memory block - <a class="el" href="struct_vma_pool_create_info.html#ae41142f2834fcdc82baa4883c187b75c" title="Maximum number of blocks that can be allocated in this pool. Optional. ">VmaPoolCreateInfo::maxBlockCount</a> must be 1. Otherwise behavior is undefined.</p>
+<h1><a class="anchor" id="buddy_algorithm"></a>
+Buddy allocation algorithm</h1>
+<p>There is another allocation algorithm that can be used with custom pools, called "buddy". Its internal data structure is based on a tree of blocks, each having size that is a power of two and a half of its parent's size. When you want to allocate memory of certain size, a free node in the tree is located. If it's too large, it is recursively split into two halves (called "buddies"). However, if requested allocation size is not a power of two, the size of a tree node is aligned up to the nearest power of two and the remaining space is wasted. When two buddy nodes become free, they are merged back into one larger node.</p>
+<div class="image">
+<img src="../gfx/Buddy_allocator.png" alt="Buddy allocator"/>
+</div>
+<p>The advantage of buddy allocation algorithm over default algorithm is faster allocation and deallocation, as well as smaller external fragmentation. The disadvantage is more wasted space (internal fragmentation).</p>
+<p>For more information, please read <a href="https://en.wikipedia.org/wiki/Buddy_memory_allocation">"Buddy memory allocation" on Wikipedia</a> or other sources that describe this concept in general.</p>
+<p>To use buddy allocation algorithm with a custom pool, add flag <a class="el" href="vk__mem__alloc_8h.html#a9a7c45f9c863695d98c83fa5ac940fe7a97a0dc38e5161b780594d998d313d35e" title="Enables alternative, buddy allocation algorithm in this pool. ">VMA_POOL_CREATE_BUDDY_ALGORITHM_BIT</a> to <a class="el" href="struct_vma_pool_create_info.html#a8405139f63d078340ae74513a59f5446" title="Use combination of VmaPoolCreateFlagBits. ">VmaPoolCreateInfo::flags</a> while creating <a class="el" href="struct_vma_pool.html" title="Represents custom memory pool. ">VmaPool</a> object.</p>
+<p>Several limitations apply to pools that use buddy algorithm:</p>
+<ul>
+<li>It is recommended to use <a class="el" href="struct_vma_pool_create_info.html#aa4265160536cdb9be821b7686c16c676" title="Size of a single VkDeviceMemory block to be allocated as part of this pool, in bytes. Optional. ">VmaPoolCreateInfo::blockSize</a> that is a power of two. Otherwise, only largest power of two smaller than the size is used for allocations. The remaining space always stays unused.</li>
+<li><a class="el" href="debugging_memory_usage.html#debugging_memory_usage_margins">Margins</a> and <a class="el" href="debugging_memory_usage.html#debugging_memory_usage_corruption_detection">corruption detection</a> don't work in such pools.</li>
+<li><a class="el" href="lost_allocations.html">Lost allocations</a> don't work in such pools. You can use them, but they never become lost. Support may be added in the future.</li>
+<li><a class="el" href="defragmentation.html">Defragmentation</a> doesn't work with allocations made from such pool. </li>
+</ul>
 </div></div><!-- contents -->
 <!-- start footer part -->
 <hr class="footer"/><address class="footer"><small>

docs/html/custom_memory_pools.html

@@ -89,7 +89,7 @@ <h1><a class="anchor" id="debugging_memory_usage_margins"></a>
 <p>If your bug goes away after enabling margins, it means it may be caused by memory being overwritten outside of allocation boundaries. It is not 100% certain though. Change in application behavior may also be caused by different order and distribution of allocations across memory blocks after margins are applied.</p>
 <p>The margin is applied also before first and after last allocation in a block. It may occur only once between two adjacent allocations.</p>
 <p>Margins work with all types of memory.</p>
-<p>Margin is applied only to allocations made out of memory blocks and not to dedicated allocations, which have their own memory block of specific size. It is thus not applied to allocations made using <a class="el" href="vk__mem__alloc_8h.html#ad9889c10c798b040d59c92f257cae597a3fc311d855c2ff53f1090ef5c722b38f" title="Set this flag if the allocation should have its own memory block. ">VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT</a> flag or those automatically decided to put into dedicated allocations, e.g. due to its large size or recommended by VK_KHR_dedicated_allocation extension.</p>
+<p>Margin is applied only to allocations made out of memory blocks and not to dedicated allocations, which have their own memory block of specific size. It is thus not applied to allocations made using <a class="el" href="vk__mem__alloc_8h.html#ad9889c10c798b040d59c92f257cae597a3fc311d855c2ff53f1090ef5c722b38f" title="Set this flag if the allocation should have its own memory block. ">VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT</a> flag or those automatically decided to put into dedicated allocations, e.g. due to its large size or recommended by VK_KHR_dedicated_allocation extension. Margins are also not active in custom pools created with <a class="el" href="vk__mem__alloc_8h.html#a9a7c45f9c863695d98c83fa5ac940fe7a97a0dc38e5161b780594d998d313d35e" title="Enables alternative, buddy allocation algorithm in this pool. ">VMA_POOL_CREATE_BUDDY_ALGORITHM_BIT</a> flag.</p>
 <p>Margins appear in <a class="el" href="statistics.html#statistics_json_dump">JSON dump</a> as part of free space.</p>
 <p>Note that enabling margins increases memory usage and fragmentation.</p>
 <h1><a class="anchor" id="debugging_memory_usage_corruption_detection"></a>

docs/html/debugging_memory_usage.html

@@ -102,6 +102,7 @@ <h1><a class="anchor" id="main_table_of_contents"></a>
 <li><a class="el" href="custom_memory_pools.html#linear_algorithm_ring_buffer">Ring buffer</a></li>
 </ul>
 </li>
+<li><a class="el" href="custom_memory_pools.html#buddy_algorithm">Buddy allocation algorithm</a></li>
 </ul>
 </li>
 <li><a class="el" href="defragmentation.html">Defragmentation</a></li>

docs/html/index.html

@@ -912,14 +912,17 @@ <h2 class="memtitle"><span class="permalink"><a href="#a9a7c45f9c863695d98c83fa5
 <tr><th colspan="2">Enumerator</th></tr><tr><td class="fieldname"><a id="a9a7c45f9c863695d98c83fa5ac940fe7a9f1a499508a8edb4e8ba40aa0290a3d2"></a>VMA_POOL_CREATE_IGNORE_BUFFER_IMAGE_GRANULARITY_BIT&#160;</td><td class="fielddoc"><p>Use this flag if you always allocate only buffers and linear images or only optimal images out of this pool and so Buffer-Image Granularity can be ignored. </p>
 <p>This is an optional optimization flag.</p>
 <p>If you always allocate using <a class="el" href="vk__mem__alloc_8h.html#ac72ee55598617e8eecca384e746bab51">vmaCreateBuffer()</a>, <a class="el" href="vk__mem__alloc_8h.html#a02a94f25679275851a53e82eacbcfc73" title="Function similar to vmaCreateBuffer(). ">vmaCreateImage()</a>, <a class="el" href="vk__mem__alloc_8h.html#a7fdf64415b6c3d83c454f28d2c53df7b">vmaAllocateMemoryForBuffer()</a>, then you don't need to use it because allocator knows exact type of your allocations so it can handle Buffer-Image Granularity in the optimal way.</p>
-<p>If you also allocate using <a class="el" href="vk__mem__alloc_8h.html#a0faa3f9e5fb233d29d1e00390650febb" title="Function similar to vmaAllocateMemoryForBuffer(). ">vmaAllocateMemoryForImage()</a> or <a class="el" href="vk__mem__alloc_8h.html#abf28077dbf82d0908b8acbe8ee8dd9b8" title="General purpose memory allocation. ">vmaAllocateMemory()</a>, exact type of such allocations is not known, so allocator must be conservative in handling Buffer-Image Granularity, which can lead to suboptimal allocation (wasted memory). In that case, if you can make sure you always allocate only buffers and linear images or only optimal images out of this pool, use this flag to make allocator disregard Buffer-Image Granularity and so make allocations more optimal. </p>
+<p>If you also allocate using <a class="el" href="vk__mem__alloc_8h.html#a0faa3f9e5fb233d29d1e00390650febb" title="Function similar to vmaAllocateMemoryForBuffer(). ">vmaAllocateMemoryForImage()</a> or <a class="el" href="vk__mem__alloc_8h.html#abf28077dbf82d0908b8acbe8ee8dd9b8" title="General purpose memory allocation. ">vmaAllocateMemory()</a>, exact type of such allocations is not known, so allocator must be conservative in handling Buffer-Image Granularity, which can lead to suboptimal allocation (wasted memory). In that case, if you can make sure you always allocate only buffers and linear images or only optimal images out of this pool, use this flag to make allocator disregard Buffer-Image Granularity and so make allocations faster and more optimal. </p>
 </td></tr>
 <tr><td class="fieldname"><a id="a9a7c45f9c863695d98c83fa5ac940fe7a13c8a444197c67866be9cb05599fc726"></a>VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT&#160;</td><td class="fielddoc"><p>Enables alternative, linear allocation algorithm in this pool. </p>
 <p>Specify this flag to enable linear allocation algorithm, which always creates new allocations after last one and doesn't reuse space from allocations freed in between. It trades memory consumption for simplified algorithm and data structure, which has better performance and uses less memory for metadata.</p>
 <p>By using this flag, you can achieve behavior of free-at-once, stack, ring buffer, and double stack. For details, see documentation chapter <a class="el" href="custom_memory_pools.html#linear_algorithm">Linear allocation algorithm</a>.</p>
-<p>When using this flag, you must specify <a class="el" href="struct_vma_pool_create_info.html#ae41142f2834fcdc82baa4883c187b75c" title="Maximum number of blocks that can be allocated in this pool. Optional. ">VmaPoolCreateInfo::maxBlockCount</a> == 1 (or 0 for default). </p>
+<p>When using this flag, you must specify <a class="el" href="struct_vma_pool_create_info.html#ae41142f2834fcdc82baa4883c187b75c" title="Maximum number of blocks that can be allocated in this pool. Optional. ">VmaPoolCreateInfo::maxBlockCount</a> == 1 (or 0 for default).</p>
+<p>For more details, see <a class="el" href="custom_memory_pools.html#linear_algorithm">Linear allocation algorithm</a>. </p>
 </td></tr>
-<tr><td class="fieldname"><a id="a9a7c45f9c863695d98c83fa5ac940fe7a97a0dc38e5161b780594d998d313d35e"></a>VMA_POOL_CREATE_BUDDY_ALGORITHM_BIT&#160;</td><td class="fielddoc"><p>TODO </p>
+<tr><td class="fieldname"><a id="a9a7c45f9c863695d98c83fa5ac940fe7a97a0dc38e5161b780594d998d313d35e"></a>VMA_POOL_CREATE_BUDDY_ALGORITHM_BIT&#160;</td><td class="fielddoc"><p>Enables alternative, buddy allocation algorithm in this pool. </p>
+<p>It operates on a tree of blocks, each having size that is a power of two and a half of its parent's size. Comparing to default algorithm, this one provides faster allocation and deallocation and decreased external fragmentation, at the expense of more memory wasted (internal fragmentation).</p>
+<p>For more details, see <a class="el" href="custom_memory_pools.html#buddy_algorithm">Buddy allocation algorithm</a>. </p>
 </td></tr>
 <tr><td class="fieldname"><a id="a9a7c45f9c863695d98c83fa5ac940fe7af4d270f8f42517a0f70037ceb6ac1d9c"></a>VMA_POOL_CREATE_ALGORITHM_MASK&#160;</td><td class="fielddoc"><p>Bit mask to extract only <code>ALGORITHM</code> bits from entire set of flags. </p>
 </td></tr>
@@ -1635,7 +1638,7 @@ <h2 class="memtitle"><span class="permalink"><a href="#a6aced90fcc7b39882b6654a7
 <p>This function works by moving allocations to different places (different <code>VkDeviceMemory</code> objects and/or different offsets) in order to optimize memory usage. Only allocations that are in pAllocations array can be moved. All other allocations are considered nonmovable in this call. Basic rules:</p>
 <ul>
 <li>Only allocations made in memory types that have <code>VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT</code> and <code>VK_MEMORY_PROPERTY_HOST_COHERENT_BIT</code> flags can be compacted. You may pass other allocations but it makes no sense - these will never be moved.</li>
-<li>Custom pools created with <a class="el" href="vk__mem__alloc_8h.html#a9a7c45f9c863695d98c83fa5ac940fe7a13c8a444197c67866be9cb05599fc726" title="Enables alternative, linear allocation algorithm in this pool. ">VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT</a> flag are not defragmented. Allocations passed to this function that come from such pools are ignored.</li>
+<li>Custom pools created with <a class="el" href="vk__mem__alloc_8h.html#a9a7c45f9c863695d98c83fa5ac940fe7a13c8a444197c67866be9cb05599fc726" title="Enables alternative, linear allocation algorithm in this pool. ">VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT</a> or <a class="el" href="vk__mem__alloc_8h.html#a9a7c45f9c863695d98c83fa5ac940fe7a97a0dc38e5161b780594d998d313d35e" title="Enables alternative, buddy allocation algorithm in this pool. ">VMA_POOL_CREATE_BUDDY_ALGORITHM_BIT</a> flag are not defragmented. Allocations passed to this function that come from such pools are ignored.</li>
 <li>Allocations created with <a class="el" href="vk__mem__alloc_8h.html#ad9889c10c798b040d59c92f257cae597a3fc311d855c2ff53f1090ef5c722b38f" title="Set this flag if the allocation should have its own memory block. ">VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT</a> or created as dedicated allocations for any other reason are also ignored.</li>
 <li>Both allocations made with or without <a class="el" href="vk__mem__alloc_8h.html#ad9889c10c798b040d59c92f257cae597a11da372cc3a82931c5e5d6146cd9dd1f" title="Set this flag to use a memory that will be persistently mapped and retrieve pointer to it...">VMA_ALLOCATION_CREATE_MAPPED_BIT</a> flag can be compacted. If not persistently mapped, memory will be mapped temporarily inside this function if needed.</li>
 <li>You must not pass same <a class="el" href="struct_vma_allocation.html" title="Represents single memory allocation. ">VmaAllocation</a> object multiple times in pAllocations array.</li>

docs/html/vk__mem__alloc_8h.html

@@ -4217,7 +4217,7 @@ void Test()
 {
     wprintf(L"TESTING:\n");
 
-    if(true)
+    if(false)
     {
         // # Temporarily insert custom tests here
         // ########################################
@@ -4249,7 +4249,7 @@ void Test()
 
     {
         FILE* file;
-        fopen_s(&file, "LinearAllocator.csv", "w");
+        fopen_s(&file, "Algorithms.csv", "w");
         assert(file != NULL);
         BenchmarkAlgorithms(file);
         fclose(file);

docs/html/vk__mem__alloc_8h_source.html

@@ -16,7 +16,7 @@ macros if you want to configure the library and then include its header to
 include all public interface declarations. Example:
 */
 
-#define VMA_HEAVY_ASSERT(expr) assert(expr)
+//#define VMA_HEAVY_ASSERT(expr) assert(expr)
 //#define VMA_USE_STL_CONTAINERS 1
 //#define VMA_DEDICATED_ALLOCATION 0
 //#define VMA_DEBUG_MARGIN 16

src/Tests.cpp

@@ -60,6 +60,7 @@ Documentation of all members: vk_mem_alloc.h
       - [Stack](@ref linear_algorithm_stack)
       - [Double stack](@ref linear_algorithm_double_stack)
       - [Ring buffer](@ref linear_algorithm_ring_buffer)
+    - [Buddy allocation algorithm](@ref buddy_algorithm)
   - \subpage defragmentation
   - \subpage lost_allocations
   - \subpage statistics
@@ -665,6 +666,42 @@ succeeds.
 Ring buffer is available only in pools with one memory block -
 VmaPoolCreateInfo::maxBlockCount must be 1. Otherwise behavior is undefined.
 
+\section buddy_algorithm Buddy allocation algorithm
+
+There is another allocation algorithm that can be used with custom pools, called
+"buddy". Its internal data structure is based on a tree of blocks, each having
+size that is a power of two and a half of its parent's size. When you want to
+allocate memory of certain size, a free node in the tree is located. If it's too
+large, it is recursively split into two halves (called "buddies"). However, if
+requested allocation size is not a power of two, the size of a tree node is
+aligned up to the nearest power of two and the remaining space is wasted. When
+two buddy nodes become free, they are merged back into one larger node.
+
+![Buddy allocator](../gfx/Buddy_allocator.png)
+
+The advantage of buddy allocation algorithm over default algorithm is faster
+allocation and deallocation, as well as smaller external fragmentation. The
+disadvantage is more wasted space (internal fragmentation).
+
+For more information, please read ["Buddy memory allocation" on Wikipedia](https://en.wikipedia.org/wiki/Buddy_memory_allocation)
+or other sources that describe this concept in general.
+
+To use buddy allocation algorithm with a custom pool, add flag
+#VMA_POOL_CREATE_BUDDY_ALGORITHM_BIT to VmaPoolCreateInfo::flags while creating
+#VmaPool object.
+
+Several limitations apply to pools that use buddy algorithm:
+
+- It is recommended to use VmaPoolCreateInfo::blockSize that is a power of two.
+  Otherwise, only largest power of two smaller than the size is used for
+  allocations. The remaining space always stays unused.
+- [Margins](@ref debugging_memory_usage_margins) and
+  [corruption detection](@ref debugging_memory_usage_corruption_detection)
+  don't work in such pools.
+- [Lost allocations](@ref lost_allocations) don't work in such pools. You can
+  use them, but they never become lost. Support may be added in the future.
+- [Defragmentation](@ref defragmentation) doesn't work with allocations made from
+  such pool.
 
 \page defragmentation Defragmentation
 
@@ -987,6 +1024,7 @@ allocations, which have their own memory block of specific size.
 It is thus not applied to allocations made using #VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT flag
 or those automatically decided to put into dedicated allocations, e.g. due to its
 large size or recommended by VK_KHR_dedicated_allocation extension.
+Margins are also not active in custom pools created with #VMA_POOL_CREATE_BUDDY_ALGORITHM_BIT flag.
 
 Margins appear in [JSON dump](@ref statistics_json_dump) as part of free space.
 
@@ -1981,10 +2019,20 @@ typedef enum VmaPoolCreateFlagBits {
     \ref linear_algorithm.
 
     When using this flag, you must specify VmaPoolCreateInfo::maxBlockCount == 1 (or 0 for default).
+
+    For more details, see [Linear allocation algorithm](@ref linear_algorithm).
     */
     VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT = 0x00000004,
 
-    /** TODO */
+    /** \brief Enables alternative, buddy allocation algorithm in this pool.
+
+    It operates on a tree of blocks, each having size that is a power of two and
+    a half of its parent's size. Comparing to default algorithm, this one provides
+    faster allocation and deallocation and decreased external fragmentation,
+    at the expense of more memory wasted (internal fragmentation).
+
+    For more details, see [Buddy allocation algorithm](@ref buddy_algorithm).
+    */
     VMA_POOL_CREATE_BUDDY_ALGORITHM_BIT = 0x00000008,
 
     /** Bit mask to extract only `ALGORITHM` bits from entire set of flags.
@@ -2446,9 +2494,9 @@ allocations are considered nonmovable in this call. Basic rules:
   `VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT` and `VK_MEMORY_PROPERTY_HOST_COHERENT_BIT`
   flags can be compacted. You may pass other allocations but it makes no sense -
   these will never be moved.
-- Custom pools created with #VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT flag are not
-  defragmented. Allocations passed to this function that come from such pools
-  are ignored.
+- Custom pools created with #VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT or
+  #VMA_POOL_CREATE_BUDDY_ALGORITHM_BIT flag are not defragmented. Allocations
+  passed to this function that come from such pools are ignored.
 - Allocations created with #VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT or
   created as dedicated allocations for any other reason are also ignored.
 - Both allocations made with or without #VMA_ALLOCATION_CREATE_MAPPED_BIT