TCMalloc源码学习（四）（小内存块释放）

pagemap_和pagemap_cache_

PageHeap有两个map，pagemap_记录某一内存页对应哪一个span，显然可能多页对应一个span，pagemap_cache_记录某一内存页对应哪一个SizeClass。

在TCMalloc源码分析（一）中有提到过pagemap_所占内存的问题，假设32位系统4GB可用内存，若pagemap_使用数组实现需要占用4MB的内存（假设一页4KB），仿佛还可以接受，但如果是64位系统呢？所以实际上TCMalloc使用了radix-tree树实现了

pagemap_（64位系统使用三层radix-tree TCMalloc_PageMap2，32位使用两层 TCMalloc_PageMap3）。

radix-tree其实是一棵多叉树，原理是这样：比如三层，会把对应的key的二进制位分成三部分（High，Medium，Low），依次来生成树的三层，最后一层是叶子节点保存key对应的value。

节点都是插入key的过程动态生成的，不像数组实现一开始就要很大一块内存，radix-tree是随着TCMalloc内存分配的增多而增大，而且因为通常都是相邻的页分配出去，也就是页号的二进位比较相似，所以可以共用high和medium节点，这样可以减缓随着页内存分配的增多radix-tree所耗内存的增速。

之前说 pagemap_是PageId ----> Span的映射，再show一下这个图

pagemap_cache_是PageId -----> SizeClass的映射，用的数据结构是一个压缩的哈希表PackedCache，实现上就是用了一个数组，然后用key做hash插入到对应的entry，但是却内部有不少细节：PackedCache有三个位数kHashbits，kValuebits，kKeybits。数组的长度是1<<kHashBits，hash函数直接是用key对长度取模（key & ( 1 << kHashBits - 1)）；kValueBits和kKeyBits代表value和key分别占多少位；有一个问题，如果kKeybits大于了kHashbits，那就有可能多个key映射到同一个entry，为了区分开，每一个entry放的值的数值二进制位是部分key位和所有value位的： |  kKeybits - kHashbits   |  kValuebits  | 。

entry的类型在64位系统上是64位，32位系统上16位，在更新每一个entry时候不存在只更新到部分数位的情况，所以可以无锁访问pagemap_cache_。

PageId -----> SizeClass的映射能用到压缩的哈希表，也是因为SizeClass是一个比较小的值，如果纯用作Value的话比如Value可能是16位或者64位，会浪费一些数位，剩余的数位也可以利用起来做key。

小内存释放

1.求出要释放的内存指针在哪一内存页上；

const PageID p = reinterpret_cast<uintptr_t >(ptr) >> kPageShift;

2.用PageID在pagemap_cache_找对应的SizeClass；

size_t cl = Static:: pageheap()->GetSizeClassIfCached (p);

3.如果cache中没有对应的PageID，用在pagemap_中找对应的Span，然后得到SizeClass；

span = Static ::pageheap()-> GetDescriptor(p );

cl = span ->sizeclass;

4.取到对应线程的ThreadCache，用参数cl调用Deallocate；

heap->Deallocate (ptr, cl);

在本地线程释放

函数ThreadCache:: Deallocate 的实现：

inline void ThreadCache:: Deallocate(void * ptr, size_t cl ) {

FreeList* list = &list_ [cl];

size_ += Static::sizemap ()->ByteSizeForClass( cl);

ssize_t size_headroom = max_size_ - size_ - 1;

// This catches back-to-back frees of allocs in the same size

// class. A more comprehensive (and expensive) test would be to walk

// the entire freelist. But this might be enough to find some bugs.

ASSERT( ptr != list ->Next());

list-> Push(ptr );

ssize_t list_headroom =

static_cast<ssize_t >(list-> max_length()) - list ->length();

// There are two relatively uncommon things that require further work.

// In the common case we're done, and in that case we need a single branch

// because of the bitwise-or trick that follows.

if (( list_headroom | size_headroom ) < 0) {

if (list_headroom < 0) {

ListTooLong(list , cl);

    }

if (size_ >= max_size_) Scavenge();

  }

}

主要是找到对应SizeClass的FreeList，把回收的内存插入空闲链表头部。ThreadCache有两个指标，一个是每一条FreeList都有的max_length限制，一个是总共使用的内存max_size_限制，如果超过了需要调整。ListTooLong就是当前FreeList长度超过max_length的调整：

void ThreadCache ::ListTooLong( FreeList* list , size_t cl) {

const int batch_size = Static:: sizemap()->num_objects_to_move (cl);

ReleaseToCentralCache( list, cl , batch_size);

// If the list is too long, we need to transfer some number of

// objects to the central cache.  Ideally, we would transfer

// num_objects_to_move, so the code below tries to make max_length

// converge on num_objects_to_move.

if ( list->max_length () < batch_size) {

// Slow start the max_length so we don't overreserve.

list->set_max_length (list-> max_length() + 1);

  } else if (list ->max_length() > batch_size) {

// If we consistently go over max_length, shrink max_length.  If we don't

// shrink it, some amount of memory will always stay in this freelist.

list->set_length_overages (list-> length_overages() + 1);

if (list ->length_overages() > kMaxOverages) {

ASSERT(list ->max_length() > batch_size);

list->set_max_length (list-> max_length() - batch_size );

list->set_length_overages (0);

    }

  }

}

当长度超过上限的时候，移回部分空闲对象到Central Cache中去，ReleaseToCentralCache实现不贴了，无非就是从线程FreeList弹出指定个内存对象插入到对应CentralFreeList中去。

在Centreal Cache中释放

对应从CentrealCache中分配内存的RemoveRange接口，把内存收回到CentrealCache中的接口是InsertRange，InsertRange的实现也是先判断转移缓存（TCEntry）中是否还有空间放置回收内存，有就放到转移缓存然后就返回了，这个在对内存频繁分配释放的时候比较高效。若没有地方放了就要转到其对应的Span了，ReleaseListToSpans调用如下：

void CentralFreeList ::ReleaseListToSpans( void* start ) {

while ( start) {

void *next = SLL_Next( start);

ReleaseToSpans(start );

start = next ;

  }

}

就是一个一个内存对象调用ReleaseToSpans 释放，ReleaseToSpans 如下:

void CentralFreeList ::ReleaseToSpans( void* object ) {

Span* span = MapObjectToSpan (object);

ASSERT( span != NULL );

ASSERT( span->refcount > 0);

// If span is empty, move it to non-empty list

if ( span->objects == NULL) {

tcmalloc::DLL_Remove (span);

tcmalloc::DLL_Prepend (&nonempty_, span);

Event(span , 'N', 0);

  }

// The following check is expensive, so it is disabled by default

if ( false) {

// Check that object does not occur in list

int got = 0;

for (void * p = span->objects ; p != NULL; p = *((void**) p)) {

ASSERT(p != object);

got++;

    }

ASSERT(got + span-> refcount ==

           ( span->length <<kPageShift) /

Static::sizemap ()->ByteSizeForClass( span->sizeclass ));

  }

counter_++;

span-> refcount--;

if ( span->refcount == 0) {

Event(span , '#', 0);

counter_ -= ((span ->length<< kPageShift) /

Static::sizemap ()->ByteSizeForClass( span->sizeclass ));

tcmalloc::DLL_Remove (span);

    -- num_spans_;

// Release central list lock while operating on pageheap

lock_.Unlock ();

                {

SpinLockHolder h(Static ::pageheap_lock());

Static::pageheap ()->Delete( span);

                }

lock_.Lock ();

  } else {

    *( reinterpret_cast<void **>(object)) = span->objects ;

span->objects = object;

  }

}

过程简单描述如下：

1.判断这个Span所标识的对象是不是之前已经分配完了，若是就要把他从CentralFreeList的empty_ Spans List列表中挪出到nonempty_  Spans List中，因为我要把返回的内存对象给这个Span了

；

2.递减Span的引用计数，如果已经没有人在引用了就要把Span标识的所有内存返还给PageHeap了。

内存在PageHeap中的释放

与在PageHeap中分配内存的New对应，释放内存是Delete，Delete主要是取消Span与某个SizeClass关联和取消这个Span正在使用的状态标记为ON_NORMAL_FREELIST，即将要放入normal list中，之后就是调用MergeIntoFreeList，即和邻近的空闲内存合并放入空闲链表中。MergeIntoFreeList的实现如下：

void PageHeap ::MergeIntoFreeList( Span* span ) {

ASSERT( span->location != Span:: IN_USE);

// Coalesce -- we guarantee that "p" != 0, so no bounds checking

// necessary.  We do not bother resetting the stale pagemap

// entries for the pieces we are merging together because we only

// care about the pagemap entries for the boundaries.

//

// Note that only similar spans are merged together.  For example,

// we do not coalesce "returned" spans with "normal" spans.

const PageID p = span-> start;

const Length n = span-> length;

Span* prev = GetDescriptor (p-1);

if ( prev != NULL && prev-> location == span ->location) {

// Merge preceding span into this span

ASSERT(prev ->start + prev->length == p);

const Length len = prev->length ;

RemoveFromFreeList(prev );

DeleteSpan(prev );

span->start -= len;

span->length += len;

pagemap_.set (span-> start, span );

Event(span , 'L', len);

  }

Span* next = GetDescriptor (p+ n);

if ( next != NULL && next-> location == span ->location) {

// Merge next span into this span

ASSERT(next ->start == p+n );

const Length len = next->length ;

RemoveFromFreeList(next );

DeleteSpan(next );

span->length += len;

pagemap_.set (span-> start + span ->length - 1, span);

Event(span , 'R', len);

  }

PrependToFreeList( span);

}

MergeIntoFreeList就是取目标Span标识的内存邻近的页对应的Span出来，判断如果能和目标Span合并就合并之，之后才插入到normal free list中去。

回到Delete，MergeIntoFreeList返回后，IncrementalScavenge调用有可能触发把一些空闲内存释放回系统的操作，释放的策略是这样：有一个scavenge_counter_计数，每次Delete调用都会降低其值，若降为0才真正去释放给系统。可以调整scavenge_counter_的值来控制释放给系统的频率，IncrementalScavenge代码如下：

void PageHeap ::IncrementalScavenge( Length n ) {

// Fast path; not yet time to release memory

scavenge_counter_ -= n;

if ( scavenge_counter_ >= 0) return ;  // Not yet time to scavenge

const double rate = FLAGS_tcmalloc_release_rate;

if ( rate <= 1e-6) {

// Tiny release rate means that releasing is disabled.

scavenge_counter_ = kDefaultReleaseDelay ;

return;

  }

Length released_pages = ReleaseAtLeastNPages (1);

if ( released_pages == 0) {

// Nothing to scavenge, delay for a while.

scavenge_counter_ = kDefaultReleaseDelay ;

  } else {

// Compute how long to wait until we return memory.

// FLAGS_tcmalloc_release_rate==1 means wait for 1000 pages

// after releasing one page.

const double mult = 1000.0 / rate;

double wait = mult * static_cast<double >(released_pages);

if (wait > kMaxReleaseDelay) {

// Avoid overflow and bound to reasonable range.

wait = kMaxReleaseDelay ;

    }

scavenge_counter_ = static_cast <int64_t>( wait);

  }

}

又是ReleaseAtLeastNPages 调用，在TCMalloc源码分析（三）中有详细分析这个调用，记住windows上内存是不还给系统的，细节不在复述了。

再回到ThreadCache

之前说到ListTooLong返还内存给Central Cache后，调整了max_length，主要是怕链表后面的空闲内存一直在本地线程中，自己不用也不释放给其他线程用。

ListTooLong是调整单个FreeList的长度，Scavenge则是在整个ThreadCache使用的内存上来考虑，当前使用的内存大于一个上限后就会被调用，Scavenge代码如下：

void ThreadCache ::Scavenge() {

// If the low-water mark for the free list is L, it means we would

// not have had to allocate anything from the central cache even if

// we had reduced the free list size by L.  We aim to get closer to

// that situation by dropping L/2 nodes from the free list.  This

// may not release much memory, but if so we will call scavenge again

// pretty soon and the low-water marks will be high on that call.

//int64 start = CycleClock::Now();

for ( int cl = 0; cl < kNumClasses; cl ++) {

FreeList* list = &list_[ cl];

const int lowmark = list->lowwatermark ();

if (lowmark > 0) {

const int drop = ( lowmark > 1) ? lowmark /2 : 1;

ReleaseToCentralCache(list , cl, drop);

// Shrink the max length if it isn't used.  Only shrink down to

// batch_size -- if the thread was active enough to get the max_length

// above batch_size, it will likely be that active again.  If

// max_length shinks below batch_size, the thread will have to

// go through the slow-start behavior again.  The slow-start is useful

// mainly for threads that stay relatively idle for their entire

// lifetime.

const int batch_size = Static::sizemap ()->num_objects_to_move( cl);

if (list ->max_length() > batch_size) {

list->set_max_length (

max<int >(list-> max_length() - batch_size , batch_size));

      }

    }

list->clear_lowwatermark ();

  }

IncreaseCacheLimit();

}

整个过程就是遍历所有FreeList进行逐一释放，每一个FreeList有一个lowwatermark L，代表上次回收内存后FreeList的长度，每次回收时释放 L/2个object，下次回收时L就表示自从上次回收后一直没有用过的内存，那就把他还给Central Cache吧。这就是这种用历史记录预测未来内存使用情况的策略。

最后就是IncreaseCacheLimit调用，实现为锁住后调用IncreaseCacheLimitLocked，IncreaseCacheLimitLocked的代码如下：

void ThreadCache ::IncreaseCacheLimitLocked() {

if ( unclaimed_cache_space_ > 0) {

// Possibly make unclaimed_cache_space_ negative.

unclaimed_cache_space_ -= kStealAmount ;

max_size_ += kStealAmount ;

return;

  }

// Don't hold pageheap_lock too long.  Try to steal from 10 other

// threads before giving up.  The i < 10 condition also prevents an

// infinite loop in case none of the existing thread heaps are

// suitable places to steal from.

for ( int i = 0; i < 10;

       ++ i, next_memory_steal_ = next_memory_steal_-> next_) {

// Reached the end of the linked list.  Start at the beginning.

if (next_memory_steal_ == NULL) {

ASSERT(thread_heaps_ != NULL);

next_memory_steal_ = thread_heaps_ ;

    }

if (next_memory_steal_ == this ||

next_memory_steal_->max_size_ <= kMinThreadCacheSize) {

continue;

    }

next_memory_steal_->max_size_ -= kStealAmount;

max_size_ += kStealAmount ;

next_memory_steal_ = next_memory_steal_ ->next_;

return;

  }

}

kStealAmount是在ThreadCache被强制Scavenge后，max_size_应该从unclaimed_cache_space_或者其他线程偷取的字节数，这样就可以使得下次 Scavenge被延迟避免频繁Scavenge。这个过程其实是在表达这样的意思：这次是我花时间把自己的内存返还给Central Cache了，下次轮到其他线程去做了。因为这个过程是在多个线程之间调整他们所能够拥有的内存上限，所以当然要用到锁了。

总结：

释放过程不像其他malloc-free实现，在内存头几个字节保存了size，而是直接算出内存所在页号，借助pagemap_cache_和pagemap_索引其应回收到的位置。内存在ThreadCache，CentralFreeList，PageHeap之间层层回收，优先回收在本地，延迟回收到下层，其间有空闲内存合并，启发式的回收策略，多个线程互相调整回收频率，为达到内存在不同线程间有效利用，高效回收。