free llvm engine/context memory when rw_engine is not used

bcc utilizes MCJIT to generate final code and the code
is actually runnable in the process. This, however,
may have negative memory consumption impact.
All data will be actually allocated in memory.

For example, the following large array,
  #define TCP6_RXMIT_BUCKET_BITS 18
  struct tcp6_rxmit_tbl {
    __u64 buckets[1 << TCP6_RXMIT_BUCKET_BITS];
  };
  BPF_ARRAY(rxmit_marking_map, struct tcp6_rxmit_tbl, 1);
is added to examples/cpp/HelloWorld.cpp.
Internally, bcc will define a structure type rxmit_marking_map_table
with members `struct tcp6_rxmit_tbl leaf` and a variable
with this structure type. (see src/cc/export/helpers.h),
so that rw_engine can traverse the type to generate
proper sscanf/snprintf functions.

Even with rw_engine is disabled, the RSS still increased
from 6MB to 12MB. The array size is roughly 2MB.
The additional 4MB overhead is due to some llvm internal overhead
which I did not root cause it.

If rw_engine is disabled, we can actually free llvm
engine and context memory after copying the section data.
We do not need to copy map section data since it is not
used later on. Only map size is needed.

We cannot free llvm context memory if rw_engine is enabled since context
is shared between regular llvm module and sscanf module.

For the above HelloWorld example, if rw_engine is disabled,
this patch is able to reduce the RSS memory from 12MB to 6MB.
Signed-off-by: Yonghong Song <yhs@fb.com>

src/cc/bpf_module.cc

@@ -140,6 +140,11 @@ BPFModule::~BPFModule() {
     v->leaf_snprintf = unimplemented_snprintf;
   }
 
+  if (!rw_engine_enabled_) {
+    for (auto section : sections_)
+      delete get<0>(section.second);
+  }
+
   engine_.reset();
   rw_engine_.reset();
   ctx_.reset();
@@ -607,14 +612,17 @@ int BPFModule::run_pass_manager(Module &mod) {
 
 int BPFModule::finalize() {
   Module *mod = &*mod_;
+  std::map<std::string, std::tuple<uint8_t *, uintptr_t>> tmp_sections,
+      *sections_p;
 
   mod->setDataLayout("e-m:e-p:64:64-i64:64-n32:64-S128");
   mod->setTargetTriple("bpf-pc-linux");
+  sections_p = rw_engine_enabled_ ? &sections_ : &tmp_sections;
 
   string err;
   EngineBuilder builder(move(mod_));
   builder.setErrorStr(&err);
-  builder.setMCJITMemoryManager(ebpf::make_unique<MyMemoryManager>(&sections_));
+  builder.setMCJITMemoryManager(ebpf::make_unique<MyMemoryManager>(sections_p));
   builder.setMArch("bpf");
   builder.setUseOrcMCJITReplacement(false);
   engine_ = unique_ptr<ExecutionEngine>(builder.create());
@@ -631,17 +639,35 @@ int BPFModule::finalize() {
 
   engine_->finalizeObject();
 
-  // give functions an id
-  for (auto section : sections_)
-    if (!strncmp(FN_PREFIX.c_str(), section.first.c_str(), FN_PREFIX.size()))
-      function_names_.push_back(section.first);
-
   if (flags_ & DEBUG_SOURCE) {
-    SourceDebugger src_debugger(mod, sections_, FN_PREFIX, mod_src_,
+    SourceDebugger src_debugger(mod, *sections_p, FN_PREFIX, mod_src_,
                                 src_dbg_fmap_);
     src_debugger.dump();
   }
 
+  if (!rw_engine_enabled_) {
+    // Setup sections_ correctly and then free llvm internal memory
+    for (auto section : tmp_sections) {
+      auto fname = section.first;
+      uintptr_t size = get<1>(section.second);
+      uint8_t *tmp_p = NULL;
+      // Only copy data for non-map sections
+      if (strncmp("maps/", section.first.c_str(), 5)) {
+        uint8_t *addr = get<0>(section.second);
+        tmp_p = new uint8_t[size];
+        memcpy(tmp_p, addr, size);
+      }
+      sections_[fname] = make_tuple(tmp_p, size);
+    }
+    engine_.reset();
+    ctx_.reset();
+  }
+
+  // give functions an id
+  for (auto section : sections_)
+    if (!strncmp(FN_PREFIX.c_str(), section.first.c_str(), FN_PREFIX.size()))
+      function_names_.push_back(section.first);
+
   return 0;
 }