Make dependency on LLVM native target optional (#2080)

* Make dependency on LLVM native target optional

Adds an option ENABLE_LLVM_NATIVECODEGEN with default value ON.
If set to off the "nativecodegen" llvm will not be enabled, thus
reducing dependencies on needed libraries (reduced text size when
building with statically linked libraries).

Code that uses native target will not be compiled reducing text size.
Currently this affects the rw_engine which needs the native target.

BPF api "rw_engine_enabled" will have default value "true" if
ENABLE_LLVM_NATIVECODEGEN="ON" and "false" if
ENABLE_LLVM_NATIVECODEGEN="OFF"

Not needed for BCC to work. It somehow brought in the interpreter and
executionengine which is needed. Those features are added instead.

* Remove garbage in code making it compile again

* Remove interpreter and executionengine LLVM dependencies

These doesn't seem to be needed on a Ubuntu 18.04 system (although
executionengine is heavily used).

Interpreter was added due to runtime dependency on ARM64. It brings in
a dependency on ffi library.

(.text._ZL10ffiTypeForPN4llvm4TypeE+0x3a): undefined reference to `ffi_type_float'
(.text._ZL10ffiTypeForPN4llvm4TypeE+0x43): undefined reference to `ffi_type_void'
(.text._ZL10ffiTypeForPN4llvm4TypeE+0x53): undefined reference to `ffi_type_pointer'
(.text._ZL10ffiTypeForPN4llvm4TypeE+0x63): undefined reference to `ffi_type_double'
(.text._ZL10ffiTypeForPN4llvm4TypeE+0x78): undefined reference to `ffi_type_sint8'
(.text._ZL10ffiTypeForPN4llvm4TypeE+0x83): undefined reference to `ffi_type_sint16'
(.text._ZL10ffiTypeForPN4llvm4TypeE+0x93): undefined reference to `ffi_type_sint64'
(.text._ZL10ffiTypeForPN4llvm4TypeE+0xb3): undefined reference to `ffi_type_sint32'
/usr/lib/llvm-6.0/lib/libLLVMInterpreter.a

CMakeLists.txt

@@ -16,6 +16,7 @@ include(GNUInstallDirs)
 include(CheckCXXCompilerFlag)
 include(cmake/FindCompilerFlag.cmake)
 
+option(ENABLE_LLVM_NATIVECODEGEN "Enable use of llvm nativecodegen module (needed by rw-engine)" ON)
 option(ENABLE_RTTI "Enable compiling with real time type information" OFF)
 option(ENABLE_LLVM_SHARED "Enable linking LLVM as a shared library" OFF)
 option(ENABLE_CLANG_JIT "Enable Loading BPF through Clang Frontend" ON)

cmake/clang_libs.cmake

@@ -2,7 +2,10 @@ if(ENABLE_LLVM_SHARED)
 set(llvm_libs "LLVM")
 else()
 set(llvm_raw_libs bitwriter bpfcodegen debuginfodwarf irreader linker
-  mcjit objcarcopts option passes nativecodegen lto)
+  mcjit objcarcopts option passes lto)
+if(ENABLE_LLVM_NATIVECODEGEN)
+set(llvm_raw_libs ${llvm_raw_libs} nativecodegen)
+endif()
 list(FIND LLVM_AVAILABLE_LIBS "LLVMCoverage" _llvm_coverage)
 if (${_llvm_coverage} GREATER -1)
   list(APPEND llvm_raw_libs coverage)

src/cc/CMakeLists.txt

@@ -34,6 +34,12 @@ if (${LLVM_PACKAGE_VERSION} VERSION_EQUAL 6 OR ${LLVM_PACKAGE_VERSION} VERSION_G
   set(bcc_common_sources ${bcc_common_sources} bcc_debug.cc)
 endif()
 
+if(ENABLE_LLVM_NATIVECODEGEN)
+set(bcc_common_sources ${bcc_common_sources} bpf_module_rw_engine.cc)
+else()
+set(bcc_common_sources ${bcc_common_sources} bpf_module_rw_engine_disabled.cc)
+endif()
+
 set(bcc_table_sources table_storage.cc shared_table.cc bpffs_table.cc json_map_decl_visitor.cc)
 set(bcc_util_sources ns_guard.cc common.cc)
 set(bcc_sym_sources bcc_syms.cc bcc_elf.c bcc_perf_map.c bcc_proc.c)

src/cc/api/BPF.h

@@ -47,7 +47,7 @@ class BPF {
   static const int BPF_MAX_STACK_DEPTH = 127;
 
   explicit BPF(unsigned int flag = 0, TableStorage* ts = nullptr,
-               bool rw_engine_enabled = true, const std::string &maps_ns = "")
+               bool rw_engine_enabled = bpf_module_rw_engine_enabled(), const std::string &maps_ns = "")
       : flag_(flag),
       bpf_module_(new BPFModule(flag, ts, rw_engine_enabled, maps_ns)) {}
   StatusTuple init(const std::string& bpf_program,

src/cc/bpf_module.h

@@ -54,10 +54,14 @@ class BLoader;
 class ClangLoader;
 class FuncSource;
 
+bool bpf_module_rw_engine_enabled(void);
+
 class BPFModule {
  private:
   static const std::string FN_PREFIX;
   int init_engine();
+  void initialize_rw_engine();
+  void cleanup_rw_engine();
   int parse(llvm::Module *mod);
   int finalize();
   int annotate();

src/cc/bpf_module_rw_engine_disabled.cc

+/*
+ * Copyright (c) 2015 PLUMgrid, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "bpf_module.h"
+
+namespace ebpf {
+
+bool bpf_module_rw_engine_enabled(void) {
+  return false;
+}
+
+void BPFModule::initialize_rw_engine() {
+}
+
+void BPFModule::cleanup_rw_engine() {
+}
+
+int BPFModule::annotate() {
+  return -1;
+}
+
+} // namespace ebpf