Commit 97858a5b authored by Kevin Modzelewski's avatar Kevin Modzelewski

Merge commit 'b2d7f222' into refcounting

parents 7eb225b1 b2d7f222
......@@ -8,6 +8,10 @@ include(ExternalProject)
set(DEPS_DIR $ENV{HOME}/pyston_deps)
if (NOT CMAKE_SIZEOF_VOID_P EQUAL 8)
message(FATAL_ERROR "Pyston does not support 32-bit systems yet")
endif()
# set build type to release by default
set(CMAKE_CONFIGURATION_TYPES "Debug;Release" CACHE STRING "" FORCE)
if(NOT CMAKE_BUILD_TYPE)
......
......@@ -240,16 +240,64 @@ void initCodegen() {
Integration tests that run `pip` will often cache downloaded packages, so you may want to try `rm -rf ~/.cache/pip`. We do not clear the cache by default because the test takes longer to run and we run into cases where the failure can occur only when packages are cached just as often as when the packages are not cached.
##### Problem: I found the crash, but the stacktrace is convoluted
##### Problem: I have a segmentation fault (SIGSEGV) or a bus error (SIGBUS) and I don't understand how it happens, GDB is crashing
Most segmentation faults are obvious (e.g. deferencing a NULL pointer) but sometimes, it is not clear how it happens. For example, accessing a field without a pointer dereference (e.g. `v.x`).
Check how large the backtrace is. It's possible that the segumentation fault or bus error is actually a stack overflow, in which case the crash could happen just about anywhere. Furthermore, GDB seems to handle stack traces badly (attempting to print a variable might crash GDB).
Also consider that the stack being run could be a generator stack, in which case local variables will have address starting with `0x427`. They look like heap objects, but they are not.
##### Problem: I found the crash, but I do not understand the stacktrace (it is convoluted or seems wrong)
The real cause of an error is often deeper down in the stack. If an assertion failed for example, it may be the case that the assertion calls an exception handler, which tries unwinding the stack for debug information, then fails again and triggers another assertion, which is the one you see. If you see anything that refers to unwinding or exception handling in the stacktrace, you may need to look deeper down.
It is also possible that an exception was not handled and unwinding fails. The error message might contain references to unwinding. For example:
```
Traceback (most recent call last):
File "/usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19-gdb.py", line 63, in <module from libstdcxx.v6.printers import register_libstdcxx_printers
ImportError: No module named 'libstdcxx'
Pyston v0.5.0 (rev dfd0b2eadb3723e5898419ade3c535c1e2f7ecf9), targeting Python 2.7.7
>> float(1<<1024)
../../src/runtime/cxx_unwind.cpp:612: void std::terminate(): Assertion `0' failed: std::terminate() called!
Someone called abort!
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
```
In that case, the backtrace might not be very helpful. For example:
```
#0 0x00007ffff5189cc9 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007ffff518d0d8 in __GI_abort () at abort.c:89
#2 0x00000000007dd05c in abort () at ../../src/codegen/unwinding.cpp:1222
#3 0x000000000087a43a in std::terminate () at ../../src/runtime/cxx_unwind.cpp:612
#4 0x00000000006285a6 in __clang_call_terminate ()
#5 0x00000000008992a1 in pyston::_floatNew<(pyston::ExceptionStyle)0> (a=0x127018ad68)
at ../../src/runtime/float.cpp:852
#6 0x00000000008986db in pyston::floatNew<(pyston::ExceptionStyle)0> (_cls=0x127000b808, a=0x127018ad68)
at ../../src/runtime/float.cpp:920
(More stack frames follow...)
(gdb) frame 5
#5 0x00000000008992a1 in pyston::_floatNew<(pyston::ExceptionStyle)0> (a=0x127018ad68)
at ../../src/runtime/float.cpp:852
852 return new BoxedFloat(static_cast<BoxedInt*>(a)->n);
```
The backtrace is misleading since it suggests that an error occured on line 852, which is entirely wrong (that line is never actually reached!).
If that happens, you should either add a try/catch pair, or use C-style exceptions instead.
##### Problem: I get a segmentation due to stackoverflow from an infinite loop of mutually recursive functions like `runtimeCall`.
Pyston started without support for [CPython slots](https://docs.python.org/2/c-api/typeobj.html#sequence-structs) (e.g. `tp_` or `mp_`, `sq_`, etc) but they were introduced later. Some legacy code will still assign functions to the `tp_` slots that do an attribute lookup for a function that contains the logic instead of just running the logic directly. For example, `slot_tp_del` might do an attribute for `__del__` instead of just calling a finalizer.
Pyston started without support for [CPython slots](https://docs.python.org/2/c-api/typeobj.html#sequence-structs) (e.g. `tp_` or `mp_`, `sq_`, etc) but they were introduced later. Some legacy code will still assign functions to the `tp_` slots that do an attribute lookup for a function that contains the logic instead of just running the logic directly. For example, `slot_tp_del` might do an attribute lookup for `__del__` instead of just calling a finalizer.
Sometimes, that attribute lookup ends up calling the `tp_` slot again which creates a loop which only terminates with a stackoverflow. The solution in those cases is often to figure out the object and the slot involved in the recursive calls, and assign a non-default function to the `tp_` slot that does not do another attribute lookup.
https://github.com/dropbox/pyston/commit/79b2e9ccd835fb2cfeb47756a92295d4d17ef91b is a commit that fixed one instance of this problem.
##### Problem: My object is not getting garbage collected
This mostly concerns code or tests that involve finalizers and weak reference callbacks in some way.
......
From fda9fd463dae38840cc02c5ae668cb1097395b9a Mon Sep 17 00:00:00 2001
From: Benjamin Kramer <benny.kra@googlemail.com>
Date: Fri, 1 May 2015 15:16:11 +0000
Subject: [PATCH] Remove std::move on return when it could prevent copy
elision.
Found by -Wpessimizing-move, no functional change. The APFloat and
PassManager change doesn't affect codegen as returning a by-value
argument will always result in a move.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@236316 91177308-0d34-0410-b5e6-96231b3b80d8
---
include/llvm/ADT/APFloat.h | 2 +-
include/llvm/IR/PassManager.h | 4 ++--
utils/yaml-bench/YAMLBench.cpp | 2 +-
3 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/include/llvm/ADT/APFloat.h b/include/llvm/ADT/APFloat.h
index 53b53c5..958e3fd 100644
--- a/include/llvm/ADT/APFloat.h
+++ b/include/llvm/ADT/APFloat.h
@@ -343,7 +343,7 @@ class APFloat {
/// copied from some other APFloat.
static APFloat copySign(APFloat Value, const APFloat &Sign) {
Value.copySign(Sign);
- return std::move(Value);
+ return Value;
}
/// @}
diff --git a/include/llvm/IR/PassManager.h b/include/llvm/IR/PassManager.h
index 3c24e72..b566f01 100644
--- a/include/llvm/IR/PassManager.h
+++ b/include/llvm/IR/PassManager.h
@@ -509,7 +509,7 @@ class AnalysisManager
PreservedAnalyses invalidateImpl(IRUnitT &IR, PreservedAnalyses PA) {
// Short circuit for a common case of all analyses being preserved.
if (PA.areAllPreserved())
- return std::move(PA);
+ return PA;
if (DebugLogging)
dbgs() << "Invalidating all non-preserved analyses for: "
@@ -549,7 +549,7 @@ class AnalysisManager
if (ResultsList.empty())
AnalysisResultLists.erase(&IR);
- return std::move(PA);
+ return PA;
}
/// \brief List of function analysis pass IDs and associated concept pointers.
diff --git a/utils/yaml-bench/YAMLBench.cpp b/utils/yaml-bench/YAMLBench.cpp
index 872f586..0fb3138 100644
--- a/utils/yaml-bench/YAMLBench.cpp
+++ b/utils/yaml-bench/YAMLBench.cpp
@@ -69,7 +69,7 @@ static std::string prettyTag(yaml::Node *N) {
if (StringRef(Tag).startswith("tag:yaml.org,2002:")) {
std::string Ret = "!!";
Ret += StringRef(Tag).substr(18);
- return std::move(Ret);
+ return Ret;
}
std::string Ret = "!<";
Ret += Tag;
# To try to make runs with different n as similar as possible, we always
# create an array of N elements by repeating the n as many times as necessary.
# This way, the control flow of the main loop will be the same regardless of
# the number of objects we create -- with such a small benchmark, this ends
# up being noticeable.
N = 1024
def f(n):
l = []
assert N % n == 0
......
# To try to make runs with different n as similar as possible, we always
# create an array of N elements by repeating the n as many times as necessary.
# This way, the control flow of the main loop will be the same regardless of
# the number of objects we create -- with such a small benchmark, this ends
# up being noticeable.
M = 64
N = 64
def f(m, n):
assert N % n == 0
assert M % m == 0
......
# To try to make runs with different n as similar as possible, we always
# create an array of N elements by repeating the n as many times as necessary.
# This way, the control flow of the main loop will be the same regardless of
# the number of objects we create -- with such a small benchmark, this ends
# up being noticeable.
N = 1024
def f(n):
l = []
assert N % n == 0
......
......@@ -80,7 +80,7 @@ void Assembler::emitArith(Immediate imm, Register r, int opcode) {
// assert(r != RSP && "This breaks unwinding, please don't use.");
int64_t amount = imm.val;
RELEASE_ASSERT((-1L << 31) <= amount && amount < (1L << 31) - 1, "");
RELEASE_ASSERT(fitsInto<int32_t>(amount), "");
assert(0 <= opcode && opcode < 8);
int rex = REX_W;
......@@ -183,7 +183,7 @@ void Assembler::mov(Immediate val, Register dest, bool force_64bit_load) {
void Assembler::movq(Immediate src, Indirect dest) {
int64_t src_val = src.val;
assert((-1L << 31) <= src_val && src_val < (1L << 31) - 1);
assert(fitsInto<int32_t>(src_val));
int rex = REX_W;
......@@ -828,7 +828,7 @@ void Assembler::cmp(Register reg, Immediate imm) {
void Assembler::cmp(Indirect mem, Immediate imm) {
int64_t val = imm.val;
assert((-1L << 31) <= val && val < (1L << 31) - 1);
assert(fitsInto<int32_t>(val));
int src_idx = mem.base.regnum;
......@@ -849,7 +849,7 @@ void Assembler::cmp(Indirect mem, Immediate imm) {
emitModRM(0b01, 7, src_idx);
emitByte(mem.offset);
} else {
assert((-1L << 31) <= mem.offset && mem.offset < (1L << 31) - 1);
assert(fitsInto<int32_t>(mem.offset));
emitModRM(0b10, 7, src_idx);
emitInt(mem.offset, 4);
}
......@@ -883,7 +883,7 @@ void Assembler::cmp(Indirect mem, Register reg) {
emitModRM(0b01, reg_idx, mem_idx);
emitByte(mem.offset);
} else {
assert((-1L << 31) <= mem.offset && mem.offset < (1L << 31) - 1);
assert(fitsInto<int32_t>(mem.offset));
emitModRM(0b10, reg_idx, mem_idx);
emitInt(mem.offset, 4);
}
......@@ -919,7 +919,7 @@ void Assembler::lea(Indirect mem, Register reg) {
if (mode == 0b01) {
emitByte(mem.offset);
} else if (mode == 0b10) {
assert((-1L << 31) <= mem.offset && mem.offset < (1L << 31) - 1);
assert(fitsInto<int32_t>(mem.offset));
emitInt(mem.offset, 4);
}
}
......@@ -999,7 +999,7 @@ void Assembler::jmp(Indirect dest) {
emitModRM(0b01, 0b100, reg_idx);
emitByte(dest.offset);
} else {
assert((-1L << 31) <= dest.offset && dest.offset < (1L << 31) - 1);
assert(fitsInto<int32_t>(dest.offset));
emitModRM(0b10, 0b100, reg_idx);
emitInt(dest.offset, 4);
}
......
......@@ -666,7 +666,7 @@ public:
static Rewriter* createRewriter(void* rtn_addr, int num_args, const char* debug_name);
static bool isLargeConstant(int64_t val) { return (val < (-1L << 31) || val >= (1L << 31) - 1); }
static bool isLargeConstant(int64_t val) { return !fitsInto<int32_t>(val); }
// The "aggressiveness" with which we should try to do this rewrite. It starts high, and decreases over time.
// The values are nominally in the range 0-100, with 0 being no aggressiveness and 100 being fully aggressive,
......
......@@ -172,9 +172,7 @@ struct JumpDestination {
int offset;
JumpDestination(OffsetType type, int64_t offset) : type(type), offset(offset) {
assert((-1L << 31) <= offset && offset < (1L << 31) - 1);
}
JumpDestination(OffsetType type, int64_t offset) : type(type), offset(offset) { assert(fitsInto<int32_t>(offset)); }
static JumpDestination fromStart(int offset) { return JumpDestination(FROM_START, offset); }
};
}
......
......@@ -2443,7 +2443,7 @@ public:
for (int i = 0; i < orig_elts.size(); i++) {
elts.push_back(orig_elts[i]->dup(cache));
}
return std::move(elts);
return elts;
}
ConcreteCompilerVariable* _makeConverted(IREmitter& emitter, const VEC& v, ConcreteCompilerType* other_type) {
......
......@@ -1200,7 +1200,7 @@ static std::vector<char> _reparse(const char* fn, const std::string& cache_fn, A
fwrite(&checksum, 1, CHECKSUM_LENGTH, cache_fp);
memcpy(&file_data[checksum_start + LENGTH_LENGTH], &checksum, CHECKSUM_LENGTH);
return std::move(file_data);
return file_data;
}
// Parsing the file is somewhat expensive since we have to shell out to cpython;
......
......@@ -19,6 +19,7 @@
#include <csignal>
#include <cstdio>
#include <cstdlib>
#include <limits>
#include <stdint.h>
#include <string>
#include <unordered_map>
......@@ -82,4 +83,11 @@ template <typename T1, typename T2, typename T3> struct hash<tuple<T1, T2, T3>>
};
}
namespace pyston {
template <typename T>
constexpr bool fitsInto(int64_t x) {
return std::numeric_limits<T>::min() <= x && x <= std::numeric_limits<T>::max();
}
}
#endif
The tests in this directory are unusual in that they don't contain typical testing assertions.
This comes from the time that Pyston didn't support exceptions or assertions: they only way
we had of checking our behavior was to compare the output. So these tests get checked by
running them under both Pyston and CPython and making sure that the output is the same.
(There are small differences allowed -- see tools/tester.py for more details.)
Now that we support proper assertions and exceptions, it can be easier to use assertions rather
than using stdout to show the behavior. But it can still be nice to use the output-checking,
for example for tricky bits of logic that would be just as error-prone to recode in a test.
For example, for slice-index handling, we exhaustively test all sets of indices and ensure
that we get the same slice as CPython.
One last feature is that it's possible to write out a manual foo.expected file. Then when the test
foo.py gets run, instead of running CPython and checking the output, foo.expected is used as the
expected output. This is less useful these days but it can still be handy.
......@@ -92,6 +92,8 @@ if __name__ == "__main__":
continue
if "Expose-getSymbolLoadAddress" in patch_fn and svn_rev <= 222840:
continue
if "Remove-move-warning" in patch_fn and svn_rev >= 236316:
continue
patch_fn = os.path.abspath(os.path.join(patch_dir, patch_fn))
code = subprocess.call(["git", "am", patch_fn], cwd=repo)
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment