GitLab

In the past badsignal would crash the program.  In
https://golang.org/cl/10757044 badsignal was changed to call sigsend,
to fix issue #3250.  The effect of this was that when a non-Go thread
received a signal, and os/signal.Notify was not being used to check
for occurrences of the signal, the signal was ignored.

This changes the code so that if os/signal.Notify is not being used,
then the signal handler is reset to what it was, and the signal is
raised again.  This lets non-Go threads handle the signal as they
wish.  In particular, it means that a segmentation violation in a
non-Go thread will ordinarily crash the process, as it should.

Fixes #10139.
Update #11794.

Change-Id: I2109444aaada9d963ad03b1d071ec667760515e5
Reviewed-on: https://go-review.googlesource.com/12503

Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>

Avoids shadowing the builtin channel close function.

Change-Id: I7a729b0937c8248fe27222be61318a88db995eee
Reviewed-on: https://go-review.googlesource.com/8898

Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: David Crawshaw <crawshaw@golang.org>

For OSes that use elf on intel, 2*Ptrsize bytes are reserved for TLS.
But only one pointer (g) has been stored in the TLS for a while now.
So we can set it to just Ptrsize, which happily matches what happens
when externally linking.

Fixes #9913

Change-Id: Ic816369d3a55a8cdcc23be349b1a1791d53f5f81
Reviewed-on: https://go-review.googlesource.com/6584


Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Error detection code copied from syscall, where presumably
we actually do it right.

Note that we throw the errno away.  The runtime doesn't use it.

Fixes #10052

Change-Id: I8de77dda6bf287276b137646c26b84fa61554ec8
Reviewed-on: https://go-review.googlesource.com/6571


Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Any place that clobbers BP in the runtime can potentially interfere
with frame pointer unwinding with GOEXPERIMENT=framepointer.  This
change eliminates uses of BP in the runtime to address this problem.
We have spare registers everywhere this occurs, so there's no downside
to eliminating BP.  Where possible, this uses the same new register as
the amd64p32 runtime, which doesn't use BP due to restrictions placed
on it by NaCL.

One nice side effect of this is that it will let perf/VTune unwind the
call stack even through a call to systemstack, which will let us get
really good call graphs from the garbage collector.

Change-Id: I0ffa14cb4dd2b613a7049b8ec59df37c52286212
Reviewed-on: https://go-review.googlesource.com/3390

Reviewed-by: Minux Ma <minux@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

This manually reverts 555da73c from #6372 which implies a
minimum FreeBSD version of 8-STABLE.
Updates docs to mention new minimum requirement.

Fixes #9627

Change-Id: I40ae64be3682d79dd55024e32581e3e5e2be8aa7
Reviewed-on: https://go-review.googlesource.com/3020

Reviewed-by: Minux Ma <minux@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

The main change is that #include "zasm_GOOS_GOARCH.h"
is now #include "go_asm.h" and/or #include "go_tls.h".

Also, because C StackGuard is now Go _StackGuard,
the assembly name changes from const_StackGuard to
const__StackGuard.

In asm_$GOARCH.s, add new function getg, formerly
implemented in C.

The renamed atomics now have Go wrappers, to get
escape analysis annotations right. Those wrappers
are in CL 174860043.

LGTM=r, aram
R=r, aram
CC=austin, dvyukov, golang-codereviews, iant, khr
https://golang.org/cl/168510043

Preparation was in CL 134570043.
This CL contains only the effect of 'hg mv src/pkg/* src'.
For more about the move, see golang.org/s/go14nopkg.

I did this just to clean things up, but it will be important
when we drop the pkg directory later.

LGTM=bradfitz
R=r, bradfitz
CC=golang-codereviews
https://golang.org/cl/132600043

To date, the C compilers and Go compilers differed only in how
values were returned from functions. This made it difficult to call
Go from C or C from Go if return values were involved. It also made
assembly called from Go and assembly called from C different.

This CL changes the C compiler to use the Go conventions, passing
results on the stack, after the arguments.
[Exception: this does not apply to C ... functions, because you can't
know where on the stack the arguments end.]

By doing this, the CL makes it possible to rewrite C functions into Go
one at a time, without worrying about which languages call that
function or which languages it calls.

This CL also updates all the assembly files in package runtime to use
the new conventions. Argument references of the form 40(SP) have
been rewritten to the form name+10(FP) instead, and there are now
Go func prototypes for every assembly function called from C or Go.
This means that 'go vet runtime' checks effectively every assembly
function, and go vet's output was used to automate the bulk of the
conversion.

Some functions, like seek and nsec on Plan 9, needed to be rewritten.

Many assembly routines called from C were reading arguments
incorrectly, using MOVL instead of MOVQ or vice versa, especially on
the less used systems like openbsd.
These were found by go vet and have been corrected too.
If we're lucky, this may reduce flakiness on those systems.

Tested on:
        darwin/386
        darwin/amd64
        linux/arm
        linux/386
        linux/amd64
If this breaks another system, the bug is almost certainly in the
sys_$GOOS_$GOARCH.s file, since the rest of the CL is tested
by the combination of the above systems.

LGTM=dvyukov, iant
R=golang-codereviews, 0intro, dave, alex.brainman, dvyukov, iant
CC=golang-codereviews, josharian, r
https://golang.org/cl/135830043

The runtime has historically held two dedicated values g (current goroutine)
and m (current thread) in 'extern register' slots (TLS on x86, real registers
backed by TLS on ARM).

This CL removes the extern register m; code now uses g->m.

On ARM, this frees up the register that formerly held m (R9).
This is important for NaCl, because NaCl ARM code cannot use R9 at all.

The Go 1 macrobenchmarks (those with per-op times >= 10 µs) are unaffected:

BenchmarkBinaryTree17              5491374955     5471024381     -0.37%
BenchmarkFannkuch11                4357101311     4275174828     -1.88%
BenchmarkGobDecode                 11029957       11364184       +3.03%
BenchmarkGobEncode                 6852205        6784822        -0.98%
BenchmarkGzip                      650795967      650152275      -0.10%
BenchmarkGunzip                    140962363      141041670      +0.06%
BenchmarkHTTPClientServer          71581          73081          +2.10%
BenchmarkJSONEncode                31928079       31913356       -0.05%
BenchmarkJSONDecode                117470065      113689916      -3.22%
BenchmarkMandelbrot200             6008923        5998712        -0.17%
BenchmarkGoParse                   6310917        6327487        +0.26%
BenchmarkRegexpMatchMedium_1K      114568         114763         +0.17%
BenchmarkRegexpMatchHard_1K        168977         169244         +0.16%
BenchmarkRevcomp                   935294971      914060918      -2.27%
BenchmarkTemplate                  145917123      148186096      +1.55%

Minux previous reported larger variations, but these were caused by
run-to-run noise, not repeatable slowdowns.

Actual code changes by Minux.
I only did the docs and the benchmarking.

LGTM=dvyukov, iant, minux
R=minux, josharian, iant, dave, bradfitz, dvyukov
CC=golang-codereviews
https://golang.org/cl/109050043

For now we don't use CLOCK_MONOTONIC_FAST instead because
it's not supported on prior to 9-STABLE.

Update #6007

LGTM=minux.ma
R=golang-codereviews, minux.ma, bradfitz
CC=golang-codereviews
https://golang.org/cl/68690043

TBR=golang-dev
CC=golang-dev
https://golang.org/cl/13720044

The kernel implementation of the fast system call path,
the one invoked by the SYSCALL instruction, is broken for
restarting system calls. A C program demonstrating this is below.

Change the system calls to use INT $0x80 instead, because
that (perhaps slightly slower) system call path actually works.

I filed http://www.freebsd.org/cgi/query-pr.cgi?pr=182161.

The C program demonstrating that it is FreeBSD's fault is below.
It reports the same "Bad address" failures from wait.

#include <sys/time.h>
#include <sys/signal.h>
#include <pthread.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>

static void handler(int);
static void* looper(void*);

int
main(void)
{
        int i;
        struct sigaction sa;
        pthread_cond_t cond;
        pthread_mutex_t mu;

        memset(&sa, 0, sizeof sa);
        sa.sa_handler = handler;
        sa.sa_flags = SA_RESTART;
        memset(&sa.sa_mask, 0xff, sizeof sa.sa_mask);
        sigaction(SIGCHLD, &sa, 0);

        for(i=0; i<2; i++)
                pthread_create(0, 0, looper, 0);

        pthread_mutex_init(&mu, 0);
        pthread_mutex_lock(&mu);
        pthread_cond_init(&cond, 0);
        for(;;)
                pthread_cond_wait(&cond, &mu);

        return 0;
}

static void
handler(int sig)
{
}

int
mywait4(int pid, int *stat, int options, struct rusage *rusage)
{
        int result;

        asm("movq %%rcx, %%r10; syscall"
                : "=a" (result)
                : "a" (7),
                  "D" (pid),
                  "S" (stat),
                  "d" (options),
                  "c" (rusage));
}

static void*
looper(void *v)
{
        int pid, stat, out;
        struct rusage rusage;

        for(;;) {
                if((pid = fork()) == 0)
                        _exit(0);
                out = mywait4(pid, &stat, 0, &rusage);
                if(out != pid) {
                        printf("wait4 returned %d\n", out);
                }
        }
}

Fixes #6372.

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/13582047

Remove NOPROF/DUPOK from everything.

Edits done with a script, except pclinetest.asm which depended
on the DUPOK flag on main().

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/12613044

Fixes #3250.

R=rsc
CC=golang-dev
https://golang.org/cl/10757044

Update #5199

R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/8825043

This provides a way to generate core dumps when people need them.
The settings are:

        GOTRACEBACK=0  no traceback on panic, just exit
        GOTRACEBACK=1  default - traceback on panic, then exit
        GOTRACEBACK=2  traceback including runtime frames on panic, then exit
        GOTRACEBACK=crash traceback including runtime frames on panic, then crash

Fixes #3257.

R=golang-dev, devon.odell, r, daniel.morsing, ality
CC=golang-dev
https://golang.org/cl/7666044

NEGL does a negation of the bottom 32 bits and then zero-extends to 64 bits,
resulting in a negative 32-bit number but a positive 64-bit number.

NEGQ does a full 64-bit negation, so that the result is negative both as
a 32-bit and as a 64-bit number.

This doesn't matter for the functions that are declared to return int32.
It only matters for the ones that return int64 or void* [sic].

This will fix the current incorrect error in the OpenBSD/amd64 build.
The build will still be broken, but it won't report a bogus error.

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/7536046

Uses AES hardware instructions on 386/amd64 to implement
a fast hash function.  Incorporates a random key to
thwart hash collision DOS attacks.
Depends on CL#7548043 for new assembly instructions.

Update #3885
Helps some by making hashing faster.  Go time drops from
0.65s to 0.51s.

R=rsc, r, bradfitz, remyoudompheng, khr, dsymonds, minux.ma, elias.naur
CC=golang-dev
https://golang.org/cl/7543043

Putting the M initialization in multiple places will not scale.
Various code assumes mstart is the start already. Make it so.

R=golang-dev, devon.odell
CC=golang-dev
https://golang.org/cl/7420048

Move the mstartfn into its own field.
Simpler, more likely to be correct.

R=golang-dev, devon.odell
CC=golang-dev
https://golang.org/cl/7414046

R=devon.odell
CC=golang-dev
https://golang.org/cl/7443046

When we release memory to the OS, if the OS doesn't want us
to release it (for example, because the program executed
mlockall(MCL_FUTURE)), madvise will fail. Ignore the failure
instead of crashing.

Fixes #3435.

R=ken2
CC=golang-dev
https://golang.org/cl/6998052

For Linux/{386,arm}, FreeBSD/{386,amd64,arm}, NetBSD/{386,amd64}, OpenBSD/{386,amd64}.
Note: our Darwin implementation already has ns resolution.

Linux/386 (Core i7-2600 @ 3.40GHz, kernel 3.5.2-gentoo)
benchmark       old ns/op    new ns/op    delta
BenchmarkNow          110          118   +7.27%

Linux/ARM (ARM Cortex-A8 @ 800MHz, kernel 2.6.32.28 android)
benchmark       old ns/op    new ns/op    delta
BenchmarkNow          625          542  -13.28%

Linux/ARM (ARM Cortex-A9 @ 1GHz, Pandaboard)
benchmark       old ns/op    new ns/op    delta
BenchmarkNow          992          909   -8.37%

FreeBSD 9-REL-p1/amd64 (Dell R610 Server with Xeon X5650 @ 2.67GHz)
benchmark       old ns/op    new ns/op    delta
BenchmarkNow          699          695   -0.57%

FreeBSD 9-REL-p1/amd64 (Atom D525 @ 1.80GHz)
benchmark       old ns/op    new ns/op    delta
BenchmarkNow         1553         1658   +6.76%

OpenBSD/amd64 (Dell E6410 with i5 CPU M 540 @ 2.53GHz)
benchmark       old ns/op    new ns/op    delta
BenchmarkNow         1262         1236   -2.06%

OpenBSD/i386 (Asus eeePC 701 with Intel Celeron M 900MHz - locked to 631MHz)
benchmark       old ns/op    new ns/op    delta
BenchmarkNow         5089         5043   -0.90%

NetBSD/i386 (VMware VM with Core i5 CPU @ 2.7GHz)
benchmark       old ns/op    new ns/op    delta
BenchmarkNow          277          278   +0.36%

NetBSD/amd64 (VMware VM with Core i5 CPU @ 2.7Ghz)
benchmark       old ns/op    new ns/op    delta
BenchmarkNow          103          105   +1.94%

Thanks Maxim Khitrov, Joel Sing, and Dave Cheney for providing benchmark data.

R=jsing, dave, rsc
CC=golang-dev
https://golang.org/cl/6820120

Make use of carry clear condition instead of low or same.

R=minux.ma, jsing, dave
CC=golang-dev
https://golang.org/cl/6844080

madvise was missing so implement it in assembler. This change
needs to be extended to the other BSD variantes (Net and Open)

Without this change the scavenger will attempt to pass memory back
to the operating system when it has become idle, but the memory is
not returned and for long running Go processes the total memory used
can grow until OOM occurs.

I have only been able to test the code on FreeBSD AMD64. The ARM
platforms needs testing.

R=golang-dev, mikioh.mikioh, dave, jgc, minux.ma
CC=golang-dev
https://golang.org/cl/6850081

Signal handlers are global resources but many language
environments (Go, C++ at Google, etc) assume they have sole
ownership of a particular handler.  Signal handlers in
mixed-language applications must therefore be robust against
unexpected delivery of certain signals, such as SIGPROF.

The default Go signal handler runtime·sigtramp assumes that it
will never be called on a non-Go thread, but this assumption
is violated by when linking in C++ code that spawns threads.
Specifically, the handler asserts the thread has an associated
"m" (Go scheduler).

This CL is a very simple workaround: discard SIGPROF delivered to non-Go threads.  runtime.badsignal(int32) now receives the signal number; if it returns without panicking (e.g. sig==SIGPROF) the signal is discarded.

I don't think there is any really satisfactory solution to the
problem of signal-based profiling in a mixed-language
application.  It's not only the issue of handler clobbering,
but also that a C++ SIGPROF handler called in a Go thread
can't unwind the Go stack (and vice versa).  The best we can
hope for is not crashing.

Note:
- I've ported this to all POSIX platforms, except ARM-linux which already ignores unexpected signals on m-less threads.
- I've avoided tail-calling runtime.badsignal because AFAICT the 6a/6l don't support it.
- I've avoided hoisting 'push sig' (common to both function calls) because it makes the code harder to read.
- Fixed an (apparently incorrect?) docstring.

R=iant, rsc, minux.ma
CC=golang-dev
https://golang.org/cl/6498057

It's the best we can do before Go 1.

For issue 3250; not a fix but at least less mysterious.

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/5797068

When a very low-level system call that should never fail
does fail, we call notok, which crashes the program.
Often, we are then left with only the program counter as
information about the crash, and it is in notok.
Instead, inline calls to notok (it is just one instruction
on most systems) so that the program counter will
tell us which system call is unhappy.

R=golang-dev, gri, minux.ma, bradfitz
CC=golang-dev
https://golang.org/cl/5792048

FreeBSD, alone among our supported operating systems,
required that usleep not be interrupted.  Don't require that.

Fixes #3217.

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/5781045

For Brad.
Now FreeBSD/386 binaries run on nearlyfreespeech.net.

Fixes #2302.

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5700060

Ignore signals while we are spawning a new thread. Previously, a
signal arriving just before runtime.minit setting up the signal
handler triggers a "double fault" in signal trampolining.
Fixes #3017.

R=rsc, mikioh.mikioh, minux.ma, adg
CC=golang-dev
https://golang.org/cl/5684060

R=mikioh.mikioh, rsc
CC=golang-dev
https://golang.org/cl/5690062

R=rsc, mikioh.mikioh
CC=golang-dev
https://golang.org/cl/5689046

R=golang-dev, jsing
CC=golang-dev
https://golang.org/cl/5528106

R=adg, mikioh.mikioh
CC=golang-dev
https://golang.org/cl/5528062

This is like the ill-fated CL 5493063 except that
I have written a shell script (autogen.sh) instead of
thinking I could possibly write a correct Makefile.

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5496075

That was the last build that was close to working.
I will try that change again next week.
Make is being very subtle today.

At the reverted-to CL, the ARM traceback appears
to be broken.  I'll look into that next week too.

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5492063

R=golang-dev, r, r
CC=golang-dev
https://golang.org/cl/5493063