Darwin separately stores bits indicating whether a root certificate
should be trusted; this changes Go to read and use those when
initializing SystemCertPool.

Unfortunately, the trust API is very slow. To avoid a delay of up to
0.5s in initializing the system cert pool, we assume that
the trust settings found in kSecTrustSettingsDomainSystem will always
indicate trust. (That is, all root certs Apple distributes are trusted.)
This is not guaranteed by the API but is true in practice.

In the non-cgo codepath, we do not have that benefit, so we must check
the trust status of every certificate. This causes about 0.5s of delay
in initializing the SystemCertPool.

On OS X 10.11 and older, the "security" command requires a certificate
to be provided in a file and not on stdin, so the non-cgo codepath
creates temporary files for each certificate, further slowing initialization.

Updates #18141.

Change-Id: If681c514047afe5e1a68de6c9d40ceabbce54755
Reviewed-on: https://go-review.googlesource.com/33721
Run-TryBot: Quentin Smith <quentin@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-on: https://go-review.googlesource.com/33728

The underlying bug is fixed on master.
We don't need to (nor want to) port the fix back,
but we do want to be able to run all.bash for
minor releases.

Change-Id: Ib612fc84fd8cd1f57456dd300855318badf97cf7
Reviewed-on: https://go-review.googlesource.com/33850
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Always close the file regardless of whether the copy succeeds or fails.
Pass along the close error if the copy succeeds

Updates #16296

Change-Id: Ib394655b91d25750f029f17b3846d985f673fb50
Reviewed-on: https://go-review.googlesource.com/30410Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/33640Reviewed-by: Chris Broadfoot <cbro@golang.org>

Change-Id: Ib8cc2acc44c94ef0f85be001c5914f29606dd80b
Reviewed-on: https://go-review.googlesource.com/25016
Run-TryBot: Chris Broadfoot <cbro@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Change-Id: Ib33d7fb529aafcaf8ca7d43b2c9480f30d5c28cc
Reviewed-on: https://go-review.googlesource.com/25011Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-on: https://go-review.googlesource.com/25015Reviewed-by: Chris Broadfoot <cbro@golang.org>

Because,

* The CGI spec defines that incoming request header "Foo: Bar" maps to
  environment variable HTTP_FOO == "Bar". (see RFC 3875 4.1.18)

* The HTTP_PROXY environment variable is conventionally used to configure
  the HTTP proxy for HTTP clients (and is respected by default for
  Go's net/http.Client and Transport)

That means Go programs running in a CGI environment (as a child
process under a CGI host) are vulnerable to an incoming request
containing "Proxy: attacker.com:1234", setting HTTP_PROXY, and
changing where Go by default proxies all outbound HTTP requests.

This is CVE-2016-5386, aka https://httpoxy.org/

Fixes #16405

Change-Id: I6f68ade85421b4807785799f6d98a8b077e871f0
Reviewed-on: https://go-review.googlesource.com/25010
Run-TryBot: Chris Broadfoot <cbro@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Chris Broadfoot <cbro@golang.org>
Reviewed-on: https://go-review.googlesource.com/25012

In the beta version of the macOS Sierra (10.12) release, the
gettimeofday system call changed on x86. Previously it always returned
the time in the AX/DX registers. Now, if AX is returned as 0, it means
that the system call has stored the values into the memory pointed to by
the first argument, just as the libc gettimeofday function does. The
libc function handles both cases, and we need to do so as well.

Fixes #16272.
Fixes #16354.

Change-Id: Ibe5ad50a2c5b125e92b5a4e787db4b5179f6b723
Reviewed-on: https://go-review.googlesource.com/24812Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-on: https://go-review.googlesource.com/24967

Change-Id: Ifc545faaed438b72bfa63beb74cde2d3a67ef0e7
Reviewed-on: https://go-review.googlesource.com/22252Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Change-Id: Ib3063719cf90dfad139dd723b3b16ef0b45e312e
Reviewed-on: https://go-review.googlesource.com/22251Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-on: https://go-review.googlesource.com/22253Reviewed-by: Andrew Gerrand <adg@golang.org>

TimeoutHandler was starting the Timer when the handler was created,
instead of when serving a request. It also was sharing it between
multiple requests, which is incorrect, as the requests might start
at different times.

Store the timeout duration and create the Timer when ServeHTTP is
called. Different requests will have different timers.

The testing plumbing was simplified to store the channel used to
control when timeout happens. It overrides the regular timer.

Fixes #14568.

Change-Id: I4bd51a83f412396f208682d3ae5e382db5f8dc81
Reviewed-on: https://go-review.googlesource.com/20046Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/22274Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Andrew Gerrand <adg@golang.org>

Missed a case for closure calls (OCALLFUNC && indirect) in
esc.go:esccall.

Cleanup to runtime code for windows to more thoroughly hide
a technical escape.  Also made code pickier about failing
to late non-optional kernel32.dll.

Revised for 1.6.2

Fixes #14409.

Change-Id: Ie75486a2c8626c4583224e02e4872c2875f7bca5
Reviewed-on: https://go-review.googlesource.com/22050
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>

The existing epoll_event structure used by many of
the epoll_* syscalls was defined incorrectly
for use with ppc64le & ppc64 in the syscall
directory.  This resulted in the caller getting
incorrect information on return from these
syscalls.  This caused failures in fsnotify as
well as builds with upstream Docker.  The
structure is defined correctly in gccgo.

This adds a pad field that is expected for
these syscalls on ppc64le, ppc64.
Fixes #15135

Fixes #15288

Change-Id: If7e8ea9eb1d1ca5182c8dc0f935b334127341ffd
Reviewed-on: https://go-review.googlesource.com/21582Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-on: https://go-review.googlesource.com/22207Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

So that all Go processes do not die on startup on a system with >256 CPUs.

I tested this by hacking osinit to set ncpu to 1000.

Updates #15131
Fixes #15160

Change-Id: I52e061a0de97be41d684dd8b748fa9087d6f1aef
Reviewed-on: https://go-review.googlesource.com/21599Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-on: https://go-review.googlesource.com/22206

Updates #15292.

Change-Id: I229f66c2a41ae0738225f2ba7a574478f5d6d620
Reviewed-on: https://go-review.googlesource.com/22163Reviewed-by: Andrew Gerrand <adg@golang.org>
Reviewed-on: https://go-review.googlesource.com/22166Reviewed-by: David Symonds <dsymonds@golang.org>

This will allow us to mechanically substitute these strings
using javascript (in a forthcoming change to x/tools/godoc).

Updates #14371

Change-Id: I96e876283060ffbc9f3eabaf55d6b880685453e1
Reviewed-on: https://go-review.googlesource.com/22055Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-on: https://go-review.googlesource.com/22069

go get -u all command updates all packages including standard
commands. We need to get commands evicted from their cache to
avoid loading old versions of the packages evicted from the
packages cache.

Fixes #14444

Change-Id: Icd581a26e1db34ca634aba595fed62b097094c2f
Reviewed-on: https://go-review.googlesource.com/19899Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-on: https://go-review.googlesource.com/22040Reviewed-by: Andrew Gerrand <adg@golang.org>

Introduced in https://golang.org/cl/20462

Change-Id: I6c55f87228f0980058a10f204b57ead7594e3492
Reviewed-on: https://go-review.googlesource.com/20490Reviewed-by: Andrew Gerrand <adg@golang.org>
Reviewed-on: https://go-review.googlesource.com/22075Reviewed-by: Minux Ma <minux@golang.org>

Fixes #14519.

Change-Id: I8f78f67a463e6467e09df90446f7ebd28789d6c9
Reviewed-on: https://go-review.googlesource.com/19933Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-on: https://go-review.googlesource.com/22071
Run-TryBot: Andrew Gerrand <adg@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Bump up the multiplier to 20.  Also run the fast version first, so that
the slow version is likely to start up faster.

Change-Id: Ia0654cc1212ab03a45da1904d3e4b57d6a8d02a0
Reviewed-on: https://go-review.googlesource.com/19835
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>
Reviewed-on: https://go-review.googlesource.com/22070Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Fixes #14613

Change-Id: I40d9696db3879549e78373ef17f6c92bd4b3470b
Reviewed-on: https://go-review.googlesource.com/21596Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/22051
Run-TryBot: Andrew Gerrand <adg@golang.org>

When we grow the heap, we create a temporary "in use" span for the
memory acquired from the OS and then free that span to link it into
the heap. Hence, we (1) increase pagesInUse when we make the temporary
span so that (2) freeing the span will correctly decrease it.

However, currently step (1) increases pagesInUse by the number of
pages requested from the heap, while step (2) decreases it by the
number of pages requested from the OS (the size of the temporary
span). These aren't necessarily the same, since we round up the number
of pages we request from the OS, so steps 1 and 2 don't necessarily
cancel out like they're supposed to. Over time, this can add up and
cause pagesInUse to underflow and wrap around to 2^64. The garbage
collector computes the sweep ratio from this, so if this happens, the
sweep ratio becomes effectively infinite, causing the first allocation
on each P in a sweep cycle to sweep the entire heap. This makes
sweeping effectively STW.

Fix this by increasing pagesInUse in step 1 by the number of pages
requested from the OS, so that the two steps correctly cancel out. We
add a test that checks that the running total matches the actual state
of the heap.

Fixes #15022. For 1.6.x.

Change-Id: Iefd9d6abe37d0d447cbdbdf9941662e4f18eeffc
Reviewed-on: https://go-review.googlesource.com/21280
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-on: https://go-review.googlesource.com/21456Reviewed-by: Andrew Gerrand <adg@golang.org>

CL 14603 attempted to preserve the callee-save registers for
the darwin/arm runtime initialization routine, but I believe it
wasn't sufficient and resulted in the crash reported in issue

Saving and restoring the registers on the stack the same way
linux/arm does seems more obvious and fixes #14778, so do that.

Even though #14778 is not reproducible on darwin/arm64, I applied
a similar change there, and to linux/arm64 which obeys the same
calling convention.

Finally, this CL is a candidate for a 1.6 minor release for the same
reason CL 14603 was in a 1.5 minor release (as CL 16968). It is
small and only touches the iOS platforms and gomobile on darwin/arm
is currently useless without it.

Fixes #14778
Fixes #12590 (again)

Change-Id: I7401daf0bbd7c579a7e84761384a7b763651752a
Reviewed-on: https://go-review.googlesource.com/20621Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/22049

Fixes #14759.

Change-Id: I6cef0b49ba00ab85b25ce1e7bad80ad85e2d5076
Reviewed-on: https://go-review.googlesource.com/20545
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
Reviewed-on: https://go-review.googlesource.com/22048Reviewed-by: Ian Lance Taylor <iant@golang.org>

Fixes #14662

Change-Id: I8f7d5e8d2dd8dc763e940e061b424427359246e7
Reviewed-on: https://go-review.googlesource.com/20462Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-on: https://go-review.googlesource.com/22047

The existing implementation deals with absolute relocations in __TEXT
for darwin/amd64 in build-mode c-shared, but it ignores c-archive.
This results in issues when trying to use a c-archive in an iOS
app on the 64-bit simulator. This patch adds c-archive to the
handling of this issue.

Fixes #14217

Change-Id: I2e4d5193caa531171ad22fd0cd420a8bfb4646a6
Reviewed-on: https://go-review.googlesource.com/19206Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/22046

The io.Reader contract makes no promises about how a Reader should
behave after it returns its first error. Usually the errors are
sticky, but they don't have to be. A regression in zlib.Reader (bug
accidentally relied on sticky errors.

Minimal fix: wrap the user's provided Reader in a Reader which
guarantees stickiness. The minimal fix is less scary than touching
the multipart state machine.

Fixes #14676

Change-Id: I8dd8814b13ae5530824ae0e68529f788974264a5
Reviewed-on: https://go-review.googlesource.com/20297
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-on: https://go-review.googlesource.com/22045

gcMarkRootCheck takes ~10ns per goroutine. This is just a debugging
check, so disable it (plus, if something is going to go wrong, it's
more likely to go wrong during concurrent mark).

We may be able to re-enable this later, or move it to after we've
started the world again. (But not for 1.6.x.)

For 1.6.x.

Fixes #14419.

name / 95%ile-time/markTerm          old          new  delta
500kIdleGs-12                24.0ms ± 0%  18.9ms ± 6%  -21.46%  (p=0.000 n=15+20)

Change-Id: Idb2a2b1771449de772c159ef95920d6df1090666
Reviewed-on: https://go-review.googlesource.com/20148Reviewed-by: Rick Hudson <rlh@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/22044Reviewed-by: Austin Clements <austin@google.com>

Currently we reset the mark state during STW sweep termination. This
involves looping over all of the goroutines. Each iteration of this
loop takes ~25ns, so at around 400k goroutines, we'll exceed our 10ms
pause goal.

However, it's safe to do this before we stop the world for sweep
termination because nothing is consuming this state yet. Hence, move
the reset to just before STW.

This isn't perfect: a long reset can still delay allocating goroutines
that block on GC starting. But it's certainly better to block some
things eventually than to block everything immediately.

For 1.6.x.

Fixes #14420.

name \ 95%ile-time/sweepTerm           old          new  delta
500kIdleGs-12                 11312µs ± 6%  18.9µs ± 6%  -99.83%  (p=0.000 n=16+20)

Change-Id: I9815c4d8d9b0d3c3e94dfdab78049cefe0dcc93c
Reviewed-on: https://go-review.googlesource.com/20147Reviewed-by: Rick Hudson <rlh@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/22043Reviewed-by: Austin Clements <austin@google.com>

Also fix compiler-invoked panics to avoid a confusing "malloc deadlock"
crash if they are invoked while executing the runtime.

Fixes #14599.

Change-Id: I89436abcbf3587901909abbdca1973301654a76e
Reviewed-on: https://go-review.googlesource.com/20219
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-on: https://go-review.googlesource.com/22042Reviewed-by: Ian Lance Taylor <iant@golang.org>

Fixes #14483.

Change-Id: I0cddfe27fd8d00ba85659d0b618410e39ebf45cb
Reviewed-on: https://go-review.googlesource.com/19860Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-on: https://go-review.googlesource.com/22041Reviewed-by: Ian Lance Taylor <iant@golang.org>

Fixes #14377

Change-Id: I130a6e1b8bc827db44efd0a74e759b894ecc4977
Reviewed-on: https://go-review.googlesource.com/19823Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/22039

Like bionic, musl also doesn't provide vsyscall helper in %gs:0x10,
and as int $0x80 is as fast as calling %gs:0x10, just use int $0x80
always.

Because we're no longer using vsyscall in VDSO, get rid of VDSO code
for linux/386 too.

Fixes #14476.

Change-Id: I00ec8652060700e0a3c9b524bfe3c16a810263f6
Reviewed-on: https://go-review.googlesource.com/19833
Run-TryBot: Minux Ma <minux@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/22038Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Fixes #14384.

Change-Id: Ib025cf2d20754b4c2db52f0a8a4717fd303371d6
Reviewed-on: https://go-review.googlesource.com/19660
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-on: https://go-review.googlesource.com/22037

Fixes #14387.

Change-Id: Icc98be80f549c5e1f55c5e693bfea97b456a6c41
Reviewed-on: https://go-review.googlesource.com/19621
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-on: https://go-review.googlesource.com/22036Reviewed-by: Ian Lance Taylor <iant@golang.org>

I had accidentally disabled a headline feature at the last second. :(

Fixes #14391

Change-Id: I1992c9b801072b7538b95c55242be174075ff932
Reviewed-on: https://go-review.googlesource.com/19672Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/22035Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Change-Id: If172cf5ee0d2bde38581486f1c887d3d4826e95c
Reviewed-on: https://go-review.googlesource.com/21828Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Andrew Gerrand <adg@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Change-Id: Icb9e947a43fb87fbfe0655b09e0d7e8f61825aeb
Reviewed-on: https://go-review.googlesource.com/21825Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-on: https://go-review.googlesource.com/21826Reviewed-by: Andrew Gerrand <adg@golang.org>

Fixes #15120

Change-Id: I1d9a192ac163826bad8b46e8c0b0b9e218e69570
Reviewed-on: https://go-review.googlesource.com/21520Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/21698
Run-TryBot: Andrew Gerrand <adg@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>

Updates the syscall generator for patchset 4 of https://golang.org/cl/21388.

Updates #14959

Change-Id: Icbd6df489887d3dcc076dfc73d4feb1376abaf8b
Reviewed-on: https://go-review.googlesource.com/21428Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-on: https://go-review.googlesource.com/21697Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Make sure that for any DLL that Go uses itself, we only look for the
DLL in the Windows System32 directory, guarding against DLL preloading
attacks.

(Unless the Windows version is ancient and LoadLibraryEx is
unavailable, in which case the user probably has bigger security
problems anyway.)

This does not change the behavior of syscall.LoadLibrary or NewLazyDLL
if the DLL name is something unused by Go itself.

This change also intentionally does not add any new API surface. Instead,
x/sys is updated with a LoadLibraryEx function and LazyDLL.Flags in:
    https://golang.org/cl/21388

Updates #14959

Change-Id: I8d29200559cc19edf8dcf41dbdd39a389cd6aeb9
Reviewed-on: https://go-review.googlesource.com/21140Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/21696Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>