The go command uses file modification times to decide when a
package is out of date: if the .a file is older than a source file,
the .a file needs to be rebuilt. This scheme breaks down when
multiple source files compile into a single .a file: if one source file
is removed but no other changes are made, there is no indication
that the .a file is out of date.

The fix is to store a value called a build ID in the package archive itself.
The build ID is a hash of the names of all source files compiled into the package.
A later go command can read the build ID out of the package archive
and compare to the build ID derived from the list of source files it now
sees in the directory. If the build IDs differ, the file list has changed,
and the package must be rebuilt.

There is a cost here: when scanning a package directory, in addition
to reading the beginning of every source file for build tags and imports,
the go command now also reads the beginning of the associated
package archive, for the build ID. This is at most a doubling in the
number of files read. On my 2012 MacBook Pro, the time for
'go list std' increases from about 0.215 seconds to about 0.23 seconds.

For executable binaries, the approach is the same except that the
build ID information is stored in a trailer at the end of the executable file.
It remains to be seen if anything objects to the trailer.
I don't expect problems except maybe on Plan 9.

Fixes #3895.

Change-Id: I21b4ebf5890c1a39e4a013eabe1ddbb5f3510c04
Reviewed-on: https://go-review.googlesource.com/9154Reviewed-by: Ian Lance Taylor <iant@golang.org>

$ sizeof -p cmd/compile/internal/gc Node
Node 144
$

Change-Id: I688e3790964fe42f48c19f697ec38094a92fe1c1
Reviewed-on: https://go-review.googlesource.com/10531Reviewed-by: Ian Lance Taylor <iant@golang.org>

Suggested during code reviews of last 15 CLs (or so).

Change-Id: If780f6eb47a7a31df133c64d5dcf0eaf04d8447b
Reviewed-on: https://go-review.googlesource.com/10675Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

stkcheck is flow-insensitive: It processes calls in PC order.
Since morestack was always the first call in a function,
it was a safe, conservative approximation to simply adjust stack
space as we went, recognizing morestack when it showed up.

Subsequent CLS will rearrange the function prologue;
morestack may no longer be the first call in a function.

Introducing flow-sensitivity to stkcheck would allow this,
and possibly allow a smaller stackguard.
It is also a high risk change and possibly expensive.

Instead, assume that all calls to morestack occur as
part of the function prologue, no matter where they
are located in the program text.

Updates #10587.

Change-Id: I4dcdd4256a980fc4bc433a68a10989ff57f7034f
Reviewed-on: https://go-review.googlesource.com/10496
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Fixes #10209.

Change-Id: I248434f9195c868befd1ed8a6000a9cac72d1df8
Reviewed-on: https://go-review.googlesource.com/10263Reviewed-by: Ian Lance Taylor <iant@golang.org>

Otherwise subsequent tests won't see any modified GOROOT.

With this CL I can move my GOROOT, set GOROOT to the new location, and
the runtime tests pass. Previously the crash_tests would instead look
for the GOROOT baked into the binary, instead of the env var:

--- FAIL: TestGcSys (0.01s)
        crash_test.go:92: building source: exit status 2
                go: cannot find GOROOT directory: /home/bradfitz/go
--- FAIL: TestGCFairness (0.01s)
        crash_test.go:92: building source: exit status 2
                go: cannot find GOROOT directory: /home/bradfitz/go
--- FAIL: TestGdbPython (0.07s)
        runtime-gdb_test.go:64: building source exit status 2
                go: cannot find GOROOT directory: /home/bradfitz/go
--- FAIL: TestLargeStringConcat (0.01s)
        crash_test.go:92: building source: exit status 2
                go: cannot find GOROOT directory: /home/bradfitz/go

Update #10029

Change-Id: If91be0f04d3acdcf39a9e773a4e7905a446bc477
Reviewed-on: https://go-review.googlesource.com/10685Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>

The only unreviewed change is in mparith3.go.

Change-Id: Iec0885e7688981cbaed04c152dc9b1c7032677e6
Reviewed-on: https://go-review.googlesource.com/10665Reviewed-by: Alan Donovan <adonovan@google.com>
Run-TryBot: Robert Griesemer <gri@golang.org>

Change-Id: Ie38742cddc5a256e2f0fc0f720c0ed2f1b2e1bca
Reviewed-on: https://go-review.googlesource.com/10664Reviewed-by: Alan Donovan <adonovan@google.com>

m was being resliced as the result of looking for the first
non zero word of the mantissa, however m was not used later
in printing.

Spotted by Gordon Klaus, https://groups.google.com/forum/#!topic/golang-nuts/MdDLbvOjb4o

Change-Id: Ifbebb51ea5e0d86cb8e0422eb184b8634639a733
Reviewed-on: https://go-review.googlesource.com/10604Reviewed-by: Robert Griesemer <gri@golang.org>

The build ID is an opaque token supplied by the build system.
The compiler writes it out early in the Go export metadata
(the second line), in a way that does not bother existing readers.

The intent is that the go command can use this to store information
about the sources for the generated code, so that it can detect
stale packages even in cases (like removed files) where mtimes fail.

Change-Id: Ib5082515d6cde8a07a8d4b5c69d1e8e4190cb5e1
Reviewed-on: https://go-review.googlesource.com/9153Reviewed-by: Ian Lance Taylor <iant@golang.org>

This one didn't get written out.

Change-Id: Iee173861fb4dc7cafa64ba5f601f4664b6e8da4e
Reviewed-on: https://go-review.googlesource.com/10681Reviewed-by: Russ Cox <rsc@golang.org>

In particular, this avoids moving the mtime on runtime/zversion.go
forward unless the file is out of date. In turn, this makes cross compiles
that run dist multiple times coexist nicely.

(It's no longer necessary to run dist multiple times to set up cross compiles,
but people still might, and it's easy to fix regardless.)

Fixes #4749.

Change-Id: Id430525f168f106bc4b821ca74b2ca498a748f14
Reviewed-on: https://go-review.googlesource.com/9152Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

It was mishandling conjunctions containing negations.

Change-Id: Ife571b28416870ba2ceadbdac5ecb4670432bba1
Reviewed-on: https://go-review.googlesource.com/9151Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Fixes #8809.

Change-Id: Id443fd406e9c611d5dfabc71a98eb71d1cc0972c
Reviewed-on: https://go-review.googlesource.com/9150Reviewed-by: Ian Lance Taylor <iant@golang.org>

Change-Id: I6476284a2cf80d50bd0a57fd9a0de9bc74273c7e
Reviewed-on: https://go-review.googlesource.com/10680Reviewed-by: Russ Cox <rsc@golang.org>

See golang.org/s/go14internal.

Fixes #10479.

Change-Id: I8c657dadeb5f10af060f22dedc15c1af989d4519
Reviewed-on: https://go-review.googlesource.com/9156Reviewed-by: Rob Pike <r@golang.org>

$ sizeof -p cmd/compile/internal/gc Node
Node 160
$

Change-Id: Ib0bd4230c8547f36972b2a9d81ba3eca81496e39
Reviewed-on: https://go-review.googlesource.com/10537Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

This is follow-up to CL10607

- Refactor AddParseTree() to use t.associate()
- Refactor Parse() to use AddParseTree() to put entries into common structure
- Clone() should not put entry in t.tmpl for undefined template
- Clarify documentation for Templates()
- Clarify documentation for AddParseTree() to include the error case

Updates #10910
Uodates #10926

Includes test cases for most of the above changes

Change-Id: I25b2fce6f9651272866f881acf44e4dbca04a4a8
Reviewed-on: https://go-review.googlesource.com/10622Reviewed-by: Rob Pike <r@golang.org>
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

$ sizeof -p cmd/compile/internal/gc Node
Node 168
$

Change-Id: Ie7e2cee8cec101560bd5dd013b23969278f89b12
Reviewed-on: https://go-review.googlesource.com/10536Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 168
$

Change-Id: I22bcea8099f308298c9db75c937f35e7fca906f1
Reviewed-on: https://go-review.googlesource.com/10535Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 168
$

Change-Id: I7decd950fe068c0f294c6c9bff07ef809c394429
Reviewed-on: https://go-review.googlesource.com/10534Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 168
$

Change-Id: I1e2f17bfee0e6ca7213706c4cef8d990f4461915
Reviewed-on: https://go-review.googlesource.com/10533Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 168
$

Change-Id: If624a2d72ec04ef30a1bc7ce76c0d61a526d8a37
Reviewed-on: https://go-review.googlesource.com/10532Reviewed-by: Ian Lance Taylor <iant@golang.org>

$ sizeof -p cmd/compile/internal/gc Node
Node 176
$

Change-Id: Ibf1ab531a60d4af8a0c242c0e504f4fd50cd5b36
Reviewed-on: https://go-review.googlesource.com/10530Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 192
$

Change-Id: I8f0c1a3cc2bf9c8eff02bbd8d061ff98affc9eb0
Reviewed-on: https://go-review.googlesource.com/10529Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 200
$

Change-Id: Iba4e88eac6bee3e2349e818a5a2326deabcb96f9
Reviewed-on: https://go-review.googlesource.com/10528Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 224
$

Change-Id: Id0969e8df99c43a5f6f8d77a38f20a71a467e7c6
Reviewed-on: https://go-review.googlesource.com/10527Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 232
$

Change-Id: I4be025f4ec11f882f24ae7582821d36d3b122b77
Reviewed-on: https://go-review.googlesource.com/10526Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Also add a reference to the strings blog post.

Fixes #11045.

Change-Id: Ic0a8908cbd7b51a36d104849fa0e8abfd54de2b9
Reviewed-on: https://go-review.googlesource.com/10662Reviewed-by: Andrew Gerrand <adg@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Fixes #10924

Change-Id: I0caf5f8f82682ee48e95270d43328550bfd8b6e1
Reviewed-on: https://go-review.googlesource.com/10440Reviewed-by: Rob Pike <r@golang.org>

Sending out the conversion of a single test to get comments on the
overall approach.  Converting more tests will follow.

Change-Id: I4755442d08aeb6f74c46856ae406fec41cf8d5dc
Reviewed-on: https://go-review.googlesource.com/10464
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

$ sizeof -p cmd/compile/internal/gc Node
Node 240
$

Change-Id: Id12710c480ed4e0a5bf4f5006f6bd56ef91a2af1
Reviewed-on: https://go-review.googlesource.com/10525Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 248
$

Change-Id: I0fbfeb0d0b36e225eb282fce9e480a96ec1d278f
Reviewed-on: https://go-review.googlesource.com/10524Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 256
$

Change-Id: I89ac8bbe077664aa076092bfd096947e84c0624c
Reviewed-on: https://go-review.googlesource.com/10523Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 264
$

Change-Id: I5c90089dcf5df51c874250f28a1bc3ec32f764b9
Reviewed-on: https://go-review.googlesource.com/10522Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 272
$

Change-Id: I3d9b67eebfc0be0a4b9768d3de3dc76300abd89c
Reviewed-on: https://go-review.googlesource.com/10521Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 288
$

Change-Id: I4e316efa246132b3faa3a892e4fe9c9039250665
Reviewed-on: https://go-review.googlesource.com/10520Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

It was an oversight (but as linux/arm64 doesn't support internal
linking and always use external linking with cgo, no harm is done.)

Change-Id: Ie5f2b445cb67a8e63d6b868e63379c68847554f9
Reviewed-on: https://go-review.googlesource.com/10636Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Currently the GODEBUG=gctrace=1 trace line includes "@n.nnns" to
indicate the time that the GC cycle ended relative to the time the
program started. This was meant to be consistent with the utilization
as of the end of the cycle, which is printed next on the trace line,
but it winds up just being confusing and unexpected.

Change the trace line to include the time that the GC cycle started
relative to the time the program started.

Change-Id: I7d64580cd696eb17540716d3e8a74a9d6ae50650
Reviewed-on: https://go-review.googlesource.com/10634Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Due to the requirements of parsing template definitions that mention
other templates that are not yet defined, a Template can be in two states:
defined and undefined. Thus, although one calls New, the resulting
template has no definition even though it exists as a data structure.

Thus, for example, will return nil for a template that is named but not
yet defined.

Fixes #10910
Fixes #10926

Clarify the documentation a little to explain this,
Also tidy up the code a little and remove a spurious call to init.

Change-Id: I22cc083291500bca424e83dc12807e0de7b00b7a
Reviewed-on: https://go-review.googlesource.com/10641Reviewed-by: Andrew Gerrand <adg@golang.org>