GitLab

This change fixes inconsistent error values on Accept{,TCP,Unix}.

Updates #4856.

Change-Id: Ie3bb534c19a724cacb3ea3f3656e46c810b2123f
Reviewed-on: https://go-review.googlesource.com/8996

Reviewed-by: Ian Lance Taylor <iant@golang.org>

This change fixes inconsistent error values on Close, CloseRead and
CloseWrite.

Updates #4856.

Change-Id: I3c4d46ccd7d6e1a2f52d8e75b512f62c533a368d
Reviewed-on: https://go-review.googlesource.com/8994

Reviewed-by: Ian Lance Taylor <iant@golang.org>

This change fixes inconsistent error values on Write,
WriteTo{,UDP,IP,Unix} and WriteMsg{UDP,IP,Unix}.

Updates #4856.

Change-Id: I4208ab6a0650455ad7d70a80a2d6169351d6055f
Reviewed-on: https://go-review.googlesource.com/8993

Reviewed-by: Ian Lance Taylor <iant@golang.org>

This change fixes inconsistent error values on Read,
ReadFrom{,UDP,IP,Unix} and ReadMsg{UDP,IP,Unix}.

Updates #4856.

Change-Id: I7de5663094e09be2d78cdb18ce6f1e7ec260888d
Reviewed-on: https://go-review.googlesource.com/8992

Reviewed-by: Ian Lance Taylor <iant@golang.org>

This change adds socket system call hooks to existing test cases for
simulating a bit complicated network conditions to help making timeout
and dual IP stack test cases work more properly in followup changes.

Also test cases print debugging information in non-short mode like the
following:

Leaked goroutines:
net.TestWriteTimeout.func2(0xc20802a5a0, 0xc20801d000, 0x1000, 0x1000, 0xc2081d2ae0)
	/go/src/net/timeout_test.go:170 +0x98
created by net.TestWriteTimeout
	/go/src/net/timeout_test.go:173 +0x745
net.runDatagramPacketConnServer(0xc2080730e0, 0x2bd270, 0x3, 0x2c1770, 0xb, 0xc2081d2ba0, 0xc2081d2c00)
	/go/src/net/server_test.go:398 +0x667
created by net.TestTimeoutUDP
	/go/src/net/timeout_test.go:247 +0xc9
	(snip)

Leaked sockets:
3: {Cookie:615726511685632 Err:<nil> SocketErr:0}
5: {Cookie:7934075906097152 Err:<nil> SocketErr:0}

Socket statistical information:
{Family:1 Type:805306370 Protocol:0 Opened:17 Accepted:0 Connected:5 Closed:17}
{Family:2 Type:805306369 Protocol:0 Opened:450 Accepted:234 Connected:279 Closed:636}
{Family:1 Type:805306369 Protocol:0 Opened:11 Accepted:5 Connected:5 Closed:16}
{Family:28 Type:805306369 Protocol:0 Opened:95 Accepted:22 Connected:16 Closed:116}
{Family:2 Type:805306370 Protocol:0 Opened:84 Accepted:0 Connected:34 Closed:83}
{Family:28 Type:805306370 Protocol:0 Opened:52 Accepted:0 Connected:4 Closed:52}

Change-Id: I0e84be59a0699bc31245c78e2249423459b8cdda
Reviewed-on: https://go-review.googlesource.com/6390

Reviewed-by: Ian Lance Taylor <iant@golang.org>

We don't need placeholders for the old built-in poll server any more.

Change-Id: I3a510aec6a30bc2ac97676c400177cdfe557b8dc
Reviewed-on: https://go-review.googlesource.com/3863

Reviewed-by: Dmitry Vyukov <dvyukov@google.com>

A few packages that handle net.IPConn in golang.org/x/net sub repository
already implement full stack test cases with more coverage than the net
package. There is no need to keep duplicate code around here.

This change removes full stack test cases for IPConn that require
knowing how to speak with each of protocol stack implementation of
supported platforms.

Change-Id: I871119a9746fc6a2b997b69cfd733463558f5816
Reviewed-on: https://go-review.googlesource.com/3404

Reviewed-by: Ian Lance Taylor <iant@golang.org>

Preventing returning io.EOF on non-connection oriented sockets is
already applied to Unix variants. This CL applies it to Windows.

Update #4856.

Change-Id: I82071d40f617e2962d0540b9d1d6a10ea4cdb2ec
Reviewed-on: https://go-review.googlesource.com/2203

Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>

Preparation was in CL 134570043.
This CL contains only the effect of 'hg mv src/pkg/* src'.
For more about the move, see golang.org/s/go14nopkg.

This CL removes sockaddrToAddr functions from socket creation
operations to avoid the bug like issue 7183.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/105100046

On Linux, adding a socket descriptor to epoll instance before getting
the EINPROGRESS return value from connect system call could be a root
cause of spurious on-connect events.

See golang.org/issue/8276, golang.org/issue/8426 for further information.

All credit to Jason Eggleston <jason@eggnet.com>

Fixes #8276.
Fixes #8426.

LGTM=dvyukov
R=dvyukov, golang-codereviews, adg, dave, iant, alex.brainman
CC=golang-codereviews
https://golang.org/cl/120820043

Remove an RUnlock of syscall.ForkLock with no matching RLock.
Holding ForkLock in netFD.dup is unnecessary: dupCloseOnExecOld
locks and unlocks the lock on its own and dupCloseOnExec doesn't
need the ForkLock to be held.

LGTM=iant
R=golang-codereviews, bradfitz, iant
CC=golang-codereviews
https://golang.org/cl/99800044

Fixes #7645

LGTM=iant, bradfitz
R=iant, bradfitz
CC=golang-codereviews
https://golang.org/cl/90170046

LGTM=iant
R=iant
CC=golang-codereviews
https://golang.org/cl/83910043

There is no way to call them from outside the net package.
They are used to implement UCPConn.ReadMsgUDP and similar.

LGTM=mikioh.mikioh
R=golang-codereviews, mikioh.mikioh
CC=golang-codereviews
https://golang.org/cl/83730044

The previous fix CL 69340044 still leaves a possibility of it.
This CL prevents the kernel, especially DragonFly BSD, from
performing unpredictable asynchronous connection establishment
on stream-based transport layer protocol sockets.

Update #7541
Update #7474

LGTM=jsing
R=jsing
CC=golang-codereviews
https://golang.org/cl/75930043

Performing multiple connect system calls on a non-blocking socket
under DragonFly BSD does not necessarily result in errors from earlier
connect calls being returned, particularly if we are connecting to
localhost. Instead, once netpoll tells us that the socket is ready,
get the SO_ERROR socket option to see if the connection succeeded
or failed.

Fixes #7474

LGTM=mikioh.mikioh
R=mikioh.mikioh
CC=golang-codereviews
https://golang.org/cl/69340044

See golang.org/s/go13nacl for design overview.

This CL is the mostly mechanical changes from rsc's Go 1.2 based NaCl branch, specifically 39cb35750369 to 500771b477cf from https://code.google.com/r/rsc-go13nacl. This CL does not include working NaCl support, there are probably two or three more large merges to come.

CL 15750044 is not included as it involves more invasive changes to the linker which will need to be merged separately.

The exact change lists included are

15050047: syscall: support for Native Client
15360044: syscall: unzip implementation for Native Client
15370044: syscall: Native Client SRPC implementation
15400047: cmd/dist, cmd/go, go/build, test: support for Native Client
15410048: runtime: support for Native Client
15410049: syscall: file descriptor table for Native Client
15410050: syscall: in-memory file system for Native Client
15440048: all: update +build lines for Native Client port
15540045: cmd/6g, cmd/8g, cmd/gc: support for Native Client
15570045: os: support for Native Client
15680044: crypto/..., hash/crc32, reflect, sync/atomic: support for amd64p32
15690044: net: support for Native Client
15690048: runtime: support for fake time like on Go Playground
15690051: build: disable various tests on Native Client

LGTM=rsc
R=rsc
CC=golang-codereviews
https://golang.org/cl/68150047

LGTM=dave, rsc
R=golang-codereviews, minux.ma, mikioh.mikioh, dave, iant, rsc
CC=golang-codereviews
https://golang.org/cl/36030043

On Solaris, if you do a in-progress connect, and then the
server accepts and closes the socket, the client's later
attempt to complete the connect will fail with EINVAL.  Handle
this case by assuming that the connect succeeded.  This code
is weird enough that it is implemented as Solaris-only so that
it doesn't hide a real error on a different OS.

Update #6828

R=golang-codereviews, bradfitz, dave
CC=golang-codereviews
https://golang.org/cl/46160043

Since CL 38bf89161a72 raw socket tests are not executed
on windows builders. This change re-enable them again.
It will attempt to run raw socket tests only if user
is permitted to create raw socket by OS.

Fixes #6392

R=golang-dev
CC=golang-dev, mikioh.mikioh, rsc
https://golang.org/cl/13422044

This CL adds minimal support of Happy Eyeballs-like TCP connection
setup to Dialer API. Happy Eyeballs and derivation techniques are
described in the following:

- Happy Eyeballs: Success with Dual-Stack Hosts
  http://tools.ietf.org/html/rfc6555

- Analysing Dual Stack Behaviour and IPv6 Quality
  http://www.potaroo.net/presentations/2012-04-17-dual-stack-quality.pdf

Usually, the techniques consist of three components below.

- DNS query racers, that run A and AAAA queries in parallel or series
- A short list of destination addresses
- TCP SYN racers, that run IPv4 and IPv6 transport in parallel or series

This CL implements only the latter two. The existing DNS query
component gathers together A and AAAA records in series, so we don't
touch it here. This CL just uses extended resolveInternetAddr and makes
it possible to run multiple Dial racers in parallel.

For example, when the given destination is a DNS name and the name has
multiple address family A and AAAA records, and it happens on the TCP
wildcard network "tcp" with DualStack=true like the following:

(&net.Dialer{DualStack: true}).Dial("tcp", "www.example.com:80")

The function will return a first established connection either TCP over
IPv4 or TCP over IPv6, and close the other connection internally.

Fixes #3610.
Fixes #5267.

Benchmark results on freebsd/amd64 virtual machine, tip vs. tip+12416043:

benchmark                           old ns/op    new ns/op    delta
BenchmarkTCP4OneShot                    50696        52141   +2.85%
BenchmarkTCP4OneShotTimeout             65775        66426   +0.99%
BenchmarkTCP4Persistent                 10986        10457   -4.82%
BenchmarkTCP4PersistentTimeout          11207        10445   -6.80%
BenchmarkTCP6OneShot                    62009        63718   +2.76%
BenchmarkTCP6OneShotTimeout             78351        79138   +1.00%
BenchmarkTCP6Persistent                 14695        14659   -0.24%
BenchmarkTCP6PersistentTimeout          15032        14646   -2.57%
BenchmarkTCP4ConcurrentReadWrite         7215         6217  -13.83%
BenchmarkTCP6ConcurrentReadWrite         7528         7493   -0.46%

benchmark                          old allocs   new allocs    delta
BenchmarkTCP4OneShot                       36           36    0.00%
BenchmarkTCP4OneShotTimeout                36           36    0.00%
BenchmarkTCP4Persistent                     0            0     n/a%
BenchmarkTCP4PersistentTimeout              0            0     n/a%
BenchmarkTCP6OneShot                       37           37    0.00%
BenchmarkTCP6OneShotTimeout                37           37    0.00%
BenchmarkTCP6Persistent                     0            0     n/a%
BenchmarkTCP6PersistentTimeout              0            0     n/a%
BenchmarkTCP4ConcurrentReadWrite            0            0     n/a%
BenchmarkTCP6ConcurrentReadWrite            0            0     n/a%

benchmark                           old bytes    new bytes    delta
BenchmarkTCP4OneShot                     2500         2503    0.12%
BenchmarkTCP4OneShotTimeout              2508         2505   -0.12%
BenchmarkTCP4Persistent                     0            0     n/a%
BenchmarkTCP4PersistentTimeout              0            0     n/a%
BenchmarkTCP6OneShot                     2713         2707   -0.22%
BenchmarkTCP6OneShotTimeout              2722         2720   -0.07%
BenchmarkTCP6Persistent                     0            0     n/a%
BenchmarkTCP6PersistentTimeout              0            0     n/a%
BenchmarkTCP4ConcurrentReadWrite            0            0     n/a%
BenchmarkTCP6ConcurrentReadWrite            0            0     n/a%

R=golang-dev, bradfitz, nightlyone, rsc
CC=golang-dev
https://golang.org/cl/12416043

This CL adds the netaddr interface that will carry a single network
endpoint address or a short list of IP addresses to dial helper
functions in the upcoming CLs.

This is in preparation for TCP connection setup with fast failover on
dual IP stack node as described in RFC 6555.

Update #3610
Update #5267

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/13368044

Fixes #4195

R=golang-dev, mikioh.mikioh
CC=golang-dev
https://golang.org/cl/12960046

Make the net package build and work on dragonfly.

R=bradfitz
CC=golang-dev
https://golang.org/cl/13173044

Update #4856

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/12763044

The mutex, fdMutex, handles locking and lifetime of sysfd,
and serializes Read and Write methods.
This allows to strip 2 sync.Mutex.Lock calls,
2 sync.Mutex.Unlock calls, 1 defer and some amount
of misc overhead from every network operation.

On linux/amd64, Intel E5-2690:
benchmark                             old ns/op    new ns/op    delta
BenchmarkTCP4Persistent                    9595         9454   -1.47%
BenchmarkTCP4Persistent-2                  8978         8772   -2.29%
BenchmarkTCP4ConcurrentReadWrite           4900         4625   -5.61%
BenchmarkTCP4ConcurrentReadWrite-2         2603         2500   -3.96%

In general it strips 70-500 ns from every network operation depending
on processor model. On my relatively new E5-2690 it accounts to ~5%
of network op cost.

Fixes #6074.

R=golang-dev, bradfitz, alex.brainman, iant, mikioh.mikioh
CC=golang-dev
https://golang.org/cl/12418043

Unlike the existing net package own pollster, runtime-integrated
network pollster on BSD variants, actually kqueue, requires a socket
that has beed passed to syscall.Listen previously for a stream
listener.

This CL separates pollDesc.Init (actually runtime_pollOpen) from newFD
to allow control of each state of sockets and adds init method to netFD
instead. Upcoming CLs will rearrange the call order of runtime-integrated
pollster and syscall functions like the following;

- For dialers that open active connections, runtime_pollOpen will be
  called in between syscall.Bind and syscall.Connect.

- For stream listeners that open passive stream connections,
  runtime_pollOpen will be called just after syscall.Listen.

- For datagram listeners that open datagram connections,
  runtime_pollOpen will be called just after syscall.Bind.

This is in preparation for runtime-integrated network pollster for BSD
variants.

Update #5199

R=dvyukov, alex.brainman, minux.ma
CC=golang-dev
https://golang.org/cl/8608044

On 10.6, OS X's fcntl returns EBADF instead of EINVAL.

R=golang-dev, iant, dave
CC=golang-dev
https://golang.org/cl/12493043

This means that in the common case (modern kernel), we only
make 1 system call to dup instead of two, and we also avoid
grabbing the syscall.ForkLock.

R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/12476043

If netFD is closed by finalizer, runtime netpoll descriptor is not freed.

R=golang-dev, dave, alex.brainman
CC=golang-dev
https://golang.org/cl/12037043

R=golang-dev, bradfitz
CC=dvyukov, golang-dev, mikioh.mikioh
https://golang.org/cl/9495044

Fixes #5355.

R=golang-dev, mikioh.mikioh, bradfitz, r
CC=golang-dev
https://golang.org/cl/8966046

Makes it possible to return the spent runtime.PollDesc to
runtime.pollcache descriptor pool when netFD.connect or
syscall.Listen fails.

Fixes #5219.

R=dvyukov, dave, bradfitz, adg
CC=golang-dev
https://golang.org/cl/8318044

Per discussions on golang-nuts and golang-dev:
"Some concerns with DialOpt"
https://groups.google.com/d/msg/golang-nuts/Hfh9aqhXyUw/W3uYi8lOdKcJ
https://groups.google.com/d/msg/golang-dev/37omSQeWv4Y/KASGIfPpXh0J

R=golang-dev, google, r
CC=golang-dev
https://golang.org/cl/8274043

Fixes #5061.

Current code relies on the fact that fd's are automatically removed from epoll set when closed. However, it is not true. Underlying file description is removed from epoll set only when *all* fd's referring to it are closed.

There are 2 bad consequences:
1. Kernel delivers notifications on already closed fd's.
2. The following sequence of events leads to error:
   - add fd1 to epoll
   - dup fd1 = fd2
   - close fd1 (not removed from epoll since we've dup'ed the fd)
   - dup fd2 = fd1 (get the same fd as fd1)
   - add fd1 to epoll = EEXIST

So, if fd can be potentially dup'ed of fork'ed, it's necessary to explicitly remove the fd from epoll set.

R=golang-dev, bradfitz, dave
CC=golang-dev
https://golang.org/cl/7870043

The problem is that new network poller can have spurious
rediness notifications. This implementation ensures that
the socket is actually connected.

R=golang-dev, rsc, akumar
CC=golang-dev
https://golang.org/cl/7785043

Introduce pollDesc struct, to split netFD struct into fd-related
and poller-related parts.

R=golang-dev, bradfitz, iant
CC=golang-dev
https://golang.org/cl/7762044

Move pollServer from fd_unix.go to fd_poll_unix.go.
Add pollServerInit(*NetFD) to allow custom initialization.
Add pollServer.Close(*NetFD) to allow custom finalization.
Move setDeadline() to fd_poll_unix.go to allow custom handling of deadlines.
Move newPollServer() to fd_poll_unix.go to allow custom initialization.
No logical code changes.
The next step will be to turn off fd_poll_unix.go for some platform
(I have changes for darwin/linux) and redirect it into runtime. See:
https://golang.org/cl/7569043/diff/2001/src/pkg/net/fd_poll_runtime.go

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/7513045

Ensure that accept/connect respect deadline,
even if the operation can be executed w/o blocking.
Note this changes external behavior, but it makes
it consistent with read/write.
Factor out deadline check into pollServer.PrepareRead/Write,
in preparation for edge triggered pollServer.
Ensure that pollServer.WaitRead/Write are not called concurrently
by adding rio/wio locks around connect/accept.

R=golang-dev, mikioh.mikioh, bradfitz, iant
CC=golang-dev
https://golang.org/cl/7436048