- 27 Oct, 2017 5 commits
-
-
Kirill Smelkov authored
This tool analyzes `go tool trace -d` output and take notices on ProcStart and GoStart events to track G->P->M relation to check how often a G changes M. Unfortunately it seems to be a frequent event and changing M probably means changing CPU and thus loosing CPU caches.
-
Kirill Smelkov authored
This patch adds pipenet - virtual network of net.Pipes. Addresses on pipenet are host:port pairs. A host is xnet.Networker and so can be worked with similarly to regular TCP network with Dial/Listen/Accept/... Example: net := pipenet.New("") h1 := net.Host("abc") h2 := net.Host("def") l, err := h1.Listen(":10") // starts listening on address "abc:10" go func() { csrv, err := l.Accept() // csrv will have LocalAddr "abc:10" }() ccli, err := h2.Dial("abc:10") // ccli will have RemoteAddr "def:10" Pipenet might be handy for testing interaction of networked applications in 1 process without going to OS networking stack.
-
Kirill Smelkov authored
This patch adds xnet.NetTrace which wraps a networker calling notification functions on Connect/Listen/Tx events. The code is draft and I'm not sure adding this functionality is good idea, but I still add it for completeness and because there is one user for it. Please do not expect the interface of xnet tracing to be stable.
-
Kirill Smelkov authored
Std net.Conn already can represent both plain TCP connections and connections wrapped with TLS. However the connections itself need to be created differently. This might become inconvenient when establishing connections should be inside server logic where it is desirable to have one codepath which works uniformly via interfaces. This patch introduces Networker - new interface which represents access-point to a streaming network. One can Dial or Listen on it and get underlying network name. Two functions are also provided to create networkers for plain TCP and to wrap a networker with TLS layer. This way one can initially decide and setup a networker, pass it to server logic, and server inside uses just Networker interface transparently either listening/connecting via regular sockets, or via sockets wrapped with TLS layer.
-
Kirill Smelkov authored
XRun (added in db941f12) and Runx (added in 486ede30) run a function and convert error <-> exception back and forth. However sometimes it is handy to only convert a function but not run it - e.g. for passing into x/sync/errgroup.Group.Go To do so this patch adds XFunc and Funcx - functional counterparts to XRun and Runx. No new tests are needed because now XRun and Runx are just tiny wrappers around new functions and we already have tests for XRun and Runx.
-
- 25 Oct, 2017 5 commits
-
-
Kirill Smelkov authored
Race-detector does not know Probe.Detach works under world stopped and that this way it cannot break consistency of probes list attached to a trace event - on event signalling either a probe will be run or not run at all. And we do not mind that e.g. while Detach was in progress a probe was read from traceevent list and decided to be run and the probe function was actually called just after Detach finished. For this reason tell race-detector to not take into account all memory read/write that are performed while the world is stopped. If we do not it complains e.g. this way: ================== WARNING: DATA RACE Read at 0x00c42000d760 by goroutine 7: lab.nexedi.com/kirr/neo/go/zodb/storage._traceCacheGCFinish_run() /home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/xcommon/tracing/tracing.go:265 +0x81 lab.nexedi.com/kirr/neo/go/zodb/storage.traceCacheGCFinish() /home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/zodb/storage/ztrace.go:22 +0x63 lab.nexedi.com/kirr/neo/go/zodb/storage.(*Cache).gc() /home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/zodb/storage/cache.go:497 +0x62c lab.nexedi.com/kirr/neo/go/zodb/storage.(*Cache).gcmain() /home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/zodb/storage/cache.go:478 +0x4c Previous write at 0x00c42000d760 by goroutine 6: lab.nexedi.com/kirr/neo/go/xcommon/tracing.(*Probe).Detach() /home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/xcommon/tracing/tracing.go:319 +0x103 lab.nexedi.com/kirr/neo/go/xcommon/tracing.(*ProbeGroup).Done() /home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/xcommon/tracing/tracing.go:344 +0xa5 lab.nexedi.com/kirr/neo/go/zodb/storage.TestCache() /home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/zodb/storage/cache_test.go:576 +0x7f94 testing.tRunner() /home/kirr/src/tools/go/go/src/testing/testing.go:746 +0x16c Goroutine 7 (running) created at: lab.nexedi.com/kirr/neo/go/zodb/storage.NewCache() /home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/zodb/storage/cache.go:129 +0x227 lab.nexedi.com/kirr/neo/go/zodb/storage.TestCache() /home/kirr/src/neo/src/lab.nexedi.com/kirr/neo/go/zodb/storage/cache_test.go:165 +0x7b1 testing.tRunner() /home/kirr/src/tools/go/go/src/testing/testing.go:746 +0x16c Goroutine 6 (finished) created at: testing.(*T).Run() /home/kirr/src/tools/go/go/src/testing/testing.go:789 +0x568 testing.runTests.func1() /home/kirr/src/tools/go/go/src/testing/testing.go:1004 +0xa7 testing.tRunner() /home/kirr/src/tools/go/go/src/testing/testing.go:746 +0x16c testing.runTests() /home/kirr/src/tools/go/go/src/testing/testing.go:1002 +0x521 testing.(*M).Run() /home/kirr/src/tools/go/go/src/testing/testing.go:921 +0x206 main.main() lab.nexedi.com/kirr/neo/go/zodb/storage/_test/_testmain.go:44 +0x1d3 ==================
-
Kirill Smelkov authored
As it was said in the previous patch here goes gotrace utility to process `//trace:event ...` and other tracing related directives. Related excerpt from the documentation: ---- 8< ---- Gotrace The way //trace:event and //trace:import works is via additional code being generated for them. Whenever a package uses any //trace: directive, it has to organize to run `gotrace gen` on its sources for them to work, usually with the help of //go:generate. For example: package hello //go:generate gotrace gen . //trace:event ... Besides `gotrace gen` gotrace has other subcommands also related to tracing, for example `gotrace list` lists trace events a package provides. ---- 8< ---- Gotrace works by parsing and typechecking go sources via go/parse & go/types and then for special comments generating corresponding additional code that is supported by tracing runtime.
-
Kirill Smelkov authored
Package tracing will provide usage and runtime support for Linux-style traceevents/tracepoints for Go. This patch comes with the runtime support to attach/detach probes to/from trace-events and stop/restart the world so that attaching/detaching can be done safely while nothing else is running. Having attach/detach under STW allows regular probe list iteration to be done without locks. The next patch will add gotrace utility which automatically turns //trace:event in-source comments into proper trace-event definitions. Below is excerpt from tracing usage. Please refer to tracing.go for full text. ---- 8< ---- Package tracing provides usage and runtime support for Go tracing facilities. Trace events A Go package can define several events of interest to trace via special comments. With such definition a tracing event becomes associated with trace function that is used to signal when the event happens. For example: package hello //trace:event traceHelloPre(who string) //trace:event traceHello(who string) func SayHello(who string) { traceHelloPre(who) fmt.Println("Hello, %s", who) traceHello(who) } By default trace function does nothing and has very small overhead. Probes However it is possible to attach probing functions to events. A probe, once attached, is called whenever event is signalled in the context which triggered the event and pauses original code execution until the probe is finished. It is possible to attach several probing functions to the same event and dynamically detach/(re-)attach them at runtime. Attaching/detaching probes must be done under tracing.Lock. For example: type saidHelloT struct { who string when time.Time } saidHello := make(chan saidHelloT) tracing.Lock() p := traceHello_Attach(nil, func(who string) { saidHello <- saidHelloT{who, time.Now()} }) tracing.Unlock() go func() { for hello := range saidHello { fmt.Printf("Said hello to %v @ %v\n", hello.who, hello.when) } }() SayHello("JP") SayHello("Kirr") SayHello("Varya") tracing.Lock() p.Detach() tracing.Unlock() close(saidHello) For convenience it is possible to keep group of attached probes and detach them all at once using ProbeGroup: pg := &tracing.ProbeGroup{} tracing.Lock() traceHelloPre_Attach(pg, func(who string) { ... }) traceHello_Attach(pg, func(who string) { ... }) tracing.Unlock() // some activity // when probes needs to be detached (no explicit tracing.Lock needed): pg.Done() Probes is general mechanism which allows various kinds of trace events usage. Three ways particularly are well-understood and handy: - recording events stream - profiling - synchronous tracing ...
-
Kirill Smelkov authored
Initial draft. The implementation is modelled after `git` and `go`.
-
Kirill Smelkov authored
4d9a613c (Relicense to GPLv3+ with wide exception for all Free Software / Open Source projects + Business options.) added more lines to my/my_test.go than removed (@@ -5,16 +5,18 @@) so the line number of myline := Line() changed by 2 (= 18 - 16). Fix the test accordingly.
-
- 24 Oct, 2017 1 commit
-
-
Kirill Smelkov authored
Relicense to GPLv3+ with wide exception for all Free Software / Open Source projects + Business options. Nexedi stack is licensed under Free Software licenses with various exceptions that cover three business cases: - Free Software - Proprietary Software - Rebranding As long as one intends to develop Free Software based on Nexedi stack, no license cost is involved. Developing proprietary software based on Nexedi stack may require a proprietary exception license. Rebranding Nexedi stack is prohibited unless rebranding license is acquired. Through this licensing approach, Nexedi expects to encourage Free Software development without restrictions and at the same time create a framework for proprietary software to contribute to the long term sustainability of the Nexedi stack. Please see https://www.nexedi.com/licensing for details, rationale and options.
-
- 11 Sep, 2017 1 commit
-
-
Kirill Smelkov authored
CeilLog2(x) = min i: 2^i >= x FloorLog2(x) = max i: 2^i <= x
-
- 08 Aug, 2017 1 commit
-
-
Kirill Smelkov authored
This way original error can still be retrieved via errors.Cause()
-
- 25 Jul, 2017 1 commit
-
-
Kirill Smelkov authored
This continues c0bbd06e (xfmt: Qpy & friends to quote string the way Python would do).
-
- 19 Jul, 2017 1 commit
-
-
Kirill Smelkov authored
This are handy utilities to reduce several errors into only 1 either picking the first error or merging, if there are several, into Errorv. It is unfortunate but an issue with Errorv was realized that even though it satisfies the error interface, it cannot be generally worked with as error because it (being []error) is uncomparable. Thus e.g. the following code, if err dynamic type is Errorv, will panic: if err == io.EOF It is pity Go slices are uncomparable.
-
- 07 Jun, 2017 1 commit
-
-
Kirill Smelkov authored
This are handy utilities to automatically prepend context on error return. For example in func myfunc(...) (..., err error) { defer xerr.Context(&err, "error context") ... if ... { return ..., errors.New("an error") } return ..., nil } while preserving nil error return on successful execution, myfunc will return error with string "error context: an error" on failure case.
-
- 20 Apr, 2017 4 commits
-
-
Kirill Smelkov authored
This is somtimes needed for checking programs output bit-to-bit where on python side repr(x), `x` or %r is used for output.
-
Kirill Smelkov authored
Std fmt works ok unless you need to do text formatting in hot codepaths. There fmt becomes inappropriate as it is slow and does allocations on every formatting. strconv also does not have append routines for every needed case, e.g. there is no strconv.AppendRune, no strconv.AppendHex etc. So xfmt 1. provides append routines for builtin types lacking in strconv 2. introduces xfmt.Stringer interface which types can implement to hook into general formatting via xfmt.Append() 3. provides xfmt.Buffer which is []byte with syntatic sugar for formatting in a way similar to printf: For example if in fmt speak you have s := fmt.Sprintf("hello %q %d %x", "world", 1, []byte("data")) xfmt analog would be buf := xfmt.Buffer{} buf .S("hello ") .Q("world") .C(' ') .D(1) .C(' ') .Xb([]byte("data")) s := buf.Bytes() and xfmt.Buffer can be reused several times via Buffer.Reset() . The above xfmt.Buffer usage is more uglier than fmt.Printf but much less uglier than direct strconv.Append* and friends calls, and works faster and without allocations compared to fmt.Printf: BenchmarkXFmt/%c(0x41)-4 20000000 65.4 ns/op 1 B/op 1 allocs/op BenchmarkXFmt/.Cb(0x41)-4 200000000 5.96 ns/op 0 B/op 0 allocs/op BenchmarkXFmt/%c(-1)-4 20000000 70.1 ns/op 3 B/op 1 allocs/op BenchmarkXFmt/.C(-1)-4 100000000 12.9 ns/op 0 B/op 0 allocs/op BenchmarkXFmt/%c(66)-4 20000000 65.8 ns/op 1 B/op 1 allocs/op BenchmarkXFmt/.C(66)-4 100000000 12.7 ns/op 0 B/op 0 allocs/op BenchmarkXFmt/%c(1080)-4 20000000 67.2 ns/op 2 B/op 1 allocs/op BenchmarkXFmt/.C(1080)-4 100000000 12.8 ns/op 0 B/op 0 allocs/op BenchmarkXFmt/%c(8364)-4 20000000 69.4 ns/op 3 B/op 1 allocs/op BenchmarkXFmt/.C(8364)-4 100000000 13.8 ns/op 0 B/op 0 allocs/op BenchmarkXFmt/%c(65537)-4 20000000 70.5 ns/op 4 B/op 1 allocs/op BenchmarkXFmt/.C(65537)-4 100000000 14.3 ns/op 0 B/op 0 allocs/op BenchmarkXFmt/%s("hello")-4 20000000 72.3 ns/op 5 B/op 1 allocs/op BenchmarkXFmt/.S("hello")-4 200000000 9.40 ns/op 0 B/op 0 allocs/op ...
-
Kirill Smelkov authored
- small addons over std bytes package: xbytes.ContainsByte - (re)allocation routines for byte slices
-
Kirill Smelkov authored
So far only one of them: CeilPow2 to return min(y) >= x: y = 2^i - i.e. next power of two >= x. This is handy to have in reallocation routines to allocate buffers from 2^i classes. Two implementations: - fast for go19 - slower fallback for go18
-
- 19 Apr, 2017 4 commits
-
-
Kirill Smelkov authored
As it was planned switch to using runtime.Frame instead of our local imitation. Add test to make sure we are not breaking anything. Adjust users in exc.
-
Kirill Smelkov authored
To determine current function's file name, line number and runtime.Frame
-
Kirill Smelkov authored
Because in general case runtime.CallersFrames is more accurate than runtime.FuncForPC - e.g. the latter does not correctly work with inlined functions.
-
Kirill Smelkov authored
This causes the following changes on client side: myname.Func -> my.FuncName myname.Pkg -> my.PkgName Reason for the change is that we are going to introduce my.File and my.Line which would not fit into myname package naming.
-
- 29 Mar, 2017 1 commit
-
-
Kirill Smelkov authored
We want to make sure that the code can be used by Free Software and Open Source projects without a problem. So change the license to be GPLv3+ with wide exception for all open-source licenses which practically cover them all.
-
- 06 Mar, 2017 1 commit
-
-
Kirill Smelkov authored
-
- 03 Mar, 2017 1 commit
-
-
Kirill Smelkov authored
exc += XRun() - utility to run a function which returns regular error, and raise exception if error is not nil This might be frequently needed in tests, possibly with some kind of "apply" syntatic sugar. Complements 486ede30 (exc += Runx() - utility to run a function and translate caught exception to regular error)
-
- 14 Dec, 2016 6 commits
-
-
Kirill Smelkov authored
Previously Error() was giving "0 errors", but "" seems to be a better choice. Anyway proper usage of Errorv is to call .Err() when handing result to some where so this way empty-vector case becomes nil.
-
Kirill Smelkov authored
- various .Append*() are functions to add errors to error vector. - .Err() returns error in canonical form accumulated in error vector - e.g. nil when error vector is empty.
-
Kirill Smelkov authored
Name for error vector: ev -> errv
-
Kirill Smelkov authored
This might be frequently needed when running a goroutine - to catch errors at top level and then convey they to other places as regular errors.
-
Kirill Smelkov authored
-
Kirill Smelkov authored
Fix this thinko in Addcallingcontext description. In current Error.Error() only function names are printed but with actual frames there in error context clients will be probably able to access this information theirselves.
-
- 13 Dec, 2016 7 commits
-
-
Kirill Smelkov authored
-
Kirill Smelkov authored
Because not only go.git wants to increase verbosity on `-v -v ...`. Code taken from go.git ec80737bdf.
-
Kirill Smelkov authored
No many details since they can be all viewed e.g. with help of godoc. Just a top-level pointer.
-
Kirill Smelkov authored
-
Kirill Smelkov authored
Move them here to go123 like in previous patches. Taken from git-backup 0be1f647. [1] https://lab.nexedi.com/kirr/git-backup
-
Kirill Smelkov authored
Pristine import + package/copyright clauses. Taken from git-backup 3aedc246. [1] https://lab.nexedi.com/kirr/git-backup
-
Kirill Smelkov authored
Export functions that client would use and adjust their names taking into account that now there is a package prefix. So e.g. raise -> exc.Raise errcatch -> exc.Catch erronunwind -> exc.Onunwind ...
-