Commit b34a88ce authored by Andrew Newdigate's avatar Andrew Newdigate

Vendor LabKit into GitLab-Shell

Done using `govendor fetch -v gitlab.com/gitlab-org/labkit/...@master`
parent 39262b39
# Changelog
## [Pending Release](https://github.com/lightstep/lightstep-tracer-go/compare/v0.15.6...HEAD)
* Thrift transport is now removed.
* Requires go >= 1.7
* Imports `context` via the standard library instead of `golang.org/x/net/context`
* Fixes [#182](https://github.com/lightstep/lightstep-tracer-go/issues/182), so that `StartSpan` can now take `SpanReference`s to non-LightStep `SpanContext`s use
## [v0.15.6](https://github.com/lightstep/lightstep-tracer-go/compare/v0.15.5...v0.15.6)
* Minor update to `sendspan` to make it easier to pick which transport protocol to use.
* Add a new field to Options: DialOptions. These allow setting custom grpc dial options when using grpc.
* This is necessary to have customer balancers or interceptors.
* DialOptions shouldn't be set unless it is needed, it will default correctly.
* Added a new field to Endpoint: Scheme. Scheme can be used to override the default schemes (http/https) or set a custom scheme (for grpc).
* This is necessary for using custom grpc resolvers.
* If callers are using struct construction without field names (i.e. Endpoint{"host", port, ...}), they will need to add a new field (scheme = "").
* Scheme shouldn't be set unless it needs to overridden, it will default correctly.
## [v0.15.5](https://github.com/lightstep/lightstep-tracer-go/compare/v0.15.4...v0.15.5)
* Internal performance optimizations and a bug fix for issue [#161](https://github.com/lightstep/lightstep-tracer-go/issues/161)
## [v0.15.4](https://github.com/lightstep/lightstep-tracer-go/compare/v0.15.3...v0.15.4)
* This change affects LightStep's internal testing, not a functional change.
## [v0.15.3](https://github.com/lightstep/lightstep-tracer-go/compare/v0.15.2...v0.15.3)
* Adds compatibility for io.Writer and io.Reader in Inject/Extract, as required by Open Tracing.
## [v0.15.2](https://github.com/lightstep/lightstep-tracer-go/compare/v0.15.1...v0.15.2)
* Adds lightstep.GetLightStepReporterID.
## [v0.15.1](https://github.com/lightstep/lightstep-tracer-go/compare/v0.15.0...v0.15.1)
* Adds Gopkg.toml
## [v0.15.0](https://github.com/lightstep/lightstep-tracer-go/compare/v0.14.0...v0.15.0)
* We are replacing the internal diagnostic logging with a more flexible “Event” framework. This enables you to track and understand tracer problems with metrics and logging tools of your choice.
* We are also changed the types of the Close() and Flush() methods to take a context parameter to support cancellation. These changes are *not* backwards compatible and *you will need to update your instrumentation*. We are providing a NewTracerv0_14() method that is a drop-in replacement for the previous version.
## [v0.14.0](https://github.com/lightstep/lightstep-tracer-go/compare/v0.13.0...v0.14.0)
* Flush buffer syncronously on Close.
* Flush twice if a flush is already in flight.
* Remove gogo in favor of golang/protobuf.
* Requires grpc-go >= 1.4.0.
## [v0.13.0](https://github.com/lightstep/lightstep-tracer-go/compare/v0.12.0...v0.13.0)
* BasicTracer has been removed.
* Tracer now takes a SpanRecorder as an option.
* Tracer interface now includes Close and Flush.
* Tests redone with ginkgo/gomega.
## [v0.12.0](https://github.com/lightstep/lightstep-tracer-go/compare/v0.11.0...v0.12.0)
* Added CloseTracer function to flush and close a lightstep recorder.
## [v0.11.0](https://github.com/lightstep/lightstep-tracer-go/compare/v0.10.0...v0.11.0)
* Thrift transport is now deprecated, gRPC is the default.
# This file is autogenerated, do not edit; changes may be undone by the next 'dep ensure'.
[[projects]]
branch = "master"
digest = "1:3b760d3b93f994df8eb1d9ebfad17d3e9e37edcb7f7efaa15b427c0d7a64f4e4"
name = "github.com/golang/protobuf"
packages = [
"proto",
"protoc-gen-go/descriptor",
"ptypes",
"ptypes/any",
"ptypes/duration",
"ptypes/timestamp",
]
pruneopts = ""
revision = "1e59b77b52bf8e4b449a57e6f79f21226d571845"
[[projects]]
digest = "1:32b27072cd55bd2fb7244de0425943d125da6a552ae2b6517cdd965a662baf18"
name = "github.com/onsi/ginkgo"
packages = [
".",
"config",
"internal/codelocation",
"internal/containernode",
"internal/failer",
"internal/leafnodes",
"internal/remote",
"internal/spec",
"internal/spec_iterator",
"internal/specrunner",
"internal/suite",
"internal/testingtproxy",
"internal/writer",
"reporters",
"reporters/stenographer",
"reporters/stenographer/support/go-colorable",
"reporters/stenographer/support/go-isatty",
"types",
]
pruneopts = ""
revision = "9eda700730cba42af70d53180f9dcce9266bc2bc"
version = "v1.4.0"
[[projects]]
digest = "1:a4e59d0b2821c983b58c317f141cd77df20570979632da8a7a352e5d12698de7"
name = "github.com/onsi/gomega"
packages = [
".",
"format",
"internal/assertion",
"internal/asyncassertion",
"internal/oraclematcher",
"internal/testingtsupport",
"matchers",
"matchers/support/goraph/bipartitegraph",
"matchers/support/goraph/edge",
"matchers/support/goraph/node",
"matchers/support/goraph/util",
"types",
]
pruneopts = ""
revision = "c893efa28eb45626cdaa76c9f653b62488858837"
version = "v1.2.0"
[[projects]]
digest = "1:78fb99d6011c2ae6c72f3293a83951311147b12b06a5ffa43abf750c4fab6ac5"
name = "github.com/opentracing/opentracing-go"
packages = [
".",
"log",
]
pruneopts = ""
revision = "1949ddbfd147afd4d964a9f00b24eb291e0e7c38"
version = "v1.0.2"
[[projects]]
branch = "master"
digest = "1:950b672f2ee80d0fc4c95a15a976ba9ee573a6fb8ede8a777770b2230776367e"
name = "golang.org/x/net"
packages = [
"context",
"html",
"html/atom",
"html/charset",
"http2",
"http2/hpack",
"idna",
"internal/timeseries",
"lex/httplex",
"trace",
]
pruneopts = ""
revision = "dc871a5d77e227f5bbf6545176ef3eeebf87e76e"
[[projects]]
branch = "master"
digest = "1:5da11ab130476e2736f32140f3c1aed2a3a96e9ba8963711a7d38db783d042bd"
name = "golang.org/x/sys"
packages = ["unix"]
pruneopts = ""
revision = "a0f4589a76f1f83070cb9e5613809e1d07b97c13"
[[projects]]
branch = "master"
digest = "1:1c70f7bb89783a026dc32920575a3feef48e065ef6e170ad227903e8194d7a36"
name = "golang.org/x/text"
packages = [
"collate",
"collate/build",
"encoding",
"encoding/charmap",
"encoding/htmlindex",
"encoding/internal",
"encoding/internal/identifier",
"encoding/japanese",
"encoding/korean",
"encoding/simplifiedchinese",
"encoding/traditionalchinese",
"encoding/unicode",
"internal/colltab",
"internal/gen",
"internal/tag",
"internal/triegen",
"internal/ucd",
"internal/utf8internal",
"language",
"runes",
"secure/bidirule",
"transform",
"unicode/bidi",
"unicode/cldr",
"unicode/norm",
"unicode/rangetable",
]
pruneopts = ""
revision = "be25de41fadfae372d6470bda81ca6beb55ef551"
[[projects]]
branch = "master"
digest = "1:6c15114fafeac4c833544476dec4207a3476799d50ab5165af79eb97806884cf"
name = "google.golang.org/genproto"
packages = [
"googleapis/api/annotations",
"googleapis/rpc/status",
]
pruneopts = ""
revision = "7f0da29060c682909f650ad8ed4e515bd74fa12a"
[[projects]]
digest = "1:6c00b4702c146631d30396090d40bfc0486f8b3af5c976d6f0daf2bc737cd7b2"
name = "google.golang.org/grpc"
packages = [
".",
"balancer",
"balancer/roundrobin",
"codes",
"connectivity",
"credentials",
"encoding",
"grpclb/grpc_lb_v1/messages",
"grpclog",
"internal",
"keepalive",
"metadata",
"naming",
"peer",
"resolver",
"resolver/dns",
"resolver/manual",
"resolver/passthrough",
"stats",
"status",
"tap",
"transport",
]
pruneopts = ""
revision = "be077907e29fdb945d351e4284eb5361e7f8924e"
version = "v1.8.1"
[[projects]]
branch = "v2"
digest = "1:f769ed60e075e4221612c2f4162fccc9d3795ef358fa463425e3b3d7a5debb27"
name = "gopkg.in/yaml.v2"
packages = ["."]
pruneopts = ""
revision = "287cf08546ab5e7e37d55a84f7ed3fd1db036de5"
[solve-meta]
analyzer-name = "dep"
analyzer-version = 1
input-imports = [
"github.com/golang/protobuf/proto",
"github.com/golang/protobuf/ptypes/timestamp",
"github.com/onsi/ginkgo",
"github.com/onsi/gomega",
"github.com/onsi/gomega/types",
"github.com/opentracing/opentracing-go",
"github.com/opentracing/opentracing-go/log",
"golang.org/x/net/context",
"google.golang.org/genproto/googleapis/api/annotations",
"google.golang.org/grpc",
"google.golang.org/grpc/credentials",
]
solver-name = "gps-cdcl"
solver-version = 1
[[constraint]]
name = "github.com/onsi/ginkgo"
version = "1.4.0"
[[constraint]]
name = "github.com/onsi/gomega"
version = "1.2.0"
[[constraint]]
name = "github.com/opentracing/opentracing-go"
version = "1.0.2"
[[constraint]]
branch = "master"
name = "golang.org/x/net"
[[constraint]]
name = "google.golang.org/grpc"
version = "1.4.3"
The MIT License (MIT)
Copyright (c) 2016
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
# tools
GO=go
DOCKER_PRESENT = $(shell command -v docker 2> /dev/null)
LOCAL_GOPATH = $(PWD)/../../../../
default: build
.PHONY: default build test
# generate_fake: runs counterfeiter in docker container to generate fake classes
# $(1) output file path
# $(2) input file path
# $(3) class name
define generate_fake
docker run --rm -v $(LOCAL_GOPATH):/usergo \
lightstep/gobuild:latest /bin/bash -c "\
cd /usergo/src/github.com/lightstep/lightstep-tracer-go; \
counterfeiter -o $(1) $(2) $(3)"
endef
collectorpb/collectorpbfakes/fake_collector_service_client.go: collectorpb/collector.pb.go
$(call generate_fake,collectorpb/collectorpbfakes/fake_collector_service_client.go,collectorpb/collector.pb.go,CollectorServiceClient)
# gRPC
ifeq (,$(wildcard lightstep-tracer-common/collector.proto))
collectorpb/collector.pb.go:
else
collectorpb/collector.pb.go: lightstep-tracer-common/collector.proto
docker run --rm -v $(shell pwd)/lightstep-tracer-common:/input:ro -v $(shell pwd)/collectorpb:/output \
lightstep/grpc-gateway:latest \
protoc -I/root/go/src/tmp/vendor/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis --go_out=plugins=grpc:/output --proto_path=/input /input/collector.proto
endif
# gRPC
ifeq (,$(wildcard lightstep-tracer-common/collector.proto))
lightsteppb/lightstep.pb.go:
else
lightsteppb/lightstep.pb.go: lightstep-tracer-common/lightstep.proto
docker run --rm -v $(shell pwd)/lightstep-tracer-common:/input:ro -v $(shell pwd)/lightsteppb:/output \
lightstep/protoc:latest \
protoc --go_out=plugins=grpc:/output --proto_path=/input /input/lightstep.proto
endif
test: collectorpb/collector.pb.go lightsteppb/lightstep.pb.go \
collectorpb/collectorpbfakes/fake_collector_service_client.go \
lightstepfakes/fake_recorder.go
ifeq ($(DOCKER_PRESENT),)
$(error "docker not found. Please install from https://www.docker.com/")
endif
docker run --rm -v $(LOCAL_GOPATH):/usergo lightstep/gobuild:latest \
ginkgo -race -p /usergo/src/github.com/lightstep/lightstep-tracer-go
bash -c "! git grep -q '[g]ithub.com/golang/glog'"
build: collectorpb/collector.pb.go lightsteppb/lightstep.pb.go \
collectorpb/collectorpbfakes/fake_collector_service_client.go version.go \
lightstepfakes/fake_recorder.go
ifeq ($(DOCKER_PRESENT),)
$(error "docker not found. Please install from https://www.docker.com/")
endif
${GO} build github.com/lightstep/lightstep-tracer-go
# When releasing significant changes, make sure to update the semantic
# version number in `./VERSION`, merge changes, then run `make release_tag`.
version.go: VERSION
./tag_version.sh
release_tag:
git tag -a v`cat ./VERSION`
git push origin v`cat ./VERSION`
# lightstep-tracer-go
[![Circle CI](https://circleci.com/gh/lightstep/lightstep-tracer-go.svg?style=shield)](https://circleci.com/gh/lightstep/lightstep-tracer-go)
[![MIT license](http://img.shields.io/badge/license-MIT-blue.svg)](http://opensource.org/licenses/MIT)
[![GoDoc](https://godoc.org/github.com/lightstep/lightstep-tracer-go?status.svg)](https://godoc.org/github.com/lightstep/lightstep-tracer-go)
The LightStep distributed tracing library for Go.
## Installation
```
$ go get 'github.com/lightstep/lightstep-tracer-go'
```
## API Documentation
Godoc: https://godoc.org/github.com/lightstep/lightstep-tracer-go
## Initialization: Starting a new tracer
To initialize a tracer, configure it with a valid Access Token and optional tuning parameters. Register the tracer as the OpenTracing global tracer so that it will become available to your installed intstrumentations libraries.
```go
import (
"github.com/opentracing/opentracing-go"
"github.com/lightstep/lightstep-tracer-go"
)
func main() {
lightstepTracer := lightstep.NewTracer(lightstep.Options{
AccessToken: "YourAccessToken",
})
opentracing.SetGlobalTracer(lightstepTracer)
}
```
## Instrumenting Code: Using the OpenTracing API
All instrumentation should be done through the OpenTracing API, rather than using the lightstep tracer type directly. For API documentation and advice on instrumentation in general, see the opentracing godocs and the opentracing website.
- https://godoc.org/github.com/opentracing/opentracing-go
- http://opentracing.io
## Flushing and Closing: Managing the tracer lifecycle
As part of managaing your application lifecycle, the lightstep tracer extends the `opentracing.Tracer` interface with methods for manual flushing and closing. To access these methods, you can take the global tracer and typecast it to a `lightstep.Tracer`. As a convenience, the lightstep package provides static methods which perform the typecasting.
```go
import (
"context"
"github.com/opentracing/opentracing-go"
"github.com/lightstep/lightstep-tracer-go"
)
func shutdown(ctx context.Context) {
// access the running tracer
tracer := opentracing.GlobalTracer()
// typecast from opentracing.Tracer to lightstep.Tracer
lsTracer, ok := tracer.(lightstep.Tracer)
if (!ok) {
return
}
lsTracer.Close(ctx)
// or use static methods
lightstep.Close(ctx, tracer)
}
```
## Event Handling: Observing the LightStep tracer
In order to connect diagnostic information from the lightstep tracer into an application's logging and metrics systems, inject an event handler using the `OnEvent` static method. Events may be typecast to check for errors or specific events such as status reports.
```go
import (
"example/logger"
"example/metrics"
"github.com/lightstep/lightstep-tracer-go"
)
logAndMetricsHandler := func(event lightstep.Event){
switch event := event.(type) {
case EventStatusReport:
metrics.Count("tracer.dropped_spans", event.DroppedSpans())
case ErrorEvent:
logger.Error("LS Tracer error: %s", event)
default:
logger.Info("LS Tracer info: %s", event)
}
}
func main() {
// setup event handler first to catch startup errors
lightstep.SetGlobalEventHandler(logAndMetricsHandler)
lightstepTracer := lightstep.NewTracer(lightstep.Options{
AccessToken: "YourAccessToken",
})
opentracing.SetGlobalTracer(lightstepTracer)
}
```
Event handlers will receive events from any active tracers, as well as errors in static functions. It is suggested that you set up event handling before initializing your tracer to catch any errors on initialization.
## Advanced Configuration: Transport and Serialization Protocols
By default, the tracer will send information to LightStep using GRPC and Protocol Buffers which is the recommended configuration. If there are no specific transport protocol needs you have, there is no need to change this default.
There are three total options for transport protocols:
- [Protocol Buffers](https://developers.google.com/protocol-buffers/) over [GRPC](https://grpc.io/) - The recommended, default, and most performant solution.
- \[ EXPERIMENTAL \] [Protocol Buffers](https://developers.google.com/protocol-buffers/) over HTTP - New transport protocol supported for use cases where GRPC isn't an option. In order to enable HTTP you will need to configure the LightStep collectors receiving the data to accept HTTP traffic. Reach out to LightStep for support in this.
You can configure which transport protocol the tracer uses using the `UseGRPC`, and `UseHttp` flags in the options.
\ No newline at end of file
package lightstep
import (
"context"
"io"
"net/http"
cpb "github.com/lightstep/lightstep-tracer-go/collectorpb"
)
// Connection describes a closable connection. Exposed for testing.
type Connection interface {
io.Closer
}
// ConnectorFactory is for testing purposes.
type ConnectorFactory func() (interface{}, Connection, error)
// collectorResponse encapsulates internal grpc/http responses.
type collectorResponse interface {
GetErrors() []string
Disable() bool
DevMode() bool
}
type reportRequest struct {
protoRequest *cpb.ReportRequest
httpRequest *http.Request
}
// collectorClient encapsulates internal grpc/http transports.
type collectorClient interface {
Report(context.Context, reportRequest) (collectorResponse, error)
Translate(context.Context, *reportBuffer) (reportRequest, error)
ConnectClient() (Connection, error)
ShouldReconnect() bool
}
func newCollectorClient(opts Options, reporterID uint64, attributes map[string]string) (collectorClient, error) {
if opts.UseHttp {
return newHTTPCollectorClient(opts, reporterID, attributes)
}
if opts.UseGRPC {
return newGrpcCollectorClient(opts, reporterID, attributes), nil
}
// No transport specified, defaulting to GRPC
return newGrpcCollectorClient(opts, reporterID, attributes), nil
}
package lightstep
import (
"context"
"fmt"
"reflect"
"time"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
// N.B.(jmacd): Do not use google.golang.org/glog in this package.
cpb "github.com/lightstep/lightstep-tracer-go/collectorpb"
)
const (
spansDropped = "spans.dropped"
logEncoderErrors = "log_encoder.errors"
)
var (
intType = reflect.TypeOf(int64(0))
)
// grpcCollectorClient specifies how to send reports back to a LightStep
// collector via grpc.
type grpcCollectorClient struct {
// auth and runtime information
attributes map[string]string
reporterID uint64
// accessToken is the access token used for explicit trace collection requests.
accessToken string
maxReportingPeriod time.Duration // set by GrpcOptions.MaxReportingPeriod
reconnectPeriod time.Duration // set by GrpcOptions.ReconnectPeriod
reportingTimeout time.Duration // set by GrpcOptions.ReportTimeout
// Remote service that will receive reports.
address string
grpcClient cpb.CollectorServiceClient
connTimestamp time.Time
dialOptions []grpc.DialOption
// converters
converter *protoConverter
// For testing purposes only
grpcConnectorFactory ConnectorFactory
}
func newGrpcCollectorClient(opts Options, reporterID uint64, attributes map[string]string) *grpcCollectorClient {
rec := &grpcCollectorClient{
attributes: attributes,
reporterID: reporterID,
accessToken: opts.AccessToken,
maxReportingPeriod: opts.ReportingPeriod,
reconnectPeriod: opts.ReconnectPeriod,
reportingTimeout: opts.ReportTimeout,
dialOptions: opts.DialOptions,
converter: newProtoConverter(opts),
grpcConnectorFactory: opts.ConnFactory,
}
if len(opts.Collector.Scheme) > 0 {
rec.address = opts.Collector.urlWithoutPath()
} else {
rec.address = opts.Collector.SocketAddress()
}
rec.dialOptions = append(rec.dialOptions, grpc.WithDefaultCallOptions(grpc.MaxCallSendMsgSize(opts.GRPCMaxCallSendMsgSizeBytes)))
if opts.Collector.Plaintext {
rec.dialOptions = append(rec.dialOptions, grpc.WithInsecure())
} else {
rec.dialOptions = append(rec.dialOptions, grpc.WithTransportCredentials(credentials.NewClientTLSFromCert(nil, "")))
}
return rec
}
func (client *grpcCollectorClient) ConnectClient() (Connection, error) {
now := time.Now()
var conn Connection
if client.grpcConnectorFactory != nil {
uncheckedClient, transport, err := client.grpcConnectorFactory()
if err != nil {
return nil, err
}
grpcClient, ok := uncheckedClient.(cpb.CollectorServiceClient)
if !ok {
return nil, fmt.Errorf("gRPC connector factory did not provide valid client")
}
conn = transport
client.grpcClient = grpcClient
} else {
transport, err := grpc.Dial(client.address, client.dialOptions...)
if err != nil {
return nil, err
}
conn = transport
client.grpcClient = cpb.NewCollectorServiceClient(transport)
}
client.connTimestamp = now
return conn, nil
}
func (client *grpcCollectorClient) ShouldReconnect() bool {
return time.Since(client.connTimestamp) > client.reconnectPeriod
}
func (client *grpcCollectorClient) Report(ctx context.Context, req reportRequest) (collectorResponse, error) {
if req.protoRequest == nil {
return nil, fmt.Errorf("protoRequest cannot be null")
}
resp, err := client.grpcClient.Report(ctx, req.protoRequest)
if err != nil {
return nil, err
}
return resp, nil
}
func (client *grpcCollectorClient) Translate(ctx context.Context, buffer *reportBuffer) (reportRequest, error) {
req := client.converter.toReportRequest(
client.reporterID,
client.attributes,
client.accessToken,
buffer,
)
return reportRequest{
protoRequest: req,
}, nil
}
package lightstep
import (
"bytes"
"context"
"fmt"
"io/ioutil"
"net"
"net/http"
"net/url"
"time"
"github.com/golang/protobuf/proto"
"github.com/lightstep/lightstep-tracer-go/collectorpb"
)
var (
acceptHeader = http.CanonicalHeaderKey("Accept")
contentTypeHeader = http.CanonicalHeaderKey("Content-Type")
)
const (
collectorHTTPMethod = "POST"
collectorHTTPPath = "/api/v2/reports"
protoContentType = "application/octet-stream"
)
// grpcCollectorClient specifies how to send reports back to a LightStep
// collector via grpc.
type httpCollectorClient struct {
// auth and runtime information
reporterID uint64
accessToken string // accessToken is the access token used for explicit trace collection requests.
attributes map[string]string
reportTimeout time.Duration
reportingPeriod time.Duration
// Remote service that will receive reports.
url *url.URL
client *http.Client
// converters
converter *protoConverter
}
type transportCloser struct {
*http.Transport
}
func (closer transportCloser) Close() error {
closer.CloseIdleConnections()
return nil
}
func newHTTPCollectorClient(
opts Options,
reporterID uint64,
attributes map[string]string,
) (*httpCollectorClient, error) {
url, err := url.Parse(opts.Collector.URL())
if err != nil {
fmt.Println("collector config does not produce valid url", err)
return nil, err
}
url.Path = collectorHTTPPath
return &httpCollectorClient{
reporterID: reporterID,
accessToken: opts.AccessToken,
attributes: attributes,
reportTimeout: opts.ReportTimeout,
reportingPeriod: opts.ReportingPeriod,
url: url,
converter: newProtoConverter(opts),
}, nil
}
func (client *httpCollectorClient) ConnectClient() (Connection, error) {
// Use a transport independent from http.DefaultTransport to provide sane
// defaults that make sense in the context of the lightstep client. The
// differences are mostly on setting timeouts based on the report timeout
// and period.
transport := &http.Transport{
Proxy: http.ProxyFromEnvironment,
DialContext: (&net.Dialer{
Timeout: client.reportTimeout / 2,
DualStack: true,
}).DialContext,
// The collector responses are very small, there is no point asking for
// a compressed payload, explicitly disabling it.
DisableCompression: true,
IdleConnTimeout: 2 * client.reportingPeriod,
TLSHandshakeTimeout: client.reportTimeout / 2,
ResponseHeaderTimeout: client.reportTimeout,
ExpectContinueTimeout: client.reportTimeout,
MaxResponseHeaderBytes: 64 * 1024, // 64 KB, just a safeguard
}
client.client = &http.Client{
Transport: transport,
Timeout: client.reportTimeout,
}
return transportCloser{transport}, nil
}
func (client *httpCollectorClient) ShouldReconnect() bool {
// http.Transport will handle connection reuse under the hood
return false
}
func (client *httpCollectorClient) Report(context context.Context, req reportRequest) (collectorResponse, error) {
if req.httpRequest == nil {
return nil, fmt.Errorf("httpRequest cannot be null")
}
httpResponse, err := client.client.Do(req.httpRequest.WithContext(context))
if err != nil {
return nil, err
}
defer httpResponse.Body.Close()
response, err := client.toResponse(httpResponse)
if err != nil {
return nil, err
}
return response, nil
}
func (client *httpCollectorClient) Translate(ctx context.Context, buffer *reportBuffer) (reportRequest, error) {
httpRequest, err := client.toRequest(ctx, buffer)
if err != nil {
return reportRequest{}, err
}
return reportRequest{
httpRequest: httpRequest,
}, nil
}
func (client *httpCollectorClient) toRequest(
context context.Context,
buffer *reportBuffer,
) (*http.Request, error) {
protoRequest := client.converter.toReportRequest(
client.reporterID,
client.attributes,
client.accessToken,
buffer,
)
buf, err := proto.Marshal(protoRequest)
if err != nil {
return nil, err
}
requestBody := bytes.NewReader(buf)
request, err := http.NewRequest(collectorHTTPMethod, client.url.String(), requestBody)
if err != nil {
return nil, err
}
request = request.WithContext(context)
request.Header.Set(contentTypeHeader, protoContentType)
request.Header.Set(acceptHeader, protoContentType)
return request, nil
}
func (client *httpCollectorClient) toResponse(response *http.Response) (collectorResponse, error) {
if response.StatusCode != http.StatusOK {
return nil, fmt.Errorf("status code (%d) is not ok", response.StatusCode)
}
body, err := ioutil.ReadAll(response.Body)
if err != nil {
return nil, err
}
protoResponse := &collectorpb.ReportResponse{}
if err := proto.Unmarshal(body, protoResponse); err != nil {
return nil, err
}
return protoResponse, nil
}
package lightstep
import (
"log"
"sync"
"sync/atomic"
)
func init() {
SetGlobalEventHandler(NewEventLogOneError())
}
var eventHandler atomic.Value
// An EventHandler can be registered with SetGlobalEventHandler to
type EventHandler func(Event)
// emitEvent is a thread-safe function for emiting tracer events.
func emitEvent(event Event) {
handler := eventHandler.Load().(EventHandler)
handler(event)
}
// SetGlobalEventHandler sets a global handler to receive tracer events as they occur. Events
// may be emitted by the tracer, or by calls to static functions in this package.
// It is suggested that you set your EventHandler before starting your tracer,
// but it is safe to set a new handler at any time. Setting a new handler removes
// the previous one.
//
// The EventHandler is called synchronously – do not block in your handler as it
// may interfere with the tracer. If no EventHandler handler is set, a
// LogOnceOnError handler is set by default.
//
// NOTE: Event handling is for reporting purposes only. It is not intended as a
// mechanism for controling or restarting the tracer. Connection issues, retry
// logic, and other transient errors are handled internally by the tracer.
func SetGlobalEventHandler(handler EventHandler) {
eventHandler.Store(handler)
}
/*
SetGlobalEventHandler Handlers
*/
// NewEventLogger logs events using the standard go logger.
func NewEventLogger() EventHandler {
return logOnEvent
}
func logOnEvent(event Event) {
switch event := event.(type) {
case ErrorEvent:
log.Println("LS Tracer error: ", event)
default:
log.Println("LS Tracer event: ", event)
}
}
// NewEventLogOneError logs the first error event that occurs.
func NewEventLogOneError() EventHandler {
logger := logOneError{}
return logger.OnEvent
}
type logOneError struct {
sync.Once
}
func (l *logOneError) OnEvent(event Event) {
switch event := event.(type) {
case ErrorEvent:
l.Once.Do(func() {
log.Printf("LS Tracer error: (%s). NOTE: Set the SetGlobalEventHandler handler to log events.\n", event.Error())
})
}
}
// NewEventChannel returns an SetGlobalEventHandler callback handler, and a channel that
// produces the errors. When the channel buffer is full, subsequent errors will
// be dropped. A buffer size of less than one is incorrect, and will be adjusted
// to a buffer size of one.
func NewEventChannel(buffer int) (EventHandler, <-chan Event) {
if buffer < 1 {
buffer = 1
}
eventChan := make(chan Event, buffer)
handler := func(event Event) {
select {
case eventChan <- event:
default:
}
}
return handler, eventChan
}
package lightstep
import (
"errors"
"fmt"
"reflect"
"time"
opentracing "github.com/opentracing/opentracing-go"
)
// An Event is emitted by the LightStep tracer as a reporting mechanism. They are
// handled by registering an EventHandler callback via SetGlobalEventHandler. The
// emitted events may be cast to specific event types in order access additional
// information.
//
// NOTE: To ensure that events can be accurately identified, each event type contains
// a sentinel method matching the name of the type. This method is a no-op, it is
// only used for type coersion.
type Event interface {
Event()
String() string
}
// The ErrorEvent type can be used to filter events. The `Err` method
// retuns the underlying error.
type ErrorEvent interface {
Event
error
Err() error
}
// EventStartError occurs if the Options passed to NewTracer are invalid, and
// the Tracer has failed to start.
type EventStartError interface {
ErrorEvent
EventStartError()
}
type eventStartError struct {
err error
}
func newEventStartError(err error) *eventStartError {
return &eventStartError{err: err}
}
func (*eventStartError) Event() {}
func (*eventStartError) EventStartError() {}
func (e *eventStartError) String() string {
return e.err.Error()
}
func (e *eventStartError) Error() string {
return e.err.Error()
}
func (e *eventStartError) Err() error {
return e.err
}
// EventFlushErrorState lists the possible causes for a flush to fail.
type EventFlushErrorState string
// Constant strings corresponding to flush errors
const (
FlushErrorTracerClosed EventFlushErrorState = "flush failed, the tracer is closed."
FlushErrorTracerDisabled EventFlushErrorState = "flush failed, the tracer is disabled."
FlushErrorTransport EventFlushErrorState = "flush failed, could not send report to Collector"
FlushErrorReport EventFlushErrorState = "flush failed, report contained errors"
FlushErrorTranslate EventFlushErrorState = "flush failed, could not translate report"
)
var (
errFlushFailedTracerClosed = errors.New(string(FlushErrorTracerClosed))
)
// EventFlushError occurs when a flush fails to send. Call the `State` method to
// determine the type of error.
type EventFlushError interface {
ErrorEvent
EventFlushError()
State() EventFlushErrorState
}
type eventFlushError struct {
err error
state EventFlushErrorState
}
func newEventFlushError(err error, state EventFlushErrorState) *eventFlushError {
return &eventFlushError{err: err, state: state}
}
func (*eventFlushError) Event() {}
func (*eventFlushError) EventFlushError() {}
func (e *eventFlushError) State() EventFlushErrorState {
return e.state
}
func (e *eventFlushError) String() string {
return e.err.Error()
}
func (e *eventFlushError) Error() string {
return e.err.Error()
}
func (e *eventFlushError) Err() error {
return e.err
}
// EventConnectionError occurs when the tracer fails to maintain it's connection
// with the Collector.
type EventConnectionError interface {
ErrorEvent
EventConnectionError()
}
type eventConnectionError struct {
err error
}
func newEventConnectionError(err error) *eventConnectionError {
return &eventConnectionError{err: err}
}
func (*eventConnectionError) Event() {}
func (*eventConnectionError) EventConnectionError() {}
func (e *eventConnectionError) String() string {
return e.err.Error()
}
func (e *eventConnectionError) Error() string {
return e.err.Error()
}
func (e *eventConnectionError) Err() error {
return e.err
}
// EventStatusReport occurs on every successful flush. It contains all metrics
// collected since the previous succesful flush.
type EventStatusReport interface {
Event
EventStatusReport()
StartTime() time.Time
FinishTime() time.Time
Duration() time.Duration
SentSpans() int
DroppedSpans() int
EncodingErrors() int
}
type eventStatusReport struct {
startTime time.Time
finishTime time.Time
sentSpans int
droppedSpans int
encodingErrors int
}
func newEventStatusReport(
startTime, finishTime time.Time,
sentSpans, droppedSpans, encodingErrors int,
) *eventStatusReport {
return &eventStatusReport{
startTime: startTime,
finishTime: finishTime,
sentSpans: sentSpans,
droppedSpans: droppedSpans,
encodingErrors: encodingErrors,
}
}
func (*eventStatusReport) Event() {}
func (*eventStatusReport) EventStatusReport() {}
func (s *eventStatusReport) SetSentSpans(sent int) {
s.sentSpans = sent
}
func (s *eventStatusReport) StartTime() time.Time {
return s.startTime
}
func (s *eventStatusReport) FinishTime() time.Time {
return s.finishTime
}
func (s *eventStatusReport) Duration() time.Duration {
return s.finishTime.Sub(s.startTime)
}
func (s *eventStatusReport) SentSpans() int {
return s.sentSpans
}
func (s *eventStatusReport) DroppedSpans() int {
return s.droppedSpans
}
func (s *eventStatusReport) EncodingErrors() int {
return s.encodingErrors
}
func (s *eventStatusReport) String() string {
return fmt.Sprint(
"STATUS REPORT start: ", s.startTime,
", end: ", s.finishTime,
", dropped spans: ", s.droppedSpans,
", encoding errors: ", s.encodingErrors,
)
}
// EventUnsupportedTracer occurs when a tracer being passed to a helper function
// fails to typecast as a LightStep tracer.
type EventUnsupportedTracer interface {
ErrorEvent
EventUnsupportedTracer()
Tracer() opentracing.Tracer
}
type eventUnsupportedTracer struct {
tracer opentracing.Tracer
err error
}
func newEventUnsupportedTracer(tracer opentracing.Tracer) EventUnsupportedTracer {
return &eventUnsupportedTracer{
tracer: tracer,
err: fmt.Errorf("unsupported tracer type: %v", reflect.TypeOf(tracer)),
}
}
func (e *eventUnsupportedTracer) Event() {}
func (e *eventUnsupportedTracer) EventUnsupportedTracer() {}
func (e *eventUnsupportedTracer) Tracer() opentracing.Tracer {
return e.tracer
}
func (e *eventUnsupportedTracer) String() string {
return e.err.Error()
}
func (e *eventUnsupportedTracer) Error() string {
return e.err.Error()
}
func (e *eventUnsupportedTracer) Err() error {
return e.err
}
// EventUnsupportedValue occurs when a tracer encounters an unserializable tag
// or log field.
type EventUnsupportedValue interface {
ErrorEvent
EventUnsupportedValue()
Key() string
Value() interface{}
}
type eventUnsupportedValue struct {
key string
value interface{}
err error
}
func newEventUnsupportedValue(key string, value interface{}, err error) EventUnsupportedValue {
if err == nil {
err = fmt.Errorf(
"value `%v` of type `%T` for key `%s` is an unsupported type",
value, value, key,
)
}
return &eventUnsupportedValue{
key: key,
value: value,
err: err,
}
}
func (e *eventUnsupportedValue) Event() {}
func (e *eventUnsupportedValue) EventUnsupportedValue() {}
func (e *eventUnsupportedValue) Key() string {
return e.key
}
func (e *eventUnsupportedValue) Value() interface{} {
return e.value
}
func (e *eventUnsupportedValue) String() string {
return e.err.Error()
}
func (e *eventUnsupportedValue) Error() string {
return e.err.Error()
}
func (e *eventUnsupportedValue) Err() error {
return e.err
}
const tracerDisabled = "the tracer has been disabled"
// EventTracerDisabled occurs when a tracer is disabled by either the user or
// the collector.
type EventTracerDisabled interface {
Event
EventTracerDisabled()
}
type eventTracerDisabled struct{}
func newEventTracerDisabled() EventTracerDisabled {
return eventTracerDisabled{}
}
func (eventTracerDisabled) Event() {}
func (eventTracerDisabled) EventTracerDisabled() {}
func (eventTracerDisabled) String() string {
return tracerDisabled
}
Changes by Version
==================
1.1.0 (unreleased)
-------------------
- Deprecate InitGlobalTracer() in favor of SetGlobalTracer()
1.0.0 (2016-09-26)
-------------------
- This release implements OpenTracing Specification 1.0 (https://opentracing.io/spec)
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "{}"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2016 The OpenTracing Authors
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
.DEFAULT_GOAL := test-and-lint
.PHONY: test-and-lint
test-and-lint: test lint
.PHONY: test
test:
go test -v -cover -race ./...
.PHONY: cover
cover:
go test -v -coverprofile=coverage.txt -covermode=atomic -race ./...
.PHONY: lint
lint:
go fmt ./...
golint ./...
@# Run again with magic to exit non-zero if golint outputs anything.
@! (golint ./... | read dummy)
go vet ./...
[![Gitter chat](http://img.shields.io/badge/gitter-join%20chat%20%E2%86%92-brightgreen.svg)](https://gitter.im/opentracing/public) [![Build Status](https://travis-ci.org/opentracing/opentracing-go.svg?branch=master)](https://travis-ci.org/opentracing/opentracing-go) [![GoDoc](https://godoc.org/github.com/opentracing/opentracing-go?status.svg)](http://godoc.org/github.com/opentracing/opentracing-go)
[![Sourcegraph Badge](https://sourcegraph.com/github.com/opentracing/opentracing-go/-/badge.svg)](https://sourcegraph.com/github.com/opentracing/opentracing-go?badge)
# OpenTracing API for Go
This package is a Go platform API for OpenTracing.
## Required Reading
In order to understand the Go platform API, one must first be familiar with the
[OpenTracing project](https://opentracing.io) and
[terminology](https://opentracing.io/specification/) more specifically.
## API overview for those adding instrumentation
Everyday consumers of this `opentracing` package really only need to worry
about a couple of key abstractions: the `StartSpan` function, the `Span`
interface, and binding a `Tracer` at `main()`-time. Here are code snippets
demonstrating some important use cases.
#### Singleton initialization
The simplest starting point is `./default_tracer.go`. As early as possible, call
```go
import "github.com/opentracing/opentracing-go"
import ".../some_tracing_impl"
func main() {
opentracing.SetGlobalTracer(
// tracing impl specific:
some_tracing_impl.New(...),
)
...
}
```
#### Non-Singleton initialization
If you prefer direct control to singletons, manage ownership of the
`opentracing.Tracer` implementation explicitly.
#### Creating a Span given an existing Go `context.Context`
If you use `context.Context` in your application, OpenTracing's Go library will
happily rely on it for `Span` propagation. To start a new (blocking child)
`Span`, you can use `StartSpanFromContext`.
```go
func xyz(ctx context.Context, ...) {
...
span, ctx := opentracing.StartSpanFromContext(ctx, "operation_name")
defer span.Finish()
span.LogFields(
log.String("event", "soft error"),
log.String("type", "cache timeout"),
log.Int("waited.millis", 1500))
...
}
```
#### Starting an empty trace by creating a "root span"
It's always possible to create a "root" `Span` with no parent or other causal
reference.
```go
func xyz() {
...
sp := opentracing.StartSpan("operation_name")
defer sp.Finish()
...
}
```
#### Creating a (child) Span given an existing (parent) Span
```go
func xyz(parentSpan opentracing.Span, ...) {
...
sp := opentracing.StartSpan(
"operation_name",
opentracing.ChildOf(parentSpan.Context()))
defer sp.Finish()
...
}
```
#### Serializing to the wire
```go
func makeSomeRequest(ctx context.Context) ... {
if span := opentracing.SpanFromContext(ctx); span != nil {
httpClient := &http.Client{}
httpReq, _ := http.NewRequest("GET", "http://myservice/", nil)
// Transmit the span's TraceContext as HTTP headers on our
// outbound request.
opentracing.GlobalTracer().Inject(
span.Context(),
opentracing.HTTPHeaders,
opentracing.HTTPHeadersCarrier(httpReq.Header))
resp, err := httpClient.Do(httpReq)
...
}
...
}
```
#### Deserializing from the wire
```go
http.HandleFunc("/", func(w http.ResponseWriter, req *http.Request) {
var serverSpan opentracing.Span
appSpecificOperationName := ...
wireContext, err := opentracing.GlobalTracer().Extract(
opentracing.HTTPHeaders,
opentracing.HTTPHeadersCarrier(req.Header))
if err != nil {
// Optionally record something about err here
}
// Create the span referring to the RPC client if available.
// If wireContext == nil, a root span will be created.
serverSpan = opentracing.StartSpan(
appSpecificOperationName,
ext.RPCServerOption(wireContext))
defer serverSpan.Finish()
ctx := opentracing.ContextWithSpan(context.Background(), serverSpan)
...
}
```
#### Conditionally capture a field using `log.Noop`
In some situations, you may want to dynamically decide whether or not
to log a field. For example, you may want to capture additional data,
such as a customer ID, in non-production environments:
```go
func Customer(order *Order) log.Field {
if os.Getenv("ENVIRONMENT") == "dev" {
return log.String("customer", order.Customer.ID)
}
return log.Noop()
}
```
#### Goroutine-safety
The entire public API is goroutine-safe and does not require external
synchronization.
## API pointers for those implementing a tracing system
Tracing system implementors may be able to reuse or copy-paste-modify the `basictracer` package, found [here](https://github.com/opentracing/basictracer-go). In particular, see `basictracer.New(...)`.
## API compatibility
For the time being, "mild" backwards-incompatible changes may be made without changing the major version number. As OpenTracing and `opentracing-go` mature, backwards compatibility will become more of a priority.
## Tracer test suite
A test suite is available in the [harness](https://godoc.org/github.com/opentracing/opentracing-go/harness) package that can assist Tracer implementors to assert that their Tracer is working correctly.
## Licensing
[Apache 2.0 License](./LICENSE).
package ext
import "github.com/opentracing/opentracing-go"
// These constants define common tag names recommended for better portability across
// tracing systems and languages/platforms.
//
// The tag names are defined as typed strings, so that in addition to the usual use
//
// span.setTag(TagName, value)
//
// they also support value type validation via this additional syntax:
//
// TagName.Set(span, value)
//
var (
//////////////////////////////////////////////////////////////////////
// SpanKind (client/server or producer/consumer)
//////////////////////////////////////////////////////////////////////
// SpanKind hints at relationship between spans, e.g. client/server
SpanKind = spanKindTagName("span.kind")
// SpanKindRPCClient marks a span representing the client-side of an RPC
// or other remote call
SpanKindRPCClientEnum = SpanKindEnum("client")
SpanKindRPCClient = opentracing.Tag{Key: string(SpanKind), Value: SpanKindRPCClientEnum}
// SpanKindRPCServer marks a span representing the server-side of an RPC
// or other remote call
SpanKindRPCServerEnum = SpanKindEnum("server")
SpanKindRPCServer = opentracing.Tag{Key: string(SpanKind), Value: SpanKindRPCServerEnum}
// SpanKindProducer marks a span representing the producer-side of a
// message bus
SpanKindProducerEnum = SpanKindEnum("producer")
SpanKindProducer = opentracing.Tag{Key: string(SpanKind), Value: SpanKindProducerEnum}
// SpanKindConsumer marks a span representing the consumer-side of a
// message bus
SpanKindConsumerEnum = SpanKindEnum("consumer")
SpanKindConsumer = opentracing.Tag{Key: string(SpanKind), Value: SpanKindConsumerEnum}
//////////////////////////////////////////////////////////////////////
// Component name
//////////////////////////////////////////////////////////////////////
// Component is a low-cardinality identifier of the module, library,
// or package that is generating a span.
Component = stringTagName("component")
//////////////////////////////////////////////////////////////////////
// Sampling hint
//////////////////////////////////////////////////////////////////////
// SamplingPriority determines the priority of sampling this Span.
SamplingPriority = uint16TagName("sampling.priority")
//////////////////////////////////////////////////////////////////////
// Peer tags. These tags can be emitted by either client-side of
// server-side to describe the other side/service in a peer-to-peer
// communications, like an RPC call.
//////////////////////////////////////////////////////////////////////
// PeerService records the service name of the peer.
PeerService = stringTagName("peer.service")
// PeerAddress records the address name of the peer. This may be a "ip:port",
// a bare "hostname", a FQDN or even a database DSN substring
// like "mysql://username@127.0.0.1:3306/dbname"
PeerAddress = stringTagName("peer.address")
// PeerHostname records the host name of the peer
PeerHostname = stringTagName("peer.hostname")
// PeerHostIPv4 records IP v4 host address of the peer
PeerHostIPv4 = ipv4Tag("peer.ipv4")
// PeerHostIPv6 records IP v6 host address of the peer
PeerHostIPv6 = stringTagName("peer.ipv6")
// PeerPort records port number of the peer
PeerPort = uint16TagName("peer.port")
//////////////////////////////////////////////////////////////////////
// HTTP Tags
//////////////////////////////////////////////////////////////////////
// HTTPUrl should be the URL of the request being handled in this segment
// of the trace, in standard URI format. The protocol is optional.
HTTPUrl = stringTagName("http.url")
// HTTPMethod is the HTTP method of the request, and is case-insensitive.
HTTPMethod = stringTagName("http.method")
// HTTPStatusCode is the numeric HTTP status code (200, 404, etc) of the
// HTTP response.
HTTPStatusCode = uint16TagName("http.status_code")
//////////////////////////////////////////////////////////////////////
// DB Tags
//////////////////////////////////////////////////////////////////////
// DBInstance is database instance name.
DBInstance = stringTagName("db.instance")
// DBStatement is a database statement for the given database type.
// It can be a query or a prepared statement (i.e., before substitution).
DBStatement = stringTagName("db.statement")
// DBType is a database type. For any SQL database, "sql".
// For others, the lower-case database category, e.g. "redis"
DBType = stringTagName("db.type")
// DBUser is a username for accessing database.
DBUser = stringTagName("db.user")
//////////////////////////////////////////////////////////////////////
// Message Bus Tag
//////////////////////////////////////////////////////////////////////
// MessageBusDestination is an address at which messages can be exchanged
MessageBusDestination = stringTagName("message_bus.destination")
//////////////////////////////////////////////////////////////////////
// Error Tag
//////////////////////////////////////////////////////////////////////
// Error indicates that operation represented by the span resulted in an error.
Error = boolTagName("error")
)
// ---
// SpanKindEnum represents common span types
type SpanKindEnum string
type spanKindTagName string
// Set adds a string tag to the `span`
func (tag spanKindTagName) Set(span opentracing.Span, value SpanKindEnum) {
span.SetTag(string(tag), value)
}
type rpcServerOption struct {
clientContext opentracing.SpanContext
}
func (r rpcServerOption) Apply(o *opentracing.StartSpanOptions) {
if r.clientContext != nil {
opentracing.ChildOf(r.clientContext).Apply(o)
}
SpanKindRPCServer.Apply(o)
}
// RPCServerOption returns a StartSpanOption appropriate for an RPC server span
// with `client` representing the metadata for the remote peer Span if available.
// In case client == nil, due to the client not being instrumented, this RPC
// server span will be a root span.
func RPCServerOption(client opentracing.SpanContext) opentracing.StartSpanOption {
return rpcServerOption{client}
}
// ---
type stringTagName string
// Set adds a string tag to the `span`
func (tag stringTagName) Set(span opentracing.Span, value string) {
span.SetTag(string(tag), value)
}
// ---
type uint32TagName string
// Set adds a uint32 tag to the `span`
func (tag uint32TagName) Set(span opentracing.Span, value uint32) {
span.SetTag(string(tag), value)
}
// ---
type uint16TagName string
// Set adds a uint16 tag to the `span`
func (tag uint16TagName) Set(span opentracing.Span, value uint16) {
span.SetTag(string(tag), value)
}
// ---
type boolTagName string
// Add adds a bool tag to the `span`
func (tag boolTagName) Set(span opentracing.Span, value bool) {
span.SetTag(string(tag), value)
}
type ipv4Tag string
// Set adds IP v4 host address of the peer as an uint32 value to the `span`, keep this for backward and zipkin compatibility
func (tag ipv4Tag) Set(span opentracing.Span, value uint32) {
span.SetTag(string(tag), value)
}
// SetString records IP v4 host address of the peer as a .-separated tuple to the `span`. E.g., "127.0.0.1"
func (tag ipv4Tag) SetString(span opentracing.Span, value string) {
span.SetTag(string(tag), value)
}
package opentracing
type registeredTracer struct {
tracer Tracer
isRegistered bool
}
var (
globalTracer = registeredTracer{NoopTracer{}, false}
)
// SetGlobalTracer sets the [singleton] opentracing.Tracer returned by
// GlobalTracer(). Those who use GlobalTracer (rather than directly manage an
// opentracing.Tracer instance) should call SetGlobalTracer as early as
// possible in main(), prior to calling the `StartSpan` global func below.
// Prior to calling `SetGlobalTracer`, any Spans started via the `StartSpan`
// (etc) globals are noops.
func SetGlobalTracer(tracer Tracer) {
globalTracer = registeredTracer{tracer, true}
}
// GlobalTracer returns the global singleton `Tracer` implementation.
// Before `SetGlobalTracer()` is called, the `GlobalTracer()` is a noop
// implementation that drops all data handed to it.
func GlobalTracer() Tracer {
return globalTracer.tracer
}
// StartSpan defers to `Tracer.StartSpan`. See `GlobalTracer()`.
func StartSpan(operationName string, opts ...StartSpanOption) Span {
return globalTracer.tracer.StartSpan(operationName, opts...)
}
// InitGlobalTracer is deprecated. Please use SetGlobalTracer.
func InitGlobalTracer(tracer Tracer) {
SetGlobalTracer(tracer)
}
// IsGlobalTracerRegistered returns a `bool` to indicate if a tracer has been globally registered
func IsGlobalTracerRegistered() bool {
return globalTracer.isRegistered
}
package opentracing
import "context"
type contextKey struct{}
var activeSpanKey = contextKey{}
// ContextWithSpan returns a new `context.Context` that holds a reference to
// `span`'s SpanContext.
func ContextWithSpan(ctx context.Context, span Span) context.Context {
return context.WithValue(ctx, activeSpanKey, span)
}
// SpanFromContext returns the `Span` previously associated with `ctx`, or
// `nil` if no such `Span` could be found.
//
// NOTE: context.Context != SpanContext: the former is Go's intra-process
// context propagation mechanism, and the latter houses OpenTracing's per-Span
// identity and baggage information.
func SpanFromContext(ctx context.Context) Span {
val := ctx.Value(activeSpanKey)
if sp, ok := val.(Span); ok {
return sp
}
return nil
}
// StartSpanFromContext starts and returns a Span with `operationName`, using
// any Span found within `ctx` as a ChildOfRef. If no such parent could be
// found, StartSpanFromContext creates a root (parentless) Span.
//
// The second return value is a context.Context object built around the
// returned Span.
//
// Example usage:
//
// SomeFunction(ctx context.Context, ...) {
// sp, ctx := opentracing.StartSpanFromContext(ctx, "SomeFunction")
// defer sp.Finish()
// ...
// }
func StartSpanFromContext(ctx context.Context, operationName string, opts ...StartSpanOption) (Span, context.Context) {
return startSpanFromContextWithTracer(ctx, GlobalTracer(), operationName, opts...)
}
// startSpanFromContextWithTracer is factored out for testing purposes.
func startSpanFromContextWithTracer(ctx context.Context, tracer Tracer, operationName string, opts ...StartSpanOption) (Span, context.Context) {
if parentSpan := SpanFromContext(ctx); parentSpan != nil {
opts = append(opts, ChildOf(parentSpan.Context()))
}
span := tracer.StartSpan(operationName, opts...)
return span, ContextWithSpan(ctx, span)
}
package log
import (
"fmt"
"math"
)
type fieldType int
const (
stringType fieldType = iota
boolType
intType
int32Type
uint32Type
int64Type
uint64Type
float32Type
float64Type
errorType
objectType
lazyLoggerType
noopType
)
// Field instances are constructed via LogBool, LogString, and so on.
// Tracing implementations may then handle them via the Field.Marshal
// method.
//
// "heavily influenced by" (i.e., partially stolen from)
// https://github.com/uber-go/zap
type Field struct {
key string
fieldType fieldType
numericVal int64
stringVal string
interfaceVal interface{}
}
// String adds a string-valued key:value pair to a Span.LogFields() record
func String(key, val string) Field {
return Field{
key: key,
fieldType: stringType,
stringVal: val,
}
}
// Bool adds a bool-valued key:value pair to a Span.LogFields() record
func Bool(key string, val bool) Field {
var numericVal int64
if val {
numericVal = 1
}
return Field{
key: key,
fieldType: boolType,
numericVal: numericVal,
}
}
// Int adds an int-valued key:value pair to a Span.LogFields() record
func Int(key string, val int) Field {
return Field{
key: key,
fieldType: intType,
numericVal: int64(val),
}
}
// Int32 adds an int32-valued key:value pair to a Span.LogFields() record
func Int32(key string, val int32) Field {
return Field{
key: key,
fieldType: int32Type,
numericVal: int64(val),
}
}
// Int64 adds an int64-valued key:value pair to a Span.LogFields() record
func Int64(key string, val int64) Field {
return Field{
key: key,
fieldType: int64Type,
numericVal: val,
}
}
// Uint32 adds a uint32-valued key:value pair to a Span.LogFields() record
func Uint32(key string, val uint32) Field {
return Field{
key: key,
fieldType: uint32Type,
numericVal: int64(val),
}
}
// Uint64 adds a uint64-valued key:value pair to a Span.LogFields() record
func Uint64(key string, val uint64) Field {
return Field{
key: key,
fieldType: uint64Type,
numericVal: int64(val),
}
}
// Float32 adds a float32-valued key:value pair to a Span.LogFields() record
func Float32(key string, val float32) Field {
return Field{
key: key,
fieldType: float32Type,
numericVal: int64(math.Float32bits(val)),
}
}
// Float64 adds a float64-valued key:value pair to a Span.LogFields() record
func Float64(key string, val float64) Field {
return Field{
key: key,
fieldType: float64Type,
numericVal: int64(math.Float64bits(val)),
}
}
// Error adds an error with the key "error" to a Span.LogFields() record
func Error(err error) Field {
return Field{
key: "error",
fieldType: errorType,
interfaceVal: err,
}
}
// Object adds an object-valued key:value pair to a Span.LogFields() record
func Object(key string, obj interface{}) Field {
return Field{
key: key,
fieldType: objectType,
interfaceVal: obj,
}
}
// LazyLogger allows for user-defined, late-bound logging of arbitrary data
type LazyLogger func(fv Encoder)
// Lazy adds a LazyLogger to a Span.LogFields() record; the tracing
// implementation will call the LazyLogger function at an indefinite time in
// the future (after Lazy() returns).
func Lazy(ll LazyLogger) Field {
return Field{
fieldType: lazyLoggerType,
interfaceVal: ll,
}
}
// Noop creates a no-op log field that should be ignored by the tracer.
// It can be used to capture optional fields, for example those that should
// only be logged in non-production environment:
//
// func customerField(order *Order) log.Field {
// if os.Getenv("ENVIRONMENT") == "dev" {
// return log.String("customer", order.Customer.ID)
// }
// return log.Noop()
// }
//
// span.LogFields(log.String("event", "purchase"), customerField(order))
//
func Noop() Field {
return Field{
fieldType: noopType,
}
}
// Encoder allows access to the contents of a Field (via a call to
// Field.Marshal).
//
// Tracer implementations typically provide an implementation of Encoder;
// OpenTracing callers typically do not need to concern themselves with it.
type Encoder interface {
EmitString(key, value string)
EmitBool(key string, value bool)
EmitInt(key string, value int)
EmitInt32(key string, value int32)
EmitInt64(key string, value int64)
EmitUint32(key string, value uint32)
EmitUint64(key string, value uint64)
EmitFloat32(key string, value float32)
EmitFloat64(key string, value float64)
EmitObject(key string, value interface{})
EmitLazyLogger(value LazyLogger)
}
// Marshal passes a Field instance through to the appropriate
// field-type-specific method of an Encoder.
func (lf Field) Marshal(visitor Encoder) {
switch lf.fieldType {
case stringType:
visitor.EmitString(lf.key, lf.stringVal)
case boolType:
visitor.EmitBool(lf.key, lf.numericVal != 0)
case intType:
visitor.EmitInt(lf.key, int(lf.numericVal))
case int32Type:
visitor.EmitInt32(lf.key, int32(lf.numericVal))
case int64Type:
visitor.EmitInt64(lf.key, int64(lf.numericVal))
case uint32Type:
visitor.EmitUint32(lf.key, uint32(lf.numericVal))
case uint64Type:
visitor.EmitUint64(lf.key, uint64(lf.numericVal))
case float32Type:
visitor.EmitFloat32(lf.key, math.Float32frombits(uint32(lf.numericVal)))
case float64Type:
visitor.EmitFloat64(lf.key, math.Float64frombits(uint64(lf.numericVal)))
case errorType:
if err, ok := lf.interfaceVal.(error); ok {
visitor.EmitString(lf.key, err.Error())
} else {
visitor.EmitString(lf.key, "<nil>")
}
case objectType:
visitor.EmitObject(lf.key, lf.interfaceVal)
case lazyLoggerType:
visitor.EmitLazyLogger(lf.interfaceVal.(LazyLogger))
case noopType:
// intentionally left blank
}
}
// Key returns the field's key.
func (lf Field) Key() string {
return lf.key
}
// Value returns the field's value as interface{}.
func (lf Field) Value() interface{} {
switch lf.fieldType {
case stringType:
return lf.stringVal
case boolType:
return lf.numericVal != 0
case intType:
return int(lf.numericVal)
case int32Type:
return int32(lf.numericVal)
case int64Type:
return int64(lf.numericVal)
case uint32Type:
return uint32(lf.numericVal)
case uint64Type:
return uint64(lf.numericVal)
case float32Type:
return math.Float32frombits(uint32(lf.numericVal))
case float64Type:
return math.Float64frombits(uint64(lf.numericVal))
case errorType, objectType, lazyLoggerType:
return lf.interfaceVal
case noopType:
return nil
default:
return nil
}
}
// String returns a string representation of the key and value.
func (lf Field) String() string {
return fmt.Sprint(lf.key, ":", lf.Value())
}
package log
import "fmt"
// InterleavedKVToFields converts keyValues a la Span.LogKV() to a Field slice
// a la Span.LogFields().
func InterleavedKVToFields(keyValues ...interface{}) ([]Field, error) {
if len(keyValues)%2 != 0 {
return nil, fmt.Errorf("non-even keyValues len: %d", len(keyValues))
}
fields := make([]Field, len(keyValues)/2)
for i := 0; i*2 < len(keyValues); i++ {
key, ok := keyValues[i*2].(string)
if !ok {
return nil, fmt.Errorf(
"non-string key (pair #%d): %T",
i, keyValues[i*2])
}
switch typedVal := keyValues[i*2+1].(type) {
case bool:
fields[i] = Bool(key, typedVal)
case string:
fields[i] = String(key, typedVal)
case int:
fields[i] = Int(key, typedVal)
case int8:
fields[i] = Int32(key, int32(typedVal))
case int16:
fields[i] = Int32(key, int32(typedVal))
case int32:
fields[i] = Int32(key, typedVal)
case int64:
fields[i] = Int64(key, typedVal)
case uint:
fields[i] = Uint64(key, uint64(typedVal))
case uint64:
fields[i] = Uint64(key, typedVal)
case uint8:
fields[i] = Uint32(key, uint32(typedVal))
case uint16:
fields[i] = Uint32(key, uint32(typedVal))
case uint32:
fields[i] = Uint32(key, typedVal)
case float32:
fields[i] = Float32(key, typedVal)
case float64:
fields[i] = Float64(key, typedVal)
default:
// When in doubt, coerce to a string
fields[i] = String(key, fmt.Sprint(typedVal))
}
}
return fields, nil
}
package opentracing
import "github.com/opentracing/opentracing-go/log"
// A NoopTracer is a trivial, minimum overhead implementation of Tracer
// for which all operations are no-ops.
//
// The primary use of this implementation is in libraries, such as RPC
// frameworks, that make tracing an optional feature controlled by the
// end user. A no-op implementation allows said libraries to use it
// as the default Tracer and to write instrumentation that does
// not need to keep checking if the tracer instance is nil.
//
// For the same reason, the NoopTracer is the default "global" tracer
// (see GlobalTracer and SetGlobalTracer functions).
//
// WARNING: NoopTracer does not support baggage propagation.
type NoopTracer struct{}
type noopSpan struct{}
type noopSpanContext struct{}
var (
defaultNoopSpanContext = noopSpanContext{}
defaultNoopSpan = noopSpan{}
defaultNoopTracer = NoopTracer{}
)
const (
emptyString = ""
)
// noopSpanContext:
func (n noopSpanContext) ForeachBaggageItem(handler func(k, v string) bool) {}
// noopSpan:
func (n noopSpan) Context() SpanContext { return defaultNoopSpanContext }
func (n noopSpan) SetBaggageItem(key, val string) Span { return defaultNoopSpan }
func (n noopSpan) BaggageItem(key string) string { return emptyString }
func (n noopSpan) SetTag(key string, value interface{}) Span { return n }
func (n noopSpan) LogFields(fields ...log.Field) {}
func (n noopSpan) LogKV(keyVals ...interface{}) {}
func (n noopSpan) Finish() {}
func (n noopSpan) FinishWithOptions(opts FinishOptions) {}
func (n noopSpan) SetOperationName(operationName string) Span { return n }
func (n noopSpan) Tracer() Tracer { return defaultNoopTracer }
func (n noopSpan) LogEvent(event string) {}
func (n noopSpan) LogEventWithPayload(event string, payload interface{}) {}
func (n noopSpan) Log(data LogData) {}
// StartSpan belongs to the Tracer interface.
func (n NoopTracer) StartSpan(operationName string, opts ...StartSpanOption) Span {
return defaultNoopSpan
}
// Inject belongs to the Tracer interface.
func (n NoopTracer) Inject(sp SpanContext, format interface{}, carrier interface{}) error {
return nil
}
// Extract belongs to the Tracer interface.
func (n NoopTracer) Extract(format interface{}, carrier interface{}) (SpanContext, error) {
return nil, ErrSpanContextNotFound
}
package opentracing
import (
"errors"
"net/http"
)
///////////////////////////////////////////////////////////////////////////////
// CORE PROPAGATION INTERFACES:
///////////////////////////////////////////////////////////////////////////////
var (
// ErrUnsupportedFormat occurs when the `format` passed to Tracer.Inject() or
// Tracer.Extract() is not recognized by the Tracer implementation.
ErrUnsupportedFormat = errors.New("opentracing: Unknown or unsupported Inject/Extract format")
// ErrSpanContextNotFound occurs when the `carrier` passed to
// Tracer.Extract() is valid and uncorrupted but has insufficient
// information to extract a SpanContext.
ErrSpanContextNotFound = errors.New("opentracing: SpanContext not found in Extract carrier")
// ErrInvalidSpanContext errors occur when Tracer.Inject() is asked to
// operate on a SpanContext which it is not prepared to handle (for
// example, since it was created by a different tracer implementation).
ErrInvalidSpanContext = errors.New("opentracing: SpanContext type incompatible with tracer")
// ErrInvalidCarrier errors occur when Tracer.Inject() or Tracer.Extract()
// implementations expect a different type of `carrier` than they are
// given.
ErrInvalidCarrier = errors.New("opentracing: Invalid Inject/Extract carrier")
// ErrSpanContextCorrupted occurs when the `carrier` passed to
// Tracer.Extract() is of the expected type but is corrupted.
ErrSpanContextCorrupted = errors.New("opentracing: SpanContext data corrupted in Extract carrier")
)
///////////////////////////////////////////////////////////////////////////////
// BUILTIN PROPAGATION FORMATS:
///////////////////////////////////////////////////////////////////////////////
// BuiltinFormat is used to demarcate the values within package `opentracing`
// that are intended for use with the Tracer.Inject() and Tracer.Extract()
// methods.
type BuiltinFormat byte
const (
// Binary represents SpanContexts as opaque binary data.
//
// For Tracer.Inject(): the carrier must be an `io.Writer`.
//
// For Tracer.Extract(): the carrier must be an `io.Reader`.
Binary BuiltinFormat = iota
// TextMap represents SpanContexts as key:value string pairs.
//
// Unlike HTTPHeaders, the TextMap format does not restrict the key or
// value character sets in any way.
//
// For Tracer.Inject(): the carrier must be a `TextMapWriter`.
//
// For Tracer.Extract(): the carrier must be a `TextMapReader`.
TextMap
// HTTPHeaders represents SpanContexts as HTTP header string pairs.
//
// Unlike TextMap, the HTTPHeaders format requires that the keys and values
// be valid as HTTP headers as-is (i.e., character casing may be unstable
// and special characters are disallowed in keys, values should be
// URL-escaped, etc).
//
// For Tracer.Inject(): the carrier must be a `TextMapWriter`.
//
// For Tracer.Extract(): the carrier must be a `TextMapReader`.
//
// See HTTPHeadersCarrier for an implementation of both TextMapWriter
// and TextMapReader that defers to an http.Header instance for storage.
// For example, Inject():
//
// carrier := opentracing.HTTPHeadersCarrier(httpReq.Header)
// err := span.Tracer().Inject(
// span.Context(), opentracing.HTTPHeaders, carrier)
//
// Or Extract():
//
// carrier := opentracing.HTTPHeadersCarrier(httpReq.Header)
// clientContext, err := tracer.Extract(
// opentracing.HTTPHeaders, carrier)
//
HTTPHeaders
)
// TextMapWriter is the Inject() carrier for the TextMap builtin format. With
// it, the caller can encode a SpanContext for propagation as entries in a map
// of unicode strings.
type TextMapWriter interface {
// Set a key:value pair to the carrier. Multiple calls to Set() for the
// same key leads to undefined behavior.
//
// NOTE: The backing store for the TextMapWriter may contain data unrelated
// to SpanContext. As such, Inject() and Extract() implementations that
// call the TextMapWriter and TextMapReader interfaces must agree on a
// prefix or other convention to distinguish their own key:value pairs.
Set(key, val string)
}
// TextMapReader is the Extract() carrier for the TextMap builtin format. With it,
// the caller can decode a propagated SpanContext as entries in a map of
// unicode strings.
type TextMapReader interface {
// ForeachKey returns TextMap contents via repeated calls to the `handler`
// function. If any call to `handler` returns a non-nil error, ForeachKey
// terminates and returns that error.
//
// NOTE: The backing store for the TextMapReader may contain data unrelated
// to SpanContext. As such, Inject() and Extract() implementations that
// call the TextMapWriter and TextMapReader interfaces must agree on a
// prefix or other convention to distinguish their own key:value pairs.
//
// The "foreach" callback pattern reduces unnecessary copying in some cases
// and also allows implementations to hold locks while the map is read.
ForeachKey(handler func(key, val string) error) error
}
// TextMapCarrier allows the use of regular map[string]string
// as both TextMapWriter and TextMapReader.
type TextMapCarrier map[string]string
// ForeachKey conforms to the TextMapReader interface.
func (c TextMapCarrier) ForeachKey(handler func(key, val string) error) error {
for k, v := range c {
if err := handler(k, v); err != nil {
return err
}
}
return nil
}
// Set implements Set() of opentracing.TextMapWriter
func (c TextMapCarrier) Set(key, val string) {
c[key] = val
}
// HTTPHeadersCarrier satisfies both TextMapWriter and TextMapReader.
//
// Example usage for server side:
//
// carrier := opentracing.HTTPHeadersCarrier(httpReq.Header)
// clientContext, err := tracer.Extract(opentracing.HTTPHeaders, carrier)
//
// Example usage for client side:
//
// carrier := opentracing.HTTPHeadersCarrier(httpReq.Header)
// err := tracer.Inject(
// span.Context(),
// opentracing.HTTPHeaders,
// carrier)
//
type HTTPHeadersCarrier http.Header
// Set conforms to the TextMapWriter interface.
func (c HTTPHeadersCarrier) Set(key, val string) {
h := http.Header(c)
h.Set(key, val)
}
// ForeachKey conforms to the TextMapReader interface.
func (c HTTPHeadersCarrier) ForeachKey(handler func(key, val string) error) error {
for k, vals := range c {
for _, v := range vals {
if err := handler(k, v); err != nil {
return err
}
}
}
return nil
}
package opentracing
import (
"time"
"github.com/opentracing/opentracing-go/log"
)
// SpanContext represents Span state that must propagate to descendant Spans and across process
// boundaries (e.g., a <trace_id, span_id, sampled> tuple).
type SpanContext interface {
// ForeachBaggageItem grants access to all baggage items stored in the
// SpanContext.
// The handler function will be called for each baggage key/value pair.
// The ordering of items is not guaranteed.
//
// The bool return value indicates if the handler wants to continue iterating
// through the rest of the baggage items; for example if the handler is trying to
// find some baggage item by pattern matching the name, it can return false
// as soon as the item is found to stop further iterations.
ForeachBaggageItem(handler func(k, v string) bool)
}
// Span represents an active, un-finished span in the OpenTracing system.
//
// Spans are created by the Tracer interface.
type Span interface {
// Sets the end timestamp and finalizes Span state.
//
// With the exception of calls to Context() (which are always allowed),
// Finish() must be the last call made to any span instance, and to do
// otherwise leads to undefined behavior.
Finish()
// FinishWithOptions is like Finish() but with explicit control over
// timestamps and log data.
FinishWithOptions(opts FinishOptions)
// Context() yields the SpanContext for this Span. Note that the return
// value of Context() is still valid after a call to Span.Finish(), as is
// a call to Span.Context() after a call to Span.Finish().
Context() SpanContext
// Sets or changes the operation name.
//
// Returns a reference to this Span for chaining.
SetOperationName(operationName string) Span
// Adds a tag to the span.
//
// If there is a pre-existing tag set for `key`, it is overwritten.
//
// Tag values can be numeric types, strings, or bools. The behavior of
// other tag value types is undefined at the OpenTracing level. If a
// tracing system does not know how to handle a particular value type, it
// may ignore the tag, but shall not panic.
//
// Returns a reference to this Span for chaining.
SetTag(key string, value interface{}) Span
// LogFields is an efficient and type-checked way to record key:value
// logging data about a Span, though the programming interface is a little
// more verbose than LogKV(). Here's an example:
//
// span.LogFields(
// log.String("event", "soft error"),
// log.String("type", "cache timeout"),
// log.Int("waited.millis", 1500))
//
// Also see Span.FinishWithOptions() and FinishOptions.BulkLogData.
LogFields(fields ...log.Field)
// LogKV is a concise, readable way to record key:value logging data about
// a Span, though unfortunately this also makes it less efficient and less
// type-safe than LogFields(). Here's an example:
//
// span.LogKV(
// "event", "soft error",
// "type", "cache timeout",
// "waited.millis", 1500)
//
// For LogKV (as opposed to LogFields()), the parameters must appear as
// key-value pairs, like
//
// span.LogKV(key1, val1, key2, val2, key3, val3, ...)
//
// The keys must all be strings. The values may be strings, numeric types,
// bools, Go error instances, or arbitrary structs.
//
// (Note to implementors: consider the log.InterleavedKVToFields() helper)
LogKV(alternatingKeyValues ...interface{})
// SetBaggageItem sets a key:value pair on this Span and its SpanContext
// that also propagates to descendants of this Span.
//
// SetBaggageItem() enables powerful functionality given a full-stack
// opentracing integration (e.g., arbitrary application data from a mobile
// app can make it, transparently, all the way into the depths of a storage
// system), and with it some powerful costs: use this feature with care.
//
// IMPORTANT NOTE #1: SetBaggageItem() will only propagate baggage items to
// *future* causal descendants of the associated Span.
//
// IMPORTANT NOTE #2: Use this thoughtfully and with care. Every key and
// value is copied into every local *and remote* child of the associated
// Span, and that can add up to a lot of network and cpu overhead.
//
// Returns a reference to this Span for chaining.
SetBaggageItem(restrictedKey, value string) Span
// Gets the value for a baggage item given its key. Returns the empty string
// if the value isn't found in this Span.
BaggageItem(restrictedKey string) string
// Provides access to the Tracer that created this Span.
Tracer() Tracer
// Deprecated: use LogFields or LogKV
LogEvent(event string)
// Deprecated: use LogFields or LogKV
LogEventWithPayload(event string, payload interface{})
// Deprecated: use LogFields or LogKV
Log(data LogData)
}
// LogRecord is data associated with a single Span log. Every LogRecord
// instance must specify at least one Field.
type LogRecord struct {
Timestamp time.Time
Fields []log.Field
}
// FinishOptions allows Span.FinishWithOptions callers to override the finish
// timestamp and provide log data via a bulk interface.
type FinishOptions struct {
// FinishTime overrides the Span's finish time, or implicitly becomes
// time.Now() if FinishTime.IsZero().
//
// FinishTime must resolve to a timestamp that's >= the Span's StartTime
// (per StartSpanOptions).
FinishTime time.Time
// LogRecords allows the caller to specify the contents of many LogFields()
// calls with a single slice. May be nil.
//
// None of the LogRecord.Timestamp values may be .IsZero() (i.e., they must
// be set explicitly). Also, they must be >= the Span's start timestamp and
// <= the FinishTime (or time.Now() if FinishTime.IsZero()). Otherwise the
// behavior of FinishWithOptions() is undefined.
//
// If specified, the caller hands off ownership of LogRecords at
// FinishWithOptions() invocation time.
//
// If specified, the (deprecated) BulkLogData must be nil or empty.
LogRecords []LogRecord
// BulkLogData is DEPRECATED.
BulkLogData []LogData
}
// LogData is DEPRECATED
type LogData struct {
Timestamp time.Time
Event string
Payload interface{}
}
// ToLogRecord converts a deprecated LogData to a non-deprecated LogRecord
func (ld *LogData) ToLogRecord() LogRecord {
var literalTimestamp time.Time
if ld.Timestamp.IsZero() {
literalTimestamp = time.Now()
} else {
literalTimestamp = ld.Timestamp
}
rval := LogRecord{
Timestamp: literalTimestamp,
}
if ld.Payload == nil {
rval.Fields = []log.Field{
log.String("event", ld.Event),
}
} else {
rval.Fields = []log.Field{
log.String("event", ld.Event),
log.Object("payload", ld.Payload),
}
}
return rval
}
package opentracing
import "time"
// Tracer is a simple, thin interface for Span creation and SpanContext
// propagation.
type Tracer interface {
// Create, start, and return a new Span with the given `operationName` and
// incorporate the given StartSpanOption `opts`. (Note that `opts` borrows
// from the "functional options" pattern, per
// http://dave.cheney.net/2014/10/17/functional-options-for-friendly-apis)
//
// A Span with no SpanReference options (e.g., opentracing.ChildOf() or
// opentracing.FollowsFrom()) becomes the root of its own trace.
//
// Examples:
//
// var tracer opentracing.Tracer = ...
//
// // The root-span case:
// sp := tracer.StartSpan("GetFeed")
//
// // The vanilla child span case:
// sp := tracer.StartSpan(
// "GetFeed",
// opentracing.ChildOf(parentSpan.Context()))
//
// // All the bells and whistles:
// sp := tracer.StartSpan(
// "GetFeed",
// opentracing.ChildOf(parentSpan.Context()),
// opentracing.Tag{"user_agent", loggedReq.UserAgent},
// opentracing.StartTime(loggedReq.Timestamp),
// )
//
StartSpan(operationName string, opts ...StartSpanOption) Span
// Inject() takes the `sm` SpanContext instance and injects it for
// propagation within `carrier`. The actual type of `carrier` depends on
// the value of `format`.
//
// OpenTracing defines a common set of `format` values (see BuiltinFormat),
// and each has an expected carrier type.
//
// Other packages may declare their own `format` values, much like the keys
// used by `context.Context` (see
// https://godoc.org/golang.org/x/net/context#WithValue).
//
// Example usage (sans error handling):
//
// carrier := opentracing.HTTPHeadersCarrier(httpReq.Header)
// err := tracer.Inject(
// span.Context(),
// opentracing.HTTPHeaders,
// carrier)
//
// NOTE: All opentracing.Tracer implementations MUST support all
// BuiltinFormats.
//
// Implementations may return opentracing.ErrUnsupportedFormat if `format`
// is not supported by (or not known by) the implementation.
//
// Implementations may return opentracing.ErrInvalidCarrier or any other
// implementation-specific error if the format is supported but injection
// fails anyway.
//
// See Tracer.Extract().
Inject(sm SpanContext, format interface{}, carrier interface{}) error
// Extract() returns a SpanContext instance given `format` and `carrier`.
//
// OpenTracing defines a common set of `format` values (see BuiltinFormat),
// and each has an expected carrier type.
//
// Other packages may declare their own `format` values, much like the keys
// used by `context.Context` (see
// https://godoc.org/golang.org/x/net/context#WithValue).
//
// Example usage (with StartSpan):
//
//
// carrier := opentracing.HTTPHeadersCarrier(httpReq.Header)
// clientContext, err := tracer.Extract(opentracing.HTTPHeaders, carrier)
//
// // ... assuming the ultimate goal here is to resume the trace with a
// // server-side Span:
// var serverSpan opentracing.Span
// if err == nil {
// span = tracer.StartSpan(
// rpcMethodName, ext.RPCServerOption(clientContext))
// } else {
// span = tracer.StartSpan(rpcMethodName)
// }
//
//
// NOTE: All opentracing.Tracer implementations MUST support all
// BuiltinFormats.
//
// Return values:
// - A successful Extract returns a SpanContext instance and a nil error
// - If there was simply no SpanContext to extract in `carrier`, Extract()
// returns (nil, opentracing.ErrSpanContextNotFound)
// - If `format` is unsupported or unrecognized, Extract() returns (nil,
// opentracing.ErrUnsupportedFormat)
// - If there are more fundamental problems with the `carrier` object,
// Extract() may return opentracing.ErrInvalidCarrier,
// opentracing.ErrSpanContextCorrupted, or implementation-specific
// errors.
//
// See Tracer.Inject().
Extract(format interface{}, carrier interface{}) (SpanContext, error)
}
// StartSpanOptions allows Tracer.StartSpan() callers and implementors a
// mechanism to override the start timestamp, specify Span References, and make
// a single Tag or multiple Tags available at Span start time.
//
// StartSpan() callers should look at the StartSpanOption interface and
// implementations available in this package.
//
// Tracer implementations can convert a slice of `StartSpanOption` instances
// into a `StartSpanOptions` struct like so:
//
// func StartSpan(opName string, opts ...opentracing.StartSpanOption) {
// sso := opentracing.StartSpanOptions{}
// for _, o := range opts {
// o.Apply(&sso)
// }
// ...
// }
//
type StartSpanOptions struct {
// Zero or more causal references to other Spans (via their SpanContext).
// If empty, start a "root" Span (i.e., start a new trace).
References []SpanReference
// StartTime overrides the Span's start time, or implicitly becomes
// time.Now() if StartTime.IsZero().
StartTime time.Time
// Tags may have zero or more entries; the restrictions on map values are
// identical to those for Span.SetTag(). May be nil.
//
// If specified, the caller hands off ownership of Tags at
// StartSpan() invocation time.
Tags map[string]interface{}
}
// StartSpanOption instances (zero or more) may be passed to Tracer.StartSpan.
//
// StartSpanOption borrows from the "functional options" pattern, per
// http://dave.cheney.net/2014/10/17/functional-options-for-friendly-apis
type StartSpanOption interface {
Apply(*StartSpanOptions)
}
// SpanReferenceType is an enum type describing different categories of
// relationships between two Spans. If Span-2 refers to Span-1, the
// SpanReferenceType describes Span-1 from Span-2's perspective. For example,
// ChildOfRef means that Span-1 created Span-2.
//
// NOTE: Span-1 and Span-2 do *not* necessarily depend on each other for
// completion; e.g., Span-2 may be part of a background job enqueued by Span-1,
// or Span-2 may be sitting in a distributed queue behind Span-1.
type SpanReferenceType int
const (
// ChildOfRef refers to a parent Span that caused *and* somehow depends
// upon the new child Span. Often (but not always), the parent Span cannot
// finish until the child Span does.
//
// An timing diagram for a ChildOfRef that's blocked on the new Span:
//
// [-Parent Span---------]
// [-Child Span----]
//
// See http://opentracing.io/spec/
//
// See opentracing.ChildOf()
ChildOfRef SpanReferenceType = iota
// FollowsFromRef refers to a parent Span that does not depend in any way
// on the result of the new child Span. For instance, one might use
// FollowsFromRefs to describe pipeline stages separated by queues,
// or a fire-and-forget cache insert at the tail end of a web request.
//
// A FollowsFromRef Span is part of the same logical trace as the new Span:
// i.e., the new Span is somehow caused by the work of its FollowsFromRef.
//
// All of the following could be valid timing diagrams for children that
// "FollowFrom" a parent.
//
// [-Parent Span-] [-Child Span-]
//
//
// [-Parent Span--]
// [-Child Span-]
//
//
// [-Parent Span-]
// [-Child Span-]
//
// See http://opentracing.io/spec/
//
// See opentracing.FollowsFrom()
FollowsFromRef
)
// SpanReference is a StartSpanOption that pairs a SpanReferenceType and a
// referenced SpanContext. See the SpanReferenceType documentation for
// supported relationships. If SpanReference is created with
// ReferencedContext==nil, it has no effect. Thus it allows for a more concise
// syntax for starting spans:
//
// sc, _ := tracer.Extract(someFormat, someCarrier)
// span := tracer.StartSpan("operation", opentracing.ChildOf(sc))
//
// The `ChildOf(sc)` option above will not panic if sc == nil, it will just
// not add the parent span reference to the options.
type SpanReference struct {
Type SpanReferenceType
ReferencedContext SpanContext
}
// Apply satisfies the StartSpanOption interface.
func (r SpanReference) Apply(o *StartSpanOptions) {
if r.ReferencedContext != nil {
o.References = append(o.References, r)
}
}
// ChildOf returns a StartSpanOption pointing to a dependent parent span.
// If sc == nil, the option has no effect.
//
// See ChildOfRef, SpanReference
func ChildOf(sc SpanContext) SpanReference {
return SpanReference{
Type: ChildOfRef,
ReferencedContext: sc,
}
}
// FollowsFrom returns a StartSpanOption pointing to a parent Span that caused
// the child Span but does not directly depend on its result in any way.
// If sc == nil, the option has no effect.
//
// See FollowsFromRef, SpanReference
func FollowsFrom(sc SpanContext) SpanReference {
return SpanReference{
Type: FollowsFromRef,
ReferencedContext: sc,
}
}
// StartTime is a StartSpanOption that sets an explicit start timestamp for the
// new Span.
type StartTime time.Time
// Apply satisfies the StartSpanOption interface.
func (t StartTime) Apply(o *StartSpanOptions) {
o.StartTime = time.Time(t)
}
// Tags are a generic map from an arbitrary string key to an opaque value type.
// The underlying tracing system is responsible for interpreting and
// serializing the values.
type Tags map[string]interface{}
// Apply satisfies the StartSpanOption interface.
func (t Tags) Apply(o *StartSpanOptions) {
if o.Tags == nil {
o.Tags = make(map[string]interface{})
}
for k, v := range t {
o.Tags[k] = v
}
}
// Tag may be passed as a StartSpanOption to add a tag to new spans,
// or its Set method may be used to apply the tag to an existing Span,
// for example:
//
// tracer.StartSpan("opName", Tag{"Key", value})
//
// or
//
// Tag{"key", value}.Set(span)
type Tag struct {
Key string
Value interface{}
}
// Apply satisfies the StartSpanOption interface.
func (t Tag) Apply(o *StartSpanOptions) {
if o.Tags == nil {
o.Tags = make(map[string]interface{})
}
o.Tags[t.Key] = t.Value
}
// Set applies the tag to an existing Span.
func (t Tag) Set(s Span) {
s.SetTag(t.Key, t.Value)
}
Copyright (c) 2014-2015, Philip Hofer
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
\ No newline at end of file
# fwd
import "github.com/philhofer/fwd"
The `fwd` package provides a buffered reader
and writer. Each has methods that help improve
the encoding/decoding performance of some binary
protocols.
The `fwd.Writer` and `fwd.Reader` type provide similar
functionality to their counterparts in `bufio`, plus
a few extra utility methods that simplify read-ahead
and write-ahead. I wrote this package to improve serialization
performance for <a href="http://github.com/tinylib/msgp">http://github.com/tinylib/msgp</a>,
where it provided about a 2x speedup over `bufio` for certain
workloads. However, care must be taken to understand the semantics of the
extra methods provided by this package, as they allow
the user to access and manipulate the buffer memory
directly.
The extra methods for `fwd.Reader` are `Peek`, `Skip`
and `Next`. `(*fwd.Reader).Peek`, unlike `(*bufio.Reader).Peek`,
will re-allocate the read buffer in order to accommodate arbitrarily
large read-ahead. `(*fwd.Reader).Skip` skips the next `n` bytes
in the stream, and uses the `io.Seeker` interface if the underlying
stream implements it. `(*fwd.Reader).Next` returns a slice pointing
to the next `n` bytes in the read buffer (like `Peek`), but also
increments the read position. This allows users to process streams
in arbitrary block sizes without having to manage appropriately-sized
slices. Additionally, obviating the need to copy the data from the
buffer to another location in memory can improve performance dramatically
in CPU-bound applications.
`fwd.Writer` only has one extra method, which is `(*fwd.Writer).Next`, which
returns a slice pointing to the next `n` bytes of the writer, and increments
the write position by the length of the returned slice. This allows users
to write directly to the end of the buffer.
## Constants
``` go
const (
// DefaultReaderSize is the default size of the read buffer
DefaultReaderSize = 2048
)
```
``` go
const (
// DefaultWriterSize is the
// default write buffer size.
DefaultWriterSize = 2048
)
```
## type Reader
``` go
type Reader struct {
// contains filtered or unexported fields
}
```
Reader is a buffered look-ahead reader
### func NewReader
``` go
func NewReader(r io.Reader) *Reader
```
NewReader returns a new *Reader that reads from 'r'
### func NewReaderSize
``` go
func NewReaderSize(r io.Reader, n int) *Reader
```
NewReaderSize returns a new *Reader that
reads from 'r' and has a buffer size 'n'
### func (\*Reader) BufferSize
``` go
func (r *Reader) BufferSize() int
```
BufferSize returns the total size of the buffer
### func (\*Reader) Buffered
``` go
func (r *Reader) Buffered() int
```
Buffered returns the number of bytes currently in the buffer
### func (\*Reader) Next
``` go
func (r *Reader) Next(n int) ([]byte, error)
```
Next returns the next 'n' bytes in the stream.
Unlike Peek, Next advances the reader position.
The returned bytes point to the same
data as the buffer, so the slice is
only valid until the next reader method call.
An EOF is considered an unexpected error.
If an the returned slice is less than the
length asked for, an error will be returned,
and the reader position will not be incremented.
### func (\*Reader) Peek
``` go
func (r *Reader) Peek(n int) ([]byte, error)
```
Peek returns the next 'n' buffered bytes,
reading from the underlying reader if necessary.
It will only return a slice shorter than 'n' bytes
if it also returns an error. Peek does not advance
the reader. EOF errors are *not* returned as
io.ErrUnexpectedEOF.
### func (\*Reader) Read
``` go
func (r *Reader) Read(b []byte) (int, error)
```
Read implements `io.Reader`
### func (\*Reader) ReadByte
``` go
func (r *Reader) ReadByte() (byte, error)
```
ReadByte implements `io.ByteReader`
### func (\*Reader) ReadFull
``` go
func (r *Reader) ReadFull(b []byte) (int, error)
```
ReadFull attempts to read len(b) bytes into
'b'. It returns the number of bytes read into
'b', and an error if it does not return len(b).
EOF is considered an unexpected error.
### func (\*Reader) Reset
``` go
func (r *Reader) Reset(rd io.Reader)
```
Reset resets the underlying reader
and the read buffer.
### func (\*Reader) Skip
``` go
func (r *Reader) Skip(n int) (int, error)
```
Skip moves the reader forward 'n' bytes.
Returns the number of bytes skipped and any
errors encountered. It is analogous to Seek(n, 1).
If the underlying reader implements io.Seeker, then
that method will be used to skip forward.
If the reader encounters
an EOF before skipping 'n' bytes, it
returns io.ErrUnexpectedEOF. If the
underlying reader implements io.Seeker, then
those rules apply instead. (Many implementations
will not return `io.EOF` until the next call
to Read.)
### func (\*Reader) WriteTo
``` go
func (r *Reader) WriteTo(w io.Writer) (int64, error)
```
WriteTo implements `io.WriterTo`
## type Writer
``` go
type Writer struct {
// contains filtered or unexported fields
}
```
Writer is a buffered writer
### func NewWriter
``` go
func NewWriter(w io.Writer) *Writer
```
NewWriter returns a new writer
that writes to 'w' and has a buffer
that is `DefaultWriterSize` bytes.
### func NewWriterSize
``` go
func NewWriterSize(w io.Writer, size int) *Writer
```
NewWriterSize returns a new writer
that writes to 'w' and has a buffer
that is 'size' bytes.
### func (\*Writer) BufferSize
``` go
func (w *Writer) BufferSize() int
```
BufferSize returns the maximum size of the buffer.
### func (\*Writer) Buffered
``` go
func (w *Writer) Buffered() int
```
Buffered returns the number of buffered bytes
in the reader.
### func (\*Writer) Flush
``` go
func (w *Writer) Flush() error
```
Flush flushes any buffered bytes
to the underlying writer.
### func (\*Writer) Next
``` go
func (w *Writer) Next(n int) ([]byte, error)
```
Next returns the next 'n' free bytes
in the write buffer, flushing the writer
as necessary. Next will return `io.ErrShortBuffer`
if 'n' is greater than the size of the write buffer.
Calls to 'next' increment the write position by
the size of the returned buffer.
### func (\*Writer) ReadFrom
``` go
func (w *Writer) ReadFrom(r io.Reader) (int64, error)
```
ReadFrom implements `io.ReaderFrom`
### func (\*Writer) Write
``` go
func (w *Writer) Write(p []byte) (int, error)
```
Write implements `io.Writer`
### func (\*Writer) WriteByte
``` go
func (w *Writer) WriteByte(b byte) error
```
WriteByte implements `io.ByteWriter`
### func (\*Writer) WriteString
``` go
func (w *Writer) WriteString(s string) (int, error)
```
WriteString is analogous to Write, but it takes a string.
- - -
Generated by [godoc2md](http://godoc.org/github.com/davecheney/godoc2md)
\ No newline at end of file
// The `fwd` package provides a buffered reader
// and writer. Each has methods that help improve
// the encoding/decoding performance of some binary
// protocols.
//
// The `fwd.Writer` and `fwd.Reader` type provide similar
// functionality to their counterparts in `bufio`, plus
// a few extra utility methods that simplify read-ahead
// and write-ahead. I wrote this package to improve serialization
// performance for http://github.com/tinylib/msgp,
// where it provided about a 2x speedup over `bufio` for certain
// workloads. However, care must be taken to understand the semantics of the
// extra methods provided by this package, as they allow
// the user to access and manipulate the buffer memory
// directly.
//
// The extra methods for `fwd.Reader` are `Peek`, `Skip`
// and `Next`. `(*fwd.Reader).Peek`, unlike `(*bufio.Reader).Peek`,
// will re-allocate the read buffer in order to accommodate arbitrarily
// large read-ahead. `(*fwd.Reader).Skip` skips the next `n` bytes
// in the stream, and uses the `io.Seeker` interface if the underlying
// stream implements it. `(*fwd.Reader).Next` returns a slice pointing
// to the next `n` bytes in the read buffer (like `Peek`), but also
// increments the read position. This allows users to process streams
// in arbitrary block sizes without having to manage appropriately-sized
// slices. Additionally, obviating the need to copy the data from the
// buffer to another location in memory can improve performance dramatically
// in CPU-bound applications.
//
// `fwd.Writer` only has one extra method, which is `(*fwd.Writer).Next`, which
// returns a slice pointing to the next `n` bytes of the writer, and increments
// the write position by the length of the returned slice. This allows users
// to write directly to the end of the buffer.
//
package fwd
import "io"
const (
// DefaultReaderSize is the default size of the read buffer
DefaultReaderSize = 2048
// minimum read buffer; straight from bufio
minReaderSize = 16
)
// NewReader returns a new *Reader that reads from 'r'
func NewReader(r io.Reader) *Reader {
return NewReaderSize(r, DefaultReaderSize)
}
// NewReaderSize returns a new *Reader that
// reads from 'r' and has a buffer size 'n'
func NewReaderSize(r io.Reader, n int) *Reader {
rd := &Reader{
r: r,
data: make([]byte, 0, max(minReaderSize, n)),
}
if s, ok := r.(io.Seeker); ok {
rd.rs = s
}
return rd
}
// Reader is a buffered look-ahead reader
type Reader struct {
r io.Reader // underlying reader
// data[n:len(data)] is buffered data; data[len(data):cap(data)] is free buffer space
data []byte // data
n int // read offset
state error // last read error
// if the reader past to NewReader was
// also an io.Seeker, this is non-nil
rs io.Seeker
}
// Reset resets the underlying reader
// and the read buffer.
func (r *Reader) Reset(rd io.Reader) {
r.r = rd
r.data = r.data[0:0]
r.n = 0
r.state = nil
if s, ok := rd.(io.Seeker); ok {
r.rs = s
} else {
r.rs = nil
}
}
// more() does one read on the underlying reader
func (r *Reader) more() {
// move data backwards so that
// the read offset is 0; this way
// we can supply the maximum number of
// bytes to the reader
if r.n != 0 {
if r.n < len(r.data) {
r.data = r.data[:copy(r.data[0:], r.data[r.n:])]
} else {
r.data = r.data[:0]
}
r.n = 0
}
var a int
a, r.state = r.r.Read(r.data[len(r.data):cap(r.data)])
if a == 0 && r.state == nil {
r.state = io.ErrNoProgress
return
} else if a > 0 && r.state == io.EOF {
// discard the io.EOF if we read more than 0 bytes.
// the next call to Read should return io.EOF again.
r.state = nil
}
r.data = r.data[:len(r.data)+a]
}
// pop error
func (r *Reader) err() (e error) {
e, r.state = r.state, nil
return
}
// pop error; EOF -> io.ErrUnexpectedEOF
func (r *Reader) noEOF() (e error) {
e, r.state = r.state, nil
if e == io.EOF {
e = io.ErrUnexpectedEOF
}
return
}
// buffered bytes
func (r *Reader) buffered() int { return len(r.data) - r.n }
// Buffered returns the number of bytes currently in the buffer
func (r *Reader) Buffered() int { return len(r.data) - r.n }
// BufferSize returns the total size of the buffer
func (r *Reader) BufferSize() int { return cap(r.data) }
// Peek returns the next 'n' buffered bytes,
// reading from the underlying reader if necessary.
// It will only return a slice shorter than 'n' bytes
// if it also returns an error. Peek does not advance
// the reader. EOF errors are *not* returned as
// io.ErrUnexpectedEOF.
func (r *Reader) Peek(n int) ([]byte, error) {
// in the degenerate case,
// we may need to realloc
// (the caller asked for more
// bytes than the size of the buffer)
if cap(r.data) < n {
old := r.data[r.n:]
r.data = make([]byte, n+r.buffered())
r.data = r.data[:copy(r.data, old)]
r.n = 0
}
// keep filling until
// we hit an error or
// read enough bytes
for r.buffered() < n && r.state == nil {
r.more()
}
// we must have hit an error
if r.buffered() < n {
return r.data[r.n:], r.err()
}
return r.data[r.n : r.n+n], nil
}
// Skip moves the reader forward 'n' bytes.
// Returns the number of bytes skipped and any
// errors encountered. It is analogous to Seek(n, 1).
// If the underlying reader implements io.Seeker, then
// that method will be used to skip forward.
//
// If the reader encounters
// an EOF before skipping 'n' bytes, it
// returns io.ErrUnexpectedEOF. If the
// underlying reader implements io.Seeker, then
// those rules apply instead. (Many implementations
// will not return `io.EOF` until the next call
// to Read.)
func (r *Reader) Skip(n int) (int, error) {
// fast path
if r.buffered() >= n {
r.n += n
return n, nil
}
// use seeker implementation
// if we can
if r.rs != nil {
return r.skipSeek(n)
}
// loop on filling
// and then erasing
o := n
for r.buffered() < n && r.state == nil {
r.more()
// we can skip forward
// up to r.buffered() bytes
step := min(r.buffered(), n)
r.n += step
n -= step
}
// at this point, n should be
// 0 if everything went smoothly
return o - n, r.noEOF()
}
// Next returns the next 'n' bytes in the stream.
// Unlike Peek, Next advances the reader position.
// The returned bytes point to the same
// data as the buffer, so the slice is
// only valid until the next reader method call.
// An EOF is considered an unexpected error.
// If an the returned slice is less than the
// length asked for, an error will be returned,
// and the reader position will not be incremented.
func (r *Reader) Next(n int) ([]byte, error) {
// in case the buffer is too small
if cap(r.data) < n {
old := r.data[r.n:]
r.data = make([]byte, n+r.buffered())
r.data = r.data[:copy(r.data, old)]
r.n = 0
}
// fill at least 'n' bytes
for r.buffered() < n && r.state == nil {
r.more()
}
if r.buffered() < n {
return r.data[r.n:], r.noEOF()
}
out := r.data[r.n : r.n+n]
r.n += n
return out, nil
}
// skipSeek uses the io.Seeker to seek forward.
// only call this function when n > r.buffered()
func (r *Reader) skipSeek(n int) (int, error) {
o := r.buffered()
// first, clear buffer
n -= o
r.n = 0
r.data = r.data[:0]
// then seek forward remaning bytes
i, err := r.rs.Seek(int64(n), 1)
return int(i) + o, err
}
// Read implements `io.Reader`
func (r *Reader) Read(b []byte) (int, error) {
// if we have data in the buffer, just
// return that.
if r.buffered() != 0 {
x := copy(b, r.data[r.n:])
r.n += x
return x, nil
}
var n int
// we have no buffered data; determine
// whether or not to buffer or call
// the underlying reader directly
if len(b) >= cap(r.data) {
n, r.state = r.r.Read(b)
} else {
r.more()
n = copy(b, r.data)
r.n = n
}
if n == 0 {
return 0, r.err()
}
return n, nil
}
// ReadFull attempts to read len(b) bytes into
// 'b'. It returns the number of bytes read into
// 'b', and an error if it does not return len(b).
// EOF is considered an unexpected error.
func (r *Reader) ReadFull(b []byte) (int, error) {
var n int // read into b
var nn int // scratch
l := len(b)
// either read buffered data,
// or read directly for the underlying
// buffer, or fetch more buffered data.
for n < l && r.state == nil {
if r.buffered() != 0 {
nn = copy(b[n:], r.data[r.n:])
n += nn
r.n += nn
} else if l-n > cap(r.data) {
nn, r.state = r.r.Read(b[n:])
n += nn
} else {
r.more()
}
}
if n < l {
return n, r.noEOF()
}
return n, nil
}
// ReadByte implements `io.ByteReader`
func (r *Reader) ReadByte() (byte, error) {
for r.buffered() < 1 && r.state == nil {
r.more()
}
if r.buffered() < 1 {
return 0, r.err()
}
b := r.data[r.n]
r.n++
return b, nil
}
// WriteTo implements `io.WriterTo`
func (r *Reader) WriteTo(w io.Writer) (int64, error) {
var (
i int64
ii int
err error
)
// first, clear buffer
if r.buffered() > 0 {
ii, err = w.Write(r.data[r.n:])
i += int64(ii)
if err != nil {
return i, err
}
r.data = r.data[0:0]
r.n = 0
}
for r.state == nil {
// here we just do
// 1:1 reads and writes
r.more()
if r.buffered() > 0 {
ii, err = w.Write(r.data)
i += int64(ii)
if err != nil {
return i, err
}
r.data = r.data[0:0]
r.n = 0
}
}
if r.state != io.EOF {
return i, r.err()
}
return i, nil
}
func min(a int, b int) int {
if a < b {
return a
}
return b
}
func max(a int, b int) int {
if a < b {
return b
}
return a
}
package fwd
import "io"
const (
// DefaultWriterSize is the
// default write buffer size.
DefaultWriterSize = 2048
minWriterSize = minReaderSize
)
// Writer is a buffered writer
type Writer struct {
w io.Writer // writer
buf []byte // 0:len(buf) is bufered data
}
// NewWriter returns a new writer
// that writes to 'w' and has a buffer
// that is `DefaultWriterSize` bytes.
func NewWriter(w io.Writer) *Writer {
if wr, ok := w.(*Writer); ok {
return wr
}
return &Writer{
w: w,
buf: make([]byte, 0, DefaultWriterSize),
}
}
// NewWriterSize returns a new writer
// that writes to 'w' and has a buffer
// that is 'size' bytes.
func NewWriterSize(w io.Writer, size int) *Writer {
if wr, ok := w.(*Writer); ok && cap(wr.buf) >= size {
return wr
}
return &Writer{
w: w,
buf: make([]byte, 0, max(size, minWriterSize)),
}
}
// Buffered returns the number of buffered bytes
// in the reader.
func (w *Writer) Buffered() int { return len(w.buf) }
// BufferSize returns the maximum size of the buffer.
func (w *Writer) BufferSize() int { return cap(w.buf) }
// Flush flushes any buffered bytes
// to the underlying writer.
func (w *Writer) Flush() error {
l := len(w.buf)
if l > 0 {
n, err := w.w.Write(w.buf)
// if we didn't write the whole
// thing, copy the unwritten
// bytes to the beginnning of the
// buffer.
if n < l && n > 0 {
w.pushback(n)
if err == nil {
err = io.ErrShortWrite
}
}
if err != nil {
return err
}
w.buf = w.buf[:0]
return nil
}
return nil
}
// Write implements `io.Writer`
func (w *Writer) Write(p []byte) (int, error) {
c, l, ln := cap(w.buf), len(w.buf), len(p)
avail := c - l
// requires flush
if avail < ln {
if err := w.Flush(); err != nil {
return 0, err
}
l = len(w.buf)
}
// too big to fit in buffer;
// write directly to w.w
if c < ln {
return w.w.Write(p)
}
// grow buf slice; copy; return
w.buf = w.buf[:l+ln]
return copy(w.buf[l:], p), nil
}
// WriteString is analogous to Write, but it takes a string.
func (w *Writer) WriteString(s string) (int, error) {
c, l, ln := cap(w.buf), len(w.buf), len(s)
avail := c - l
// requires flush
if avail < ln {
if err := w.Flush(); err != nil {
return 0, err
}
l = len(w.buf)
}
// too big to fit in buffer;
// write directly to w.w
//
// yes, this is unsafe. *but*
// io.Writer is not allowed
// to mutate its input or
// maintain a reference to it,
// per the spec in package io.
//
// plus, if the string is really
// too big to fit in the buffer, then
// creating a copy to write it is
// expensive (and, strictly speaking,
// unnecessary)
if c < ln {
return w.w.Write(unsafestr(s))
}
// grow buf slice; copy; return
w.buf = w.buf[:l+ln]
return copy(w.buf[l:], s), nil
}
// WriteByte implements `io.ByteWriter`
func (w *Writer) WriteByte(b byte) error {
if len(w.buf) == cap(w.buf) {
if err := w.Flush(); err != nil {
return err
}
}
w.buf = append(w.buf, b)
return nil
}
// Next returns the next 'n' free bytes
// in the write buffer, flushing the writer
// as necessary. Next will return `io.ErrShortBuffer`
// if 'n' is greater than the size of the write buffer.
// Calls to 'next' increment the write position by
// the size of the returned buffer.
func (w *Writer) Next(n int) ([]byte, error) {
c, l := cap(w.buf), len(w.buf)
if n > c {
return nil, io.ErrShortBuffer
}
avail := c - l
if avail < n {
if err := w.Flush(); err != nil {
return nil, err
}
l = len(w.buf)
}
w.buf = w.buf[:l+n]
return w.buf[l:], nil
}
// take the bytes from w.buf[n:len(w.buf)]
// and put them at the beginning of w.buf,
// and resize to the length of the copied segment.
func (w *Writer) pushback(n int) {
w.buf = w.buf[:copy(w.buf, w.buf[n:])]
}
// ReadFrom implements `io.ReaderFrom`
func (w *Writer) ReadFrom(r io.Reader) (int64, error) {
// anticipatory flush
if err := w.Flush(); err != nil {
return 0, err
}
w.buf = w.buf[0:cap(w.buf)] // expand buffer
var nn int64 // written
var err error // error
var x int // read
// 1:1 reads and writes
for err == nil {
x, err = r.Read(w.buf)
if x > 0 {
n, werr := w.w.Write(w.buf[:x])
nn += int64(n)
if err != nil {
if n < x && n > 0 {
w.pushback(n - x)
}
return nn, werr
}
if n < x {
w.pushback(n - x)
return nn, io.ErrShortWrite
}
} else if err == nil {
err = io.ErrNoProgress
break
}
}
if err != io.EOF {
return nn, err
}
// we only clear here
// because we are sure
// the writes have
// succeeded. otherwise,
// we retain the data in case
// future writes succeed.
w.buf = w.buf[0:0]
return nn, nil
}
// +build appengine
package fwd
func unsafestr(s string) []byte { return []byte(s) }
// +build !appengine
package fwd
import (
"reflect"
"unsafe"
)
// unsafe cast string as []byte
func unsafestr(b string) []byte {
l := len(b)
return *(*[]byte)(unsafe.Pointer(&reflect.SliceHeader{
Len: l,
Cap: l,
Data: (*reflect.StringHeader)(unsafe.Pointer(&b)).Data,
}))
}
Copyright (c) 2014 Philip Hofer
Portions Copyright (c) 2009 The Go Authors (license at http://golang.org) where indicated
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
\ No newline at end of file
// +build linux,!appengine
package msgp
import (
"os"
"syscall"
)
func adviseRead(mem []byte) {
syscall.Madvise(mem, syscall.MADV_SEQUENTIAL|syscall.MADV_WILLNEED)
}
func adviseWrite(mem []byte) {
syscall.Madvise(mem, syscall.MADV_SEQUENTIAL)
}
func fallocate(f *os.File, sz int64) error {
err := syscall.Fallocate(int(f.Fd()), 0, 0, sz)
if err == syscall.ENOTSUP {
return f.Truncate(sz)
}
return err
}
// +build !linux appengine
package msgp
import (
"os"
)
// TODO: darwin, BSD support
func adviseRead(mem []byte) {}
func adviseWrite(mem []byte) {}
func fallocate(f *os.File, sz int64) error {
return f.Truncate(sz)
}
package msgp
type timer interface {
StartTimer()
StopTimer()
}
// EndlessReader is an io.Reader
// that loops over the same data
// endlessly. It is used for benchmarking.
type EndlessReader struct {
tb timer
data []byte
offset int
}
// NewEndlessReader returns a new endless reader
func NewEndlessReader(b []byte, tb timer) *EndlessReader {
return &EndlessReader{tb: tb, data: b, offset: 0}
}
// Read implements io.Reader. In practice, it
// always returns (len(p), nil), although it
// fills the supplied slice while the benchmark
// timer is stopped.
func (c *EndlessReader) Read(p []byte) (int, error) {
c.tb.StopTimer()
var n int
l := len(p)
m := len(c.data)
for n < l {
nn := copy(p[n:], c.data[c.offset:])
n += nn
c.offset += nn
c.offset %= m
}
c.tb.StartTimer()
return n, nil
}
// This package is the support library for the msgp code generator (http://github.com/tinylib/msgp).
//
// This package defines the utilites used by the msgp code generator for encoding and decoding MessagePack
// from []byte and io.Reader/io.Writer types. Much of this package is devoted to helping the msgp code
// generator implement the Marshaler/Unmarshaler and Encodable/Decodable interfaces.
//
// This package defines four "families" of functions:
// - AppendXxxx() appends an object to a []byte in MessagePack encoding.
// - ReadXxxxBytes() reads an object from a []byte and returns the remaining bytes.
// - (*Writer).WriteXxxx() writes an object to the buffered *Writer type.
// - (*Reader).ReadXxxx() reads an object from a buffered *Reader type.
//
// Once a type has satisfied the `Encodable` and `Decodable` interfaces,
// it can be written and read from arbitrary `io.Writer`s and `io.Reader`s using
// msgp.Encode(io.Writer, msgp.Encodable)
// and
// msgp.Decode(io.Reader, msgp.Decodable)
//
// There are also methods for converting MessagePack to JSON without
// an explicit de-serialization step.
//
// For additional tips, tricks, and gotchas, please visit
// the wiki at http://github.com/tinylib/msgp
package msgp
const last4 = 0x0f
const first4 = 0xf0
const last5 = 0x1f
const first3 = 0xe0
const last7 = 0x7f
func isfixint(b byte) bool {
return b>>7 == 0
}
func isnfixint(b byte) bool {
return b&first3 == mnfixint
}
func isfixmap(b byte) bool {
return b&first4 == mfixmap
}
func isfixarray(b byte) bool {
return b&first4 == mfixarray
}
func isfixstr(b byte) bool {
return b&first3 == mfixstr
}
func wfixint(u uint8) byte {
return u & last7
}
func rfixint(b byte) uint8 {
return b
}
func wnfixint(i int8) byte {
return byte(i) | mnfixint
}
func rnfixint(b byte) int8 {
return int8(b)
}
func rfixmap(b byte) uint8 {
return b & last4
}
func wfixmap(u uint8) byte {
return mfixmap | (u & last4)
}
func rfixstr(b byte) uint8 {
return b & last5
}
func wfixstr(u uint8) byte {
return (u & last5) | mfixstr
}
func rfixarray(b byte) uint8 {
return (b & last4)
}
func wfixarray(u uint8) byte {
return (u & last4) | mfixarray
}
// These are all the byte
// prefixes defined by the
// msgpack standard
const (
// 0XXXXXXX
mfixint uint8 = 0x00
// 111XXXXX
mnfixint uint8 = 0xe0
// 1000XXXX
mfixmap uint8 = 0x80
// 1001XXXX
mfixarray uint8 = 0x90
// 101XXXXX
mfixstr uint8 = 0xa0
mnil uint8 = 0xc0
mfalse uint8 = 0xc2
mtrue uint8 = 0xc3
mbin8 uint8 = 0xc4
mbin16 uint8 = 0xc5
mbin32 uint8 = 0xc6
mext8 uint8 = 0xc7
mext16 uint8 = 0xc8
mext32 uint8 = 0xc9
mfloat32 uint8 = 0xca
mfloat64 uint8 = 0xcb
muint8 uint8 = 0xcc
muint16 uint8 = 0xcd
muint32 uint8 = 0xce
muint64 uint8 = 0xcf
mint8 uint8 = 0xd0
mint16 uint8 = 0xd1
mint32 uint8 = 0xd2
mint64 uint8 = 0xd3
mfixext1 uint8 = 0xd4
mfixext2 uint8 = 0xd5
mfixext4 uint8 = 0xd6
mfixext8 uint8 = 0xd7
mfixext16 uint8 = 0xd8
mstr8 uint8 = 0xd9
mstr16 uint8 = 0xda
mstr32 uint8 = 0xdb
marray16 uint8 = 0xdc
marray32 uint8 = 0xdd
mmap16 uint8 = 0xde
mmap32 uint8 = 0xdf
)
package msgp
import (
"math"
)
// Locate returns a []byte pointing to the field
// in a messagepack map with the provided key. (The returned []byte
// points to a sub-slice of 'raw'; Locate does no allocations.) If the
// key doesn't exist in the map, a zero-length []byte will be returned.
func Locate(key string, raw []byte) []byte {
s, n := locate(raw, key)
return raw[s:n]
}
// Replace takes a key ("key") in a messagepack map ("raw")
// and replaces its value with the one provided and returns
// the new []byte. The returned []byte may point to the same
// memory as "raw". Replace makes no effort to evaluate the validity
// of the contents of 'val'. It may use up to the full capacity of 'raw.'
// Replace returns 'nil' if the field doesn't exist or if the object in 'raw'
// is not a map.
func Replace(key string, raw []byte, val []byte) []byte {
start, end := locate(raw, key)
if start == end {
return nil
}
return replace(raw, start, end, val, true)
}
// CopyReplace works similarly to Replace except that the returned
// byte slice does not point to the same memory as 'raw'. CopyReplace
// returns 'nil' if the field doesn't exist or 'raw' isn't a map.
func CopyReplace(key string, raw []byte, val []byte) []byte {
start, end := locate(raw, key)
if start == end {
return nil
}
return replace(raw, start, end, val, false)
}
// Remove removes a key-value pair from 'raw'. It returns
// 'raw' unchanged if the key didn't exist.
func Remove(key string, raw []byte) []byte {
start, end := locateKV(raw, key)
if start == end {
return raw
}
raw = raw[:start+copy(raw[start:], raw[end:])]
return resizeMap(raw, -1)
}
// HasKey returns whether the map in 'raw' has
// a field with key 'key'
func HasKey(key string, raw []byte) bool {
sz, bts, err := ReadMapHeaderBytes(raw)
if err != nil {
return false
}
var field []byte
for i := uint32(0); i < sz; i++ {
field, bts, err = ReadStringZC(bts)
if err != nil {
return false
}
if UnsafeString(field) == key {
return true
}
}
return false
}
func replace(raw []byte, start int, end int, val []byte, inplace bool) []byte {
ll := end - start // length of segment to replace
lv := len(val)
if inplace {
extra := lv - ll
// fastest case: we're doing
// a 1:1 replacement
if extra == 0 {
copy(raw[start:], val)
return raw
} else if extra < 0 {
// 'val' smaller than replaced value
// copy in place and shift back
x := copy(raw[start:], val)
y := copy(raw[start+x:], raw[end:])
return raw[:start+x+y]
} else if extra < cap(raw)-len(raw) {
// 'val' less than (cap-len) extra bytes
// copy in place and shift forward
raw = raw[0 : len(raw)+extra]
// shift end forward
copy(raw[end+extra:], raw[end:])
copy(raw[start:], val)
return raw
}
}
// we have to allocate new space
out := make([]byte, len(raw)+len(val)-ll)
x := copy(out, raw[:start])
y := copy(out[x:], val)
copy(out[x+y:], raw[end:])
return out
}
// locate does a naive O(n) search for the map key; returns start, end
// (returns 0,0 on error)
func locate(raw []byte, key string) (start int, end int) {
var (
sz uint32
bts []byte
field []byte
err error
)
sz, bts, err = ReadMapHeaderBytes(raw)
if err != nil {
return
}
// loop and locate field
for i := uint32(0); i < sz; i++ {
field, bts, err = ReadStringZC(bts)
if err != nil {
return 0, 0
}
if UnsafeString(field) == key {
// start location
l := len(raw)
start = l - len(bts)
bts, err = Skip(bts)
if err != nil {
return 0, 0
}
end = l - len(bts)
return
}
bts, err = Skip(bts)
if err != nil {
return 0, 0
}
}
return 0, 0
}
// locate key AND value
func locateKV(raw []byte, key string) (start int, end int) {
var (
sz uint32
bts []byte
field []byte
err error
)
sz, bts, err = ReadMapHeaderBytes(raw)
if err != nil {
return 0, 0
}
for i := uint32(0); i < sz; i++ {
tmp := len(bts)
field, bts, err = ReadStringZC(bts)
if err != nil {
return 0, 0
}
if UnsafeString(field) == key {
start = len(raw) - tmp
bts, err = Skip(bts)
if err != nil {
return 0, 0
}
end = len(raw) - len(bts)
return
}
bts, err = Skip(bts)
if err != nil {
return 0, 0
}
}
return 0, 0
}
// delta is delta on map size
func resizeMap(raw []byte, delta int64) []byte {
var sz int64
switch raw[0] {
case mmap16:
sz = int64(big.Uint16(raw[1:]))
if sz+delta <= math.MaxUint16 {
big.PutUint16(raw[1:], uint16(sz+delta))
return raw
}
if cap(raw)-len(raw) >= 2 {
raw = raw[0 : len(raw)+2]
copy(raw[5:], raw[3:])
raw[0] = mmap32
big.PutUint32(raw[1:], uint32(sz+delta))
return raw
}
n := make([]byte, 0, len(raw)+5)
n = AppendMapHeader(n, uint32(sz+delta))
return append(n, raw[3:]...)
case mmap32:
sz = int64(big.Uint32(raw[1:]))
big.PutUint32(raw[1:], uint32(sz+delta))
return raw
default:
sz = int64(rfixmap(raw[0]))
if sz+delta < 16 {
raw[0] = wfixmap(uint8(sz + delta))
return raw
} else if sz+delta <= math.MaxUint16 {
if cap(raw)-len(raw) >= 2 {
raw = raw[0 : len(raw)+2]
copy(raw[3:], raw[1:])
raw[0] = mmap16
big.PutUint16(raw[1:], uint16(sz+delta))
return raw
}
n := make([]byte, 0, len(raw)+5)
n = AppendMapHeader(n, uint32(sz+delta))
return append(n, raw[1:]...)
}
if cap(raw)-len(raw) >= 4 {
raw = raw[0 : len(raw)+4]
copy(raw[5:], raw[1:])
raw[0] = mmap32
big.PutUint32(raw[1:], uint32(sz+delta))
return raw
}
n := make([]byte, 0, len(raw)+5)
n = AppendMapHeader(n, uint32(sz+delta))
return append(n, raw[1:]...)
}
}
package msgp
// size of every object on the wire,
// plus type information. gives us
// constant-time type information
// for traversing composite objects.
//
var sizes = [256]bytespec{
mnil: {size: 1, extra: constsize, typ: NilType},
mfalse: {size: 1, extra: constsize, typ: BoolType},
mtrue: {size: 1, extra: constsize, typ: BoolType},
mbin8: {size: 2, extra: extra8, typ: BinType},
mbin16: {size: 3, extra: extra16, typ: BinType},
mbin32: {size: 5, extra: extra32, typ: BinType},
mext8: {size: 3, extra: extra8, typ: ExtensionType},
mext16: {size: 4, extra: extra16, typ: ExtensionType},
mext32: {size: 6, extra: extra32, typ: ExtensionType},
mfloat32: {size: 5, extra: constsize, typ: Float32Type},
mfloat64: {size: 9, extra: constsize, typ: Float64Type},
muint8: {size: 2, extra: constsize, typ: UintType},
muint16: {size: 3, extra: constsize, typ: UintType},
muint32: {size: 5, extra: constsize, typ: UintType},
muint64: {size: 9, extra: constsize, typ: UintType},
mint8: {size: 2, extra: constsize, typ: IntType},
mint16: {size: 3, extra: constsize, typ: IntType},
mint32: {size: 5, extra: constsize, typ: IntType},
mint64: {size: 9, extra: constsize, typ: IntType},
mfixext1: {size: 3, extra: constsize, typ: ExtensionType},
mfixext2: {size: 4, extra: constsize, typ: ExtensionType},
mfixext4: {size: 6, extra: constsize, typ: ExtensionType},
mfixext8: {size: 10, extra: constsize, typ: ExtensionType},
mfixext16: {size: 18, extra: constsize, typ: ExtensionType},
mstr8: {size: 2, extra: extra8, typ: StrType},
mstr16: {size: 3, extra: extra16, typ: StrType},
mstr32: {size: 5, extra: extra32, typ: StrType},
marray16: {size: 3, extra: array16v, typ: ArrayType},
marray32: {size: 5, extra: array32v, typ: ArrayType},
mmap16: {size: 3, extra: map16v, typ: MapType},
mmap32: {size: 5, extra: map32v, typ: MapType},
}
func init() {
// set up fixed fields
// fixint
for i := mfixint; i < 0x80; i++ {
sizes[i] = bytespec{size: 1, extra: constsize, typ: IntType}
}
// nfixint
for i := uint16(mnfixint); i < 0x100; i++ {
sizes[uint8(i)] = bytespec{size: 1, extra: constsize, typ: IntType}
}
// fixstr gets constsize,
// since the prefix yields the size
for i := mfixstr; i < 0xc0; i++ {
sizes[i] = bytespec{size: 1 + rfixstr(i), extra: constsize, typ: StrType}
}
// fixmap
for i := mfixmap; i < 0x90; i++ {
sizes[i] = bytespec{size: 1, extra: varmode(2 * rfixmap(i)), typ: MapType}
}
// fixarray
for i := mfixarray; i < 0xa0; i++ {
sizes[i] = bytespec{size: 1, extra: varmode(rfixarray(i)), typ: ArrayType}
}
}
// a valid bytespsec has
// non-zero 'size' and
// non-zero 'typ'
type bytespec struct {
size uint8 // prefix size information
extra varmode // extra size information
typ Type // type
_ byte // makes bytespec 4 bytes (yes, this matters)
}
// size mode
// if positive, # elements for composites
type varmode int8
const (
constsize varmode = 0 // constant size (size bytes + uint8(varmode) objects)
extra8 = -1 // has uint8(p[1]) extra bytes
extra16 = -2 // has be16(p[1:]) extra bytes
extra32 = -3 // has be32(p[1:]) extra bytes
map16v = -4 // use map16
map32v = -5 // use map32
array16v = -6 // use array16
array32v = -7 // use array32
)
func getType(v byte) Type {
return sizes[v].typ
}
package msgp
import (
"fmt"
"reflect"
)
const resumableDefault = false
var (
// ErrShortBytes is returned when the
// slice being decoded is too short to
// contain the contents of the message
ErrShortBytes error = errShort{}
// this error is only returned
// if we reach code that should
// be unreachable
fatal error = errFatal{}
)
// Error is the interface satisfied
// by all of the errors that originate
// from this package.
type Error interface {
error
// Resumable returns whether
// or not the error means that
// the stream of data is malformed
// and the information is unrecoverable.
Resumable() bool
}
// contextError allows msgp Error instances to be enhanced with additional
// context about their origin.
type contextError interface {
Error
// withContext must not modify the error instance - it must clone and
// return a new error with the context added.
withContext(ctx string) error
}
// Cause returns the underlying cause of an error that has been wrapped
// with additional context.
func Cause(e error) error {
out := e
if e, ok := e.(errWrapped); ok && e.cause != nil {
out = e.cause
}
return out
}
// Resumable returns whether or not the error means that the stream of data is
// malformed and the information is unrecoverable.
func Resumable(e error) bool {
if e, ok := e.(Error); ok {
return e.Resumable()
}
return resumableDefault
}
// WrapError wraps an error with additional context that allows the part of the
// serialized type that caused the problem to be identified. Underlying errors
// can be retrieved using Cause()
//
// The input error is not modified - a new error should be returned.
//
// ErrShortBytes is not wrapped with any context due to backward compatibility
// issues with the public API.
//
func WrapError(err error, ctx ...interface{}) error {
switch e := err.(type) {
case errShort:
return e
case contextError:
return e.withContext(ctxString(ctx))
default:
return errWrapped{cause: err, ctx: ctxString(ctx)}
}
}
// ctxString converts the incoming interface{} slice into a single string.
func ctxString(ctx []interface{}) string {
out := ""
for idx, cv := range ctx {
if idx > 0 {
out += "/"
}
out += fmt.Sprintf("%v", cv)
}
return out
}
func addCtx(ctx, add string) string {
if ctx != "" {
return add + "/" + ctx
} else {
return add
}
}
// errWrapped allows arbitrary errors passed to WrapError to be enhanced with
// context and unwrapped with Cause()
type errWrapped struct {
cause error
ctx string
}
func (e errWrapped) Error() string {
if e.ctx != "" {
return fmt.Sprintf("%s at %s", e.cause, e.ctx)
} else {
return e.cause.Error()
}
}
func (e errWrapped) Resumable() bool {
if e, ok := e.cause.(Error); ok {
return e.Resumable()
}
return resumableDefault
}
type errShort struct{}
func (e errShort) Error() string { return "msgp: too few bytes left to read object" }
func (e errShort) Resumable() bool { return false }
type errFatal struct {
ctx string
}
func (f errFatal) Error() string {
out := "msgp: fatal decoding error (unreachable code)"
if f.ctx != "" {
out += " at " + f.ctx
}
return out
}
func (f errFatal) Resumable() bool { return false }
func (f errFatal) withContext(ctx string) error { f.ctx = addCtx(f.ctx, ctx); return f }
// ArrayError is an error returned
// when decoding a fix-sized array
// of the wrong size
type ArrayError struct {
Wanted uint32
Got uint32
ctx string
}
// Error implements the error interface
func (a ArrayError) Error() string {
out := fmt.Sprintf("msgp: wanted array of size %d; got %d", a.Wanted, a.Got)
if a.ctx != "" {
out += " at " + a.ctx
}
return out
}
// Resumable is always 'true' for ArrayErrors
func (a ArrayError) Resumable() bool { return true }
func (a ArrayError) withContext(ctx string) error { a.ctx = addCtx(a.ctx, ctx); return a }
// IntOverflow is returned when a call
// would downcast an integer to a type
// with too few bits to hold its value.
type IntOverflow struct {
Value int64 // the value of the integer
FailedBitsize int // the bit size that the int64 could not fit into
ctx string
}
// Error implements the error interface
func (i IntOverflow) Error() string {
str := fmt.Sprintf("msgp: %d overflows int%d", i.Value, i.FailedBitsize)
if i.ctx != "" {
str += " at " + i.ctx
}
return str
}
// Resumable is always 'true' for overflows
func (i IntOverflow) Resumable() bool { return true }
func (i IntOverflow) withContext(ctx string) error { i.ctx = addCtx(i.ctx, ctx); return i }
// UintOverflow is returned when a call
// would downcast an unsigned integer to a type
// with too few bits to hold its value
type UintOverflow struct {
Value uint64 // value of the uint
FailedBitsize int // the bit size that couldn't fit the value
ctx string
}
// Error implements the error interface
func (u UintOverflow) Error() string {
str := fmt.Sprintf("msgp: %d overflows uint%d", u.Value, u.FailedBitsize)
if u.ctx != "" {
str += " at " + u.ctx
}
return str
}
// Resumable is always 'true' for overflows
func (u UintOverflow) Resumable() bool { return true }
func (u UintOverflow) withContext(ctx string) error { u.ctx = addCtx(u.ctx, ctx); return u }
// UintBelowZero is returned when a call
// would cast a signed integer below zero
// to an unsigned integer.
type UintBelowZero struct {
Value int64 // value of the incoming int
ctx string
}
// Error implements the error interface
func (u UintBelowZero) Error() string {
str := fmt.Sprintf("msgp: attempted to cast int %d to unsigned", u.Value)
if u.ctx != "" {
str += " at " + u.ctx
}
return str
}
// Resumable is always 'true' for overflows
func (u UintBelowZero) Resumable() bool { return true }
func (u UintBelowZero) withContext(ctx string) error {
u.ctx = ctx
return u
}
// A TypeError is returned when a particular
// decoding method is unsuitable for decoding
// a particular MessagePack value.
type TypeError struct {
Method Type // Type expected by method
Encoded Type // Type actually encoded
ctx string
}
// Error implements the error interface
func (t TypeError) Error() string {
out := fmt.Sprintf("msgp: attempted to decode type %q with method for %q", t.Encoded, t.Method)
if t.ctx != "" {
out += " at " + t.ctx
}
return out
}
// Resumable returns 'true' for TypeErrors
func (t TypeError) Resumable() bool { return true }
func (t TypeError) withContext(ctx string) error { t.ctx = addCtx(t.ctx, ctx); return t }
// returns either InvalidPrefixError or
// TypeError depending on whether or not
// the prefix is recognized
func badPrefix(want Type, lead byte) error {
t := sizes[lead].typ
if t == InvalidType {
return InvalidPrefixError(lead)
}
return TypeError{Method: want, Encoded: t}
}
// InvalidPrefixError is returned when a bad encoding
// uses a prefix that is not recognized in the MessagePack standard.
// This kind of error is unrecoverable.
type InvalidPrefixError byte
// Error implements the error interface
func (i InvalidPrefixError) Error() string {
return fmt.Sprintf("msgp: unrecognized type prefix 0x%x", byte(i))
}
// Resumable returns 'false' for InvalidPrefixErrors
func (i InvalidPrefixError) Resumable() bool { return false }
// ErrUnsupportedType is returned
// when a bad argument is supplied
// to a function that takes `interface{}`.
type ErrUnsupportedType struct {
T reflect.Type
ctx string
}
// Error implements error
func (e *ErrUnsupportedType) Error() string {
out := fmt.Sprintf("msgp: type %q not supported", e.T)
if e.ctx != "" {
out += " at " + e.ctx
}
return out
}
// Resumable returns 'true' for ErrUnsupportedType
func (e *ErrUnsupportedType) Resumable() bool { return true }
func (e *ErrUnsupportedType) withContext(ctx string) error {
o := *e
o.ctx = addCtx(o.ctx, ctx)
return &o
}
package msgp
import (
"fmt"
"math"
)
const (
// Complex64Extension is the extension number used for complex64
Complex64Extension = 3
// Complex128Extension is the extension number used for complex128
Complex128Extension = 4
// TimeExtension is the extension number used for time.Time
TimeExtension = 5
)
// our extensions live here
var extensionReg = make(map[int8]func() Extension)
// RegisterExtension registers extensions so that they
// can be initialized and returned by methods that
// decode `interface{}` values. This should only
// be called during initialization. f() should return
// a newly-initialized zero value of the extension. Keep in
// mind that extensions 3, 4, and 5 are reserved for
// complex64, complex128, and time.Time, respectively,
// and that MessagePack reserves extension types from -127 to -1.
//
// For example, if you wanted to register a user-defined struct:
//
// msgp.RegisterExtension(10, func() msgp.Extension { &MyExtension{} })
//
// RegisterExtension will panic if you call it multiple times
// with the same 'typ' argument, or if you use a reserved
// type (3, 4, or 5).
func RegisterExtension(typ int8, f func() Extension) {
switch typ {
case Complex64Extension, Complex128Extension, TimeExtension:
panic(fmt.Sprint("msgp: forbidden extension type:", typ))
}
if _, ok := extensionReg[typ]; ok {
panic(fmt.Sprint("msgp: RegisterExtension() called with typ", typ, "more than once"))
}
extensionReg[typ] = f
}
// ExtensionTypeError is an error type returned
// when there is a mis-match between an extension type
// and the type encoded on the wire
type ExtensionTypeError struct {
Got int8
Want int8
}
// Error implements the error interface
func (e ExtensionTypeError) Error() string {
return fmt.Sprintf("msgp: error decoding extension: wanted type %d; got type %d", e.Want, e.Got)
}
// Resumable returns 'true' for ExtensionTypeErrors
func (e ExtensionTypeError) Resumable() bool { return true }
func errExt(got int8, wanted int8) error {
return ExtensionTypeError{Got: got, Want: wanted}
}
// Extension is the interface fulfilled
// by types that want to define their
// own binary encoding.
type Extension interface {
// ExtensionType should return
// a int8 that identifies the concrete
// type of the extension. (Types <0 are
// officially reserved by the MessagePack
// specifications.)
ExtensionType() int8
// Len should return the length
// of the data to be encoded
Len() int
// MarshalBinaryTo should copy
// the data into the supplied slice,
// assuming that the slice has length Len()
MarshalBinaryTo([]byte) error
UnmarshalBinary([]byte) error
}
// RawExtension implements the Extension interface
type RawExtension struct {
Data []byte
Type int8
}
// ExtensionType implements Extension.ExtensionType, and returns r.Type
func (r *RawExtension) ExtensionType() int8 { return r.Type }
// Len implements Extension.Len, and returns len(r.Data)
func (r *RawExtension) Len() int { return len(r.Data) }
// MarshalBinaryTo implements Extension.MarshalBinaryTo,
// and returns a copy of r.Data
func (r *RawExtension) MarshalBinaryTo(d []byte) error {
copy(d, r.Data)
return nil
}
// UnmarshalBinary implements Extension.UnmarshalBinary,
// and sets r.Data to the contents of the provided slice
func (r *RawExtension) UnmarshalBinary(b []byte) error {
if cap(r.Data) >= len(b) {
r.Data = r.Data[0:len(b)]
} else {
r.Data = make([]byte, len(b))
}
copy(r.Data, b)
return nil
}
// WriteExtension writes an extension type to the writer
func (mw *Writer) WriteExtension(e Extension) error {
l := e.Len()
var err error
switch l {
case 0:
o, err := mw.require(3)
if err != nil {
return err
}
mw.buf[o] = mext8
mw.buf[o+1] = 0
mw.buf[o+2] = byte(e.ExtensionType())
case 1:
o, err := mw.require(2)
if err != nil {
return err
}
mw.buf[o] = mfixext1
mw.buf[o+1] = byte(e.ExtensionType())
case 2:
o, err := mw.require(2)
if err != nil {
return err
}
mw.buf[o] = mfixext2
mw.buf[o+1] = byte(e.ExtensionType())
case 4:
o, err := mw.require(2)
if err != nil {
return err
}
mw.buf[o] = mfixext4
mw.buf[o+1] = byte(e.ExtensionType())
case 8:
o, err := mw.require(2)
if err != nil {
return err
}
mw.buf[o] = mfixext8
mw.buf[o+1] = byte(e.ExtensionType())
case 16:
o, err := mw.require(2)
if err != nil {
return err
}
mw.buf[o] = mfixext16
mw.buf[o+1] = byte(e.ExtensionType())
default:
switch {
case l < math.MaxUint8:
o, err := mw.require(3)
if err != nil {
return err
}
mw.buf[o] = mext8
mw.buf[o+1] = byte(uint8(l))
mw.buf[o+2] = byte(e.ExtensionType())
case l < math.MaxUint16:
o, err := mw.require(4)
if err != nil {
return err
}
mw.buf[o] = mext16
big.PutUint16(mw.buf[o+1:], uint16(l))
mw.buf[o+3] = byte(e.ExtensionType())
default:
o, err := mw.require(6)
if err != nil {
return err
}
mw.buf[o] = mext32
big.PutUint32(mw.buf[o+1:], uint32(l))
mw.buf[o+5] = byte(e.ExtensionType())
}
}
// we can only write directly to the
// buffer if we're sure that it
// fits the object
if l <= mw.bufsize() {
o, err := mw.require(l)
if err != nil {
return err
}
return e.MarshalBinaryTo(mw.buf[o:])
}
// here we create a new buffer
// just large enough for the body
// and save it as the write buffer
err = mw.flush()
if err != nil {
return err
}
buf := make([]byte, l)
err = e.MarshalBinaryTo(buf)
if err != nil {
return err
}
mw.buf = buf
mw.wloc = l
return nil
}
// peek at the extension type, assuming the next
// kind to be read is Extension
func (m *Reader) peekExtensionType() (int8, error) {
p, err := m.R.Peek(2)
if err != nil {
return 0, err
}
spec := sizes[p[0]]
if spec.typ != ExtensionType {
return 0, badPrefix(ExtensionType, p[0])
}
if spec.extra == constsize {
return int8(p[1]), nil
}
size := spec.size
p, err = m.R.Peek(int(size))
if err != nil {
return 0, err
}
return int8(p[size-1]), nil
}
// peekExtension peeks at the extension encoding type
// (must guarantee at least 1 byte in 'b')
func peekExtension(b []byte) (int8, error) {
spec := sizes[b[0]]
size := spec.size
if spec.typ != ExtensionType {
return 0, badPrefix(ExtensionType, b[0])
}
if len(b) < int(size) {
return 0, ErrShortBytes
}
// for fixed extensions,
// the type information is in
// the second byte
if spec.extra == constsize {
return int8(b[1]), nil
}
// otherwise, it's in the last
// part of the prefix
return int8(b[size-1]), nil
}
// ReadExtension reads the next object from the reader
// as an extension. ReadExtension will fail if the next
// object in the stream is not an extension, or if
// e.Type() is not the same as the wire type.
func (m *Reader) ReadExtension(e Extension) (err error) {
var p []byte
p, err = m.R.Peek(2)
if err != nil {
return
}
lead := p[0]
var read int
var off int
switch lead {
case mfixext1:
if int8(p[1]) != e.ExtensionType() {
err = errExt(int8(p[1]), e.ExtensionType())
return
}
p, err = m.R.Peek(3)
if err != nil {
return
}
err = e.UnmarshalBinary(p[2:])
if err == nil {
_, err = m.R.Skip(3)
}
return
case mfixext2:
if int8(p[1]) != e.ExtensionType() {
err = errExt(int8(p[1]), e.ExtensionType())
return
}
p, err = m.R.Peek(4)
if err != nil {
return
}
err = e.UnmarshalBinary(p[2:])
if err == nil {
_, err = m.R.Skip(4)
}
return
case mfixext4:
if int8(p[1]) != e.ExtensionType() {
err = errExt(int8(p[1]), e.ExtensionType())
return
}
p, err = m.R.Peek(6)
if err != nil {
return
}
err = e.UnmarshalBinary(p[2:])
if err == nil {
_, err = m.R.Skip(6)
}
return
case mfixext8:
if int8(p[1]) != e.ExtensionType() {
err = errExt(int8(p[1]), e.ExtensionType())
return
}
p, err = m.R.Peek(10)
if err != nil {
return
}
err = e.UnmarshalBinary(p[2:])
if err == nil {
_, err = m.R.Skip(10)
}
return
case mfixext16:
if int8(p[1]) != e.ExtensionType() {
err = errExt(int8(p[1]), e.ExtensionType())
return
}
p, err = m.R.Peek(18)
if err != nil {
return
}
err = e.UnmarshalBinary(p[2:])
if err == nil {
_, err = m.R.Skip(18)
}
return
case mext8:
p, err = m.R.Peek(3)
if err != nil {
return
}
if int8(p[2]) != e.ExtensionType() {
err = errExt(int8(p[2]), e.ExtensionType())
return
}
read = int(uint8(p[1]))
off = 3
case mext16:
p, err = m.R.Peek(4)
if err != nil {
return
}
if int8(p[3]) != e.ExtensionType() {
err = errExt(int8(p[3]), e.ExtensionType())
return
}
read = int(big.Uint16(p[1:]))
off = 4
case mext32:
p, err = m.R.Peek(6)
if err != nil {
return
}
if int8(p[5]) != e.ExtensionType() {
err = errExt(int8(p[5]), e.ExtensionType())
return
}
read = int(big.Uint32(p[1:]))
off = 6
default:
err = badPrefix(ExtensionType, lead)
return
}
p, err = m.R.Peek(read + off)
if err != nil {
return
}
err = e.UnmarshalBinary(p[off:])
if err == nil {
_, err = m.R.Skip(read + off)
}
return
}
// AppendExtension appends a MessagePack extension to the provided slice
func AppendExtension(b []byte, e Extension) ([]byte, error) {
l := e.Len()
var o []byte
var n int
switch l {
case 0:
o, n = ensure(b, 3)
o[n] = mext8
o[n+1] = 0
o[n+2] = byte(e.ExtensionType())
return o[:n+3], nil
case 1:
o, n = ensure(b, 3)
o[n] = mfixext1
o[n+1] = byte(e.ExtensionType())
n += 2
case 2:
o, n = ensure(b, 4)
o[n] = mfixext2
o[n+1] = byte(e.ExtensionType())
n += 2
case 4:
o, n = ensure(b, 6)
o[n] = mfixext4
o[n+1] = byte(e.ExtensionType())
n += 2
case 8:
o, n = ensure(b, 10)
o[n] = mfixext8
o[n+1] = byte(e.ExtensionType())
n += 2
case 16:
o, n = ensure(b, 18)
o[n] = mfixext16
o[n+1] = byte(e.ExtensionType())
n += 2
default:
switch {
case l < math.MaxUint8:
o, n = ensure(b, l+3)
o[n] = mext8
o[n+1] = byte(uint8(l))
o[n+2] = byte(e.ExtensionType())
n += 3
case l < math.MaxUint16:
o, n = ensure(b, l+4)
o[n] = mext16
big.PutUint16(o[n+1:], uint16(l))
o[n+3] = byte(e.ExtensionType())
n += 4
default:
o, n = ensure(b, l+6)
o[n] = mext32
big.PutUint32(o[n+1:], uint32(l))
o[n+5] = byte(e.ExtensionType())
n += 6
}
}
return o, e.MarshalBinaryTo(o[n:])
}
// ReadExtensionBytes reads an extension from 'b' into 'e'
// and returns any remaining bytes.
// Possible errors:
// - ErrShortBytes ('b' not long enough)
// - ExtensionTypeErorr{} (wire type not the same as e.Type())
// - TypeErorr{} (next object not an extension)
// - InvalidPrefixError
// - An umarshal error returned from e.UnmarshalBinary
func ReadExtensionBytes(b []byte, e Extension) ([]byte, error) {
l := len(b)
if l < 3 {
return b, ErrShortBytes
}
lead := b[0]
var (
sz int // size of 'data'
off int // offset of 'data'
typ int8
)
switch lead {
case mfixext1:
typ = int8(b[1])
sz = 1
off = 2
case mfixext2:
typ = int8(b[1])
sz = 2
off = 2
case mfixext4:
typ = int8(b[1])
sz = 4
off = 2
case mfixext8:
typ = int8(b[1])
sz = 8
off = 2
case mfixext16:
typ = int8(b[1])
sz = 16
off = 2
case mext8:
sz = int(uint8(b[1]))
typ = int8(b[2])
off = 3
if sz == 0 {
return b[3:], e.UnmarshalBinary(b[3:3])
}
case mext16:
if l < 4 {
return b, ErrShortBytes
}
sz = int(big.Uint16(b[1:]))
typ = int8(b[3])
off = 4
case mext32:
if l < 6 {
return b, ErrShortBytes
}
sz = int(big.Uint32(b[1:]))
typ = int8(b[5])
off = 6
default:
return b, badPrefix(ExtensionType, lead)
}
if typ != e.ExtensionType() {
return b, errExt(typ, e.ExtensionType())
}
// the data of the extension starts
// at 'off' and is 'sz' bytes long
if len(b[off:]) < sz {
return b, ErrShortBytes
}
tot := off + sz
return b[tot:], e.UnmarshalBinary(b[off:tot])
}
// +build linux darwin dragonfly freebsd netbsd openbsd
// +build !appengine
package msgp
import (
"os"
"syscall"
)
// ReadFile reads a file into 'dst' using
// a read-only memory mapping. Consequently,
// the file must be mmap-able, and the
// Unmarshaler should never write to
// the source memory. (Methods generated
// by the msgp tool obey that constraint, but
// user-defined implementations may not.)
//
// Reading and writing through file mappings
// is only efficient for large files; small
// files are best read and written using
// the ordinary streaming interfaces.
//
func ReadFile(dst Unmarshaler, file *os.File) error {
stat, err := file.Stat()
if err != nil {
return err
}
data, err := syscall.Mmap(int(file.Fd()), 0, int(stat.Size()), syscall.PROT_READ, syscall.MAP_SHARED)
if err != nil {
return err
}
adviseRead(data)
_, err = dst.UnmarshalMsg(data)
uerr := syscall.Munmap(data)
if err == nil {
err = uerr
}
return err
}
// MarshalSizer is the combination
// of the Marshaler and Sizer
// interfaces.
type MarshalSizer interface {
Marshaler
Sizer
}
// WriteFile writes a file from 'src' using
// memory mapping. It overwrites the entire
// contents of the previous file.
// The mapping size is calculated
// using the `Msgsize()` method
// of 'src', so it must produce a result
// equal to or greater than the actual encoded
// size of the object. Otherwise,
// a fault (SIGBUS) will occur.
//
// Reading and writing through file mappings
// is only efficient for large files; small
// files are best read and written using
// the ordinary streaming interfaces.
//
// NOTE: The performance of this call
// is highly OS- and filesystem-dependent.
// Users should take care to test that this
// performs as expected in a production environment.
// (Linux users should run a kernel and filesystem
// that support fallocate(2) for the best results.)
func WriteFile(src MarshalSizer, file *os.File) error {
sz := src.Msgsize()
err := fallocate(file, int64(sz))
if err != nil {
return err
}
data, err := syscall.Mmap(int(file.Fd()), 0, sz, syscall.PROT_READ|syscall.PROT_WRITE, syscall.MAP_SHARED)
if err != nil {
return err
}
adviseWrite(data)
chunk := data[:0]
chunk, err = src.MarshalMsg(chunk)
if err != nil {
return err
}
uerr := syscall.Munmap(data)
if uerr != nil {
return uerr
}
return file.Truncate(int64(len(chunk)))
}
// +build windows appengine
package msgp
import (
"io/ioutil"
"os"
)
// MarshalSizer is the combination
// of the Marshaler and Sizer
// interfaces.
type MarshalSizer interface {
Marshaler
Sizer
}
func ReadFile(dst Unmarshaler, file *os.File) error {
if u, ok := dst.(Decodable); ok {
return u.DecodeMsg(NewReader(file))
}
data, err := ioutil.ReadAll(file)
if err != nil {
return err
}
_, err = dst.UnmarshalMsg(data)
return err
}
func WriteFile(src MarshalSizer, file *os.File) error {
if e, ok := src.(Encodable); ok {
w := NewWriter(file)
err := e.EncodeMsg(w)
if err == nil {
err = w.Flush()
}
return err
}
raw, err := src.MarshalMsg(nil)
if err != nil {
return err
}
_, err = file.Write(raw)
return err
}
package msgp
/* ----------------------------------
integer encoding utilities
(inline-able)
TODO(tinylib): there are faster,
albeit non-portable solutions
to the code below. implement
byteswap?
---------------------------------- */
func putMint64(b []byte, i int64) {
b[0] = mint64
b[1] = byte(i >> 56)
b[2] = byte(i >> 48)
b[3] = byte(i >> 40)
b[4] = byte(i >> 32)
b[5] = byte(i >> 24)
b[6] = byte(i >> 16)
b[7] = byte(i >> 8)
b[8] = byte(i)
}
func getMint64(b []byte) int64 {
return (int64(b[1]) << 56) | (int64(b[2]) << 48) |
(int64(b[3]) << 40) | (int64(b[4]) << 32) |
(int64(b[5]) << 24) | (int64(b[6]) << 16) |
(int64(b[7]) << 8) | (int64(b[8]))
}
func putMint32(b []byte, i int32) {
b[0] = mint32
b[1] = byte(i >> 24)
b[2] = byte(i >> 16)
b[3] = byte(i >> 8)
b[4] = byte(i)
}
func getMint32(b []byte) int32 {
return (int32(b[1]) << 24) | (int32(b[2]) << 16) | (int32(b[3]) << 8) | (int32(b[4]))
}
func putMint16(b []byte, i int16) {
b[0] = mint16
b[1] = byte(i >> 8)
b[2] = byte(i)
}
func getMint16(b []byte) (i int16) {
return (int16(b[1]) << 8) | int16(b[2])
}
func putMint8(b []byte, i int8) {
b[0] = mint8
b[1] = byte(i)
}
func getMint8(b []byte) (i int8) {
return int8(b[1])
}
func putMuint64(b []byte, u uint64) {
b[0] = muint64
b[1] = byte(u >> 56)
b[2] = byte(u >> 48)
b[3] = byte(u >> 40)
b[4] = byte(u >> 32)
b[5] = byte(u >> 24)
b[6] = byte(u >> 16)
b[7] = byte(u >> 8)
b[8] = byte(u)
}
func getMuint64(b []byte) uint64 {
return (uint64(b[1]) << 56) | (uint64(b[2]) << 48) |
(uint64(b[3]) << 40) | (uint64(b[4]) << 32) |
(uint64(b[5]) << 24) | (uint64(b[6]) << 16) |
(uint64(b[7]) << 8) | (uint64(b[8]))
}
func putMuint32(b []byte, u uint32) {
b[0] = muint32
b[1] = byte(u >> 24)
b[2] = byte(u >> 16)
b[3] = byte(u >> 8)
b[4] = byte(u)
}
func getMuint32(b []byte) uint32 {
return (uint32(b[1]) << 24) | (uint32(b[2]) << 16) | (uint32(b[3]) << 8) | (uint32(b[4]))
}
func putMuint16(b []byte, u uint16) {
b[0] = muint16
b[1] = byte(u >> 8)
b[2] = byte(u)
}
func getMuint16(b []byte) uint16 {
return (uint16(b[1]) << 8) | uint16(b[2])
}
func putMuint8(b []byte, u uint8) {
b[0] = muint8
b[1] = byte(u)
}
func getMuint8(b []byte) uint8 {
return uint8(b[1])
}
func getUnix(b []byte) (sec int64, nsec int32) {
sec = (int64(b[0]) << 56) | (int64(b[1]) << 48) |
(int64(b[2]) << 40) | (int64(b[3]) << 32) |
(int64(b[4]) << 24) | (int64(b[5]) << 16) |
(int64(b[6]) << 8) | (int64(b[7]))
nsec = (int32(b[8]) << 24) | (int32(b[9]) << 16) | (int32(b[10]) << 8) | (int32(b[11]))
return
}
func putUnix(b []byte, sec int64, nsec int32) {
b[0] = byte(sec >> 56)
b[1] = byte(sec >> 48)
b[2] = byte(sec >> 40)
b[3] = byte(sec >> 32)
b[4] = byte(sec >> 24)
b[5] = byte(sec >> 16)
b[6] = byte(sec >> 8)
b[7] = byte(sec)
b[8] = byte(nsec >> 24)
b[9] = byte(nsec >> 16)
b[10] = byte(nsec >> 8)
b[11] = byte(nsec)
}
/* -----------------------------
prefix utilities
----------------------------- */
// write prefix and uint8
func prefixu8(b []byte, pre byte, sz uint8) {
b[0] = pre
b[1] = byte(sz)
}
// write prefix and big-endian uint16
func prefixu16(b []byte, pre byte, sz uint16) {
b[0] = pre
b[1] = byte(sz >> 8)
b[2] = byte(sz)
}
// write prefix and big-endian uint32
func prefixu32(b []byte, pre byte, sz uint32) {
b[0] = pre
b[1] = byte(sz >> 24)
b[2] = byte(sz >> 16)
b[3] = byte(sz >> 8)
b[4] = byte(sz)
}
func prefixu64(b []byte, pre byte, sz uint64) {
b[0] = pre
b[1] = byte(sz >> 56)
b[2] = byte(sz >> 48)
b[3] = byte(sz >> 40)
b[4] = byte(sz >> 32)
b[5] = byte(sz >> 24)
b[6] = byte(sz >> 16)
b[7] = byte(sz >> 8)
b[8] = byte(sz)
}
package msgp
import (
"bufio"
"encoding/base64"
"encoding/json"
"io"
"strconv"
"unicode/utf8"
)
var (
null = []byte("null")
hex = []byte("0123456789abcdef")
)
var defuns [_maxtype]func(jsWriter, *Reader) (int, error)
// note: there is an initialization loop if
// this isn't set up during init()
func init() {
// since none of these functions are inline-able,
// there is not much of a penalty to the indirect
// call. however, this is best expressed as a jump-table...
defuns = [_maxtype]func(jsWriter, *Reader) (int, error){
StrType: rwString,
BinType: rwBytes,
MapType: rwMap,
ArrayType: rwArray,
Float64Type: rwFloat64,
Float32Type: rwFloat32,
BoolType: rwBool,
IntType: rwInt,
UintType: rwUint,
NilType: rwNil,
ExtensionType: rwExtension,
Complex64Type: rwExtension,
Complex128Type: rwExtension,
TimeType: rwTime,
}
}
// this is the interface
// used to write json
type jsWriter interface {
io.Writer
io.ByteWriter
WriteString(string) (int, error)
}
// CopyToJSON reads MessagePack from 'src' and copies it
// as JSON to 'dst' until EOF.
func CopyToJSON(dst io.Writer, src io.Reader) (n int64, err error) {
r := NewReader(src)
n, err = r.WriteToJSON(dst)
freeR(r)
return
}
// WriteToJSON translates MessagePack from 'r' and writes it as
// JSON to 'w' until the underlying reader returns io.EOF. It returns
// the number of bytes written, and an error if it stopped before EOF.
func (r *Reader) WriteToJSON(w io.Writer) (n int64, err error) {
var j jsWriter
var bf *bufio.Writer
if jsw, ok := w.(jsWriter); ok {
j = jsw
} else {
bf = bufio.NewWriter(w)
j = bf
}
var nn int
for err == nil {
nn, err = rwNext(j, r)
n += int64(nn)
}
if err != io.EOF {
if bf != nil {
bf.Flush()
}
return
}
err = nil
if bf != nil {
err = bf.Flush()
}
return
}
func rwNext(w jsWriter, src *Reader) (int, error) {
t, err := src.NextType()
if err != nil {
return 0, err
}
return defuns[t](w, src)
}
func rwMap(dst jsWriter, src *Reader) (n int, err error) {
var comma bool
var sz uint32
var field []byte
sz, err = src.ReadMapHeader()
if err != nil {
return
}
if sz == 0 {
return dst.WriteString("{}")
}
err = dst.WriteByte('{')
if err != nil {
return
}
n++
var nn int
for i := uint32(0); i < sz; i++ {
if comma {
err = dst.WriteByte(',')
if err != nil {
return
}
n++
}
field, err = src.ReadMapKeyPtr()
if err != nil {
return
}
nn, err = rwquoted(dst, field)
n += nn
if err != nil {
return
}
err = dst.WriteByte(':')
if err != nil {
return
}
n++
nn, err = rwNext(dst, src)
n += nn
if err != nil {
return
}
if !comma {
comma = true
}
}
err = dst.WriteByte('}')
if err != nil {
return
}
n++
return
}
func rwArray(dst jsWriter, src *Reader) (n int, err error) {
err = dst.WriteByte('[')
if err != nil {
return
}
var sz uint32
var nn int
sz, err = src.ReadArrayHeader()
if err != nil {
return
}
comma := false
for i := uint32(0); i < sz; i++ {
if comma {
err = dst.WriteByte(',')
if err != nil {
return
}
n++
}
nn, err = rwNext(dst, src)
n += nn
if err != nil {
return
}
comma = true
}
err = dst.WriteByte(']')
if err != nil {
return
}
n++
return
}
func rwNil(dst jsWriter, src *Reader) (int, error) {
err := src.ReadNil()
if err != nil {
return 0, err
}
return dst.Write(null)
}
func rwFloat32(dst jsWriter, src *Reader) (int, error) {
f, err := src.ReadFloat32()
if err != nil {
return 0, err
}
src.scratch = strconv.AppendFloat(src.scratch[:0], float64(f), 'f', -1, 64)
return dst.Write(src.scratch)
}
func rwFloat64(dst jsWriter, src *Reader) (int, error) {
f, err := src.ReadFloat64()
if err != nil {
return 0, err
}
src.scratch = strconv.AppendFloat(src.scratch[:0], f, 'f', -1, 32)
return dst.Write(src.scratch)
}
func rwInt(dst jsWriter, src *Reader) (int, error) {
i, err := src.ReadInt64()
if err != nil {
return 0, err
}
src.scratch = strconv.AppendInt(src.scratch[:0], i, 10)
return dst.Write(src.scratch)
}
func rwUint(dst jsWriter, src *Reader) (int, error) {
u, err := src.ReadUint64()
if err != nil {
return 0, err
}
src.scratch = strconv.AppendUint(src.scratch[:0], u, 10)
return dst.Write(src.scratch)
}
func rwBool(dst jsWriter, src *Reader) (int, error) {
b, err := src.ReadBool()
if err != nil {
return 0, err
}
if b {
return dst.WriteString("true")
}
return dst.WriteString("false")
}
func rwTime(dst jsWriter, src *Reader) (int, error) {
t, err := src.ReadTime()
if err != nil {
return 0, err
}
bts, err := t.MarshalJSON()
if err != nil {
return 0, err
}
return dst.Write(bts)
}
func rwExtension(dst jsWriter, src *Reader) (n int, err error) {
et, err := src.peekExtensionType()
if err != nil {
return 0, err
}
// registered extensions can override
// the JSON encoding
if j, ok := extensionReg[et]; ok {
var bts []byte
e := j()
err = src.ReadExtension(e)
if err != nil {
return
}
bts, err = json.Marshal(e)
if err != nil {
return
}
return dst.Write(bts)
}
e := RawExtension{}
e.Type = et
err = src.ReadExtension(&e)
if err != nil {
return
}
var nn int
err = dst.WriteByte('{')
if err != nil {
return
}
n++
nn, err = dst.WriteString(`"type:"`)
n += nn
if err != nil {
return
}
src.scratch = strconv.AppendInt(src.scratch[0:0], int64(e.Type), 10)
nn, err = dst.Write(src.scratch)
n += nn
if err != nil {
return
}
nn, err = dst.WriteString(`,"data":"`)
n += nn
if err != nil {
return
}
enc := base64.NewEncoder(base64.StdEncoding, dst)
nn, err = enc.Write(e.Data)
n += nn
if err != nil {
return
}
err = enc.Close()
if err != nil {
return
}
nn, err = dst.WriteString(`"}`)
n += nn
return
}
func rwString(dst jsWriter, src *Reader) (n int, err error) {
var p []byte
p, err = src.R.Peek(1)
if err != nil {
return
}
lead := p[0]
var read int
if isfixstr(lead) {
read = int(rfixstr(lead))
src.R.Skip(1)
goto write
}
switch lead {
case mstr8:
p, err = src.R.Next(2)
if err != nil {
return
}
read = int(uint8(p[1]))
case mstr16:
p, err = src.R.Next(3)
if err != nil {
return
}
read = int(big.Uint16(p[1:]))
case mstr32:
p, err = src.R.Next(5)
if err != nil {
return
}
read = int(big.Uint32(p[1:]))
default:
err = badPrefix(StrType, lead)
return
}
write:
p, err = src.R.Next(read)
if err != nil {
return
}
n, err = rwquoted(dst, p)
return
}
func rwBytes(dst jsWriter, src *Reader) (n int, err error) {
var nn int
err = dst.WriteByte('"')
if err != nil {
return
}
n++
src.scratch, err = src.ReadBytes(src.scratch[:0])
if err != nil {
return
}
enc := base64.NewEncoder(base64.StdEncoding, dst)
nn, err = enc.Write(src.scratch)
n += nn
if err != nil {
return
}
err = enc.Close()
if err != nil {
return
}
err = dst.WriteByte('"')
if err != nil {
return
}
n++
return
}
// Below (c) The Go Authors, 2009-2014
// Subject to the BSD-style license found at http://golang.org
//
// see: encoding/json/encode.go:(*encodeState).stringbytes()
func rwquoted(dst jsWriter, s []byte) (n int, err error) {
var nn int
err = dst.WriteByte('"')
if err != nil {
return
}
n++
start := 0
for i := 0; i < len(s); {
if b := s[i]; b < utf8.RuneSelf {
if 0x20 <= b && b != '\\' && b != '"' && b != '<' && b != '>' && b != '&' {
i++
continue
}
if start < i {
nn, err = dst.Write(s[start:i])
n += nn
if err != nil {
return
}
}
switch b {
case '\\', '"':
err = dst.WriteByte('\\')
if err != nil {
return
}
n++
err = dst.WriteByte(b)
if err != nil {
return
}
n++
case '\n':
err = dst.WriteByte('\\')
if err != nil {
return
}
n++
err = dst.WriteByte('n')
if err != nil {
return
}
n++
case '\r':
err = dst.WriteByte('\\')
if err != nil {
return
}
n++
err = dst.WriteByte('r')
if err != nil {
return
}
n++
default:
nn, err = dst.WriteString(`\u00`)
n += nn
if err != nil {
return
}
err = dst.WriteByte(hex[b>>4])
if err != nil {
return
}
n++
err = dst.WriteByte(hex[b&0xF])
if err != nil {
return
}
n++
}
i++
start = i
continue
}
c, size := utf8.DecodeRune(s[i:])
if c == utf8.RuneError && size == 1 {
if start < i {
nn, err = dst.Write(s[start:i])
n += nn
if err != nil {
return
}
nn, err = dst.WriteString(`\ufffd`)
n += nn
if err != nil {
return
}
i += size
start = i
continue
}
}
if c == '\u2028' || c == '\u2029' {
if start < i {
nn, err = dst.Write(s[start:i])
n += nn
if err != nil {
return
}
nn, err = dst.WriteString(`\u202`)
n += nn
if err != nil {
return
}
err = dst.WriteByte(hex[c&0xF])
if err != nil {
return
}
n++
}
}
i += size
}
if start < len(s) {
nn, err = dst.Write(s[start:])
n += nn
if err != nil {
return
}
}
err = dst.WriteByte('"')
if err != nil {
return
}
n++
return
}
package msgp
import (
"bufio"
"encoding/base64"
"encoding/json"
"io"
"strconv"
"time"
)
var unfuns [_maxtype]func(jsWriter, []byte, []byte) ([]byte, []byte, error)
func init() {
// NOTE(pmh): this is best expressed as a jump table,
// but gc doesn't do that yet. revisit post-go1.5.
unfuns = [_maxtype]func(jsWriter, []byte, []byte) ([]byte, []byte, error){
StrType: rwStringBytes,
BinType: rwBytesBytes,
MapType: rwMapBytes,
ArrayType: rwArrayBytes,
Float64Type: rwFloat64Bytes,
Float32Type: rwFloat32Bytes,
BoolType: rwBoolBytes,
IntType: rwIntBytes,
UintType: rwUintBytes,
NilType: rwNullBytes,
ExtensionType: rwExtensionBytes,
Complex64Type: rwExtensionBytes,
Complex128Type: rwExtensionBytes,
TimeType: rwTimeBytes,
}
}
// UnmarshalAsJSON takes raw messagepack and writes
// it as JSON to 'w'. If an error is returned, the
// bytes not translated will also be returned. If
// no errors are encountered, the length of the returned
// slice will be zero.
func UnmarshalAsJSON(w io.Writer, msg []byte) ([]byte, error) {
var (
scratch []byte
cast bool
dst jsWriter
err error
)
if jsw, ok := w.(jsWriter); ok {
dst = jsw
cast = true
} else {
dst = bufio.NewWriterSize(w, 512)
}
for len(msg) > 0 && err == nil {
msg, scratch, err = writeNext(dst, msg, scratch)
}
if !cast && err == nil {
err = dst.(*bufio.Writer).Flush()
}
return msg, err
}
func writeNext(w jsWriter, msg []byte, scratch []byte) ([]byte, []byte, error) {
if len(msg) < 1 {
return msg, scratch, ErrShortBytes
}
t := getType(msg[0])
if t == InvalidType {
return msg, scratch, InvalidPrefixError(msg[0])
}
if t == ExtensionType {
et, err := peekExtension(msg)
if err != nil {
return nil, scratch, err
}
if et == TimeExtension {
t = TimeType
}
}
return unfuns[t](w, msg, scratch)
}
func rwArrayBytes(w jsWriter, msg []byte, scratch []byte) ([]byte, []byte, error) {
sz, msg, err := ReadArrayHeaderBytes(msg)
if err != nil {
return msg, scratch, err
}
err = w.WriteByte('[')
if err != nil {
return msg, scratch, err
}
for i := uint32(0); i < sz; i++ {
if i != 0 {
err = w.WriteByte(',')
if err != nil {
return msg, scratch, err
}
}
msg, scratch, err = writeNext(w, msg, scratch)
if err != nil {
return msg, scratch, err
}
}
err = w.WriteByte(']')
return msg, scratch, err
}
func rwMapBytes(w jsWriter, msg []byte, scratch []byte) ([]byte, []byte, error) {
sz, msg, err := ReadMapHeaderBytes(msg)
if err != nil {
return msg, scratch, err
}
err = w.WriteByte('{')
if err != nil {
return msg, scratch, err
}
for i := uint32(0); i < sz; i++ {
if i != 0 {
err = w.WriteByte(',')
if err != nil {
return msg, scratch, err
}
}
msg, scratch, err = rwMapKeyBytes(w, msg, scratch)
if err != nil {
return msg, scratch, err
}
err = w.WriteByte(':')
if err != nil {
return msg, scratch, err
}
msg, scratch, err = writeNext(w, msg, scratch)
if err != nil {
return msg, scratch, err
}
}
err = w.WriteByte('}')
return msg, scratch, err
}
func rwMapKeyBytes(w jsWriter, msg []byte, scratch []byte) ([]byte, []byte, error) {
msg, scratch, err := rwStringBytes(w, msg, scratch)
if err != nil {
if tperr, ok := err.(TypeError); ok && tperr.Encoded == BinType {
return rwBytesBytes(w, msg, scratch)
}
}
return msg, scratch, err
}
func rwStringBytes(w jsWriter, msg []byte, scratch []byte) ([]byte, []byte, error) {
str, msg, err := ReadStringZC(msg)
if err != nil {
return msg, scratch, err
}
_, err = rwquoted(w, str)
return msg, scratch, err
}
func rwBytesBytes(w jsWriter, msg []byte, scratch []byte) ([]byte, []byte, error) {
bts, msg, err := ReadBytesZC(msg)
if err != nil {
return msg, scratch, err
}
l := base64.StdEncoding.EncodedLen(len(bts))
if cap(scratch) >= l {
scratch = scratch[0:l]
} else {
scratch = make([]byte, l)
}
base64.StdEncoding.Encode(scratch, bts)
err = w.WriteByte('"')
if err != nil {
return msg, scratch, err
}
_, err = w.Write(scratch)
if err != nil {
return msg, scratch, err
}
err = w.WriteByte('"')
return msg, scratch, err
}
func rwNullBytes(w jsWriter, msg []byte, scratch []byte) ([]byte, []byte, error) {
msg, err := ReadNilBytes(msg)
if err != nil {
return msg, scratch, err
}
_, err = w.Write(null)
return msg, scratch, err
}
func rwBoolBytes(w jsWriter, msg []byte, scratch []byte) ([]byte, []byte, error) {
b, msg, err := ReadBoolBytes(msg)
if err != nil {
return msg, scratch, err
}
if b {
_, err = w.WriteString("true")
return msg, scratch, err
}
_, err = w.WriteString("false")
return msg, scratch, err
}
func rwIntBytes(w jsWriter, msg []byte, scratch []byte) ([]byte, []byte, error) {
i, msg, err := ReadInt64Bytes(msg)
if err != nil {
return msg, scratch, err
}
scratch = strconv.AppendInt(scratch[0:0], i, 10)
_, err = w.Write(scratch)
return msg, scratch, err
}
func rwUintBytes(w jsWriter, msg []byte, scratch []byte) ([]byte, []byte, error) {
u, msg, err := ReadUint64Bytes(msg)
if err != nil {
return msg, scratch, err
}
scratch = strconv.AppendUint(scratch[0:0], u, 10)
_, err = w.Write(scratch)
return msg, scratch, err
}
func rwFloatBytes(w jsWriter, msg []byte, f64 bool, scratch []byte) ([]byte, []byte, error) {
var f float64
var err error
var sz int
if f64 {
sz = 64
f, msg, err = ReadFloat64Bytes(msg)
} else {
sz = 32
var v float32
v, msg, err = ReadFloat32Bytes(msg)
f = float64(v)
}
if err != nil {
return msg, scratch, err
}
scratch = strconv.AppendFloat(scratch, f, 'f', -1, sz)
_, err = w.Write(scratch)
return msg, scratch, err
}
func rwFloat32Bytes(w jsWriter, msg []byte, scratch []byte) ([]byte, []byte, error) {
var f float32
var err error
f, msg, err = ReadFloat32Bytes(msg)
if err != nil {
return msg, scratch, err
}
scratch = strconv.AppendFloat(scratch[:0], float64(f), 'f', -1, 32)
_, err = w.Write(scratch)
return msg, scratch, err
}
func rwFloat64Bytes(w jsWriter, msg []byte, scratch []byte) ([]byte, []byte, error) {
var f float64
var err error
f, msg, err = ReadFloat64Bytes(msg)
if err != nil {
return msg, scratch, err
}
scratch = strconv.AppendFloat(scratch[:0], f, 'f', -1, 64)
_, err = w.Write(scratch)
return msg, scratch, err
}
func rwTimeBytes(w jsWriter, msg []byte, scratch []byte) ([]byte, []byte, error) {
var t time.Time
var err error
t, msg, err = ReadTimeBytes(msg)
if err != nil {
return msg, scratch, err
}
bts, err := t.MarshalJSON()
if err != nil {
return msg, scratch, err
}
_, err = w.Write(bts)
return msg, scratch, err
}
func rwExtensionBytes(w jsWriter, msg []byte, scratch []byte) ([]byte, []byte, error) {
var err error
var et int8
et, err = peekExtension(msg)
if err != nil {
return msg, scratch, err
}
// if it's time.Time
if et == TimeExtension {
var tm time.Time
tm, msg, err = ReadTimeBytes(msg)
if err != nil {
return msg, scratch, err
}
bts, err := tm.MarshalJSON()
if err != nil {
return msg, scratch, err
}
_, err = w.Write(bts)
return msg, scratch, err
}
// if the extension is registered,
// use its canonical JSON form
if f, ok := extensionReg[et]; ok {
e := f()
msg, err = ReadExtensionBytes(msg, e)
if err != nil {
return msg, scratch, err
}
bts, err := json.Marshal(e)
if err != nil {
return msg, scratch, err
}
_, err = w.Write(bts)
return msg, scratch, err
}
// otherwise, write `{"type": <num>, "data": "<base64data>"}`
r := RawExtension{}
r.Type = et
msg, err = ReadExtensionBytes(msg, &r)
if err != nil {
return msg, scratch, err
}
scratch, err = writeExt(w, r, scratch)
return msg, scratch, err
}
func writeExt(w jsWriter, r RawExtension, scratch []byte) ([]byte, error) {
_, err := w.WriteString(`{"type":`)
if err != nil {
return scratch, err
}
scratch = strconv.AppendInt(scratch[0:0], int64(r.Type), 10)
_, err = w.Write(scratch)
if err != nil {
return scratch, err
}
_, err = w.WriteString(`,"data":"`)
if err != nil {
return scratch, err
}
l := base64.StdEncoding.EncodedLen(len(r.Data))
if cap(scratch) >= l {
scratch = scratch[0:l]
} else {
scratch = make([]byte, l)
}
base64.StdEncoding.Encode(scratch, r.Data)
_, err = w.Write(scratch)
if err != nil {
return scratch, err
}
_, err = w.WriteString(`"}`)
return scratch, err
}
package msgp
import (
"math"
"strconv"
)
// The portable parts of the Number implementation
// Number can be
// an int64, uint64, float32,
// or float64 internally.
// It can decode itself
// from any of the native
// messagepack number types.
// The zero-value of Number
// is Int(0). Using the equality
// operator with Number compares
// both the type and the value
// of the number.
type Number struct {
// internally, this
// is just a tagged union.
// the raw bits of the number
// are stored the same way regardless.
bits uint64
typ Type
}
// AsInt sets the number to an int64.
func (n *Number) AsInt(i int64) {
// we always store int(0)
// as {0, InvalidType} in
// order to preserve
// the behavior of the == operator
if i == 0 {
n.typ = InvalidType
n.bits = 0
return
}
n.typ = IntType
n.bits = uint64(i)
}
// AsUint sets the number to a uint64.
func (n *Number) AsUint(u uint64) {
n.typ = UintType
n.bits = u
}
// AsFloat32 sets the value of the number
// to a float32.
func (n *Number) AsFloat32(f float32) {
n.typ = Float32Type
n.bits = uint64(math.Float32bits(f))
}
// AsFloat64 sets the value of the
// number to a float64.
func (n *Number) AsFloat64(f float64) {
n.typ = Float64Type
n.bits = math.Float64bits(f)
}
// Int casts the number as an int64, and
// returns whether or not that was the
// underlying type.
func (n *Number) Int() (int64, bool) {
return int64(n.bits), n.typ == IntType || n.typ == InvalidType
}
// Uint casts the number as a uint64, and returns
// whether or not that was the underlying type.
func (n *Number) Uint() (uint64, bool) {
return n.bits, n.typ == UintType
}
// Float casts the number to a float64, and
// returns whether or not that was the underlying
// type (either a float64 or a float32).
func (n *Number) Float() (float64, bool) {
switch n.typ {
case Float32Type:
return float64(math.Float32frombits(uint32(n.bits))), true
case Float64Type:
return math.Float64frombits(n.bits), true
default:
return 0.0, false
}
}
// Type will return one of:
// Float64Type, Float32Type, UintType, or IntType.
func (n *Number) Type() Type {
if n.typ == InvalidType {
return IntType
}
return n.typ
}
// DecodeMsg implements msgp.Decodable
func (n *Number) DecodeMsg(r *Reader) error {
typ, err := r.NextType()
if err != nil {
return err
}
switch typ {
case Float32Type:
f, err := r.ReadFloat32()
if err != nil {
return err
}
n.AsFloat32(f)
return nil
case Float64Type:
f, err := r.ReadFloat64()
if err != nil {
return err
}
n.AsFloat64(f)
return nil
case IntType:
i, err := r.ReadInt64()
if err != nil {
return err
}
n.AsInt(i)
return nil
case UintType:
u, err := r.ReadUint64()
if err != nil {
return err
}
n.AsUint(u)
return nil
default:
return TypeError{Encoded: typ, Method: IntType}
}
}
// UnmarshalMsg implements msgp.Unmarshaler
func (n *Number) UnmarshalMsg(b []byte) ([]byte, error) {
typ := NextType(b)
switch typ {
case IntType:
i, o, err := ReadInt64Bytes(b)
if err != nil {
return b, err
}
n.AsInt(i)
return o, nil
case UintType:
u, o, err := ReadUint64Bytes(b)
if err != nil {
return b, err
}
n.AsUint(u)
return o, nil
case Float64Type:
f, o, err := ReadFloat64Bytes(b)
if err != nil {
return b, err
}
n.AsFloat64(f)
return o, nil
case Float32Type:
f, o, err := ReadFloat32Bytes(b)
if err != nil {
return b, err
}
n.AsFloat32(f)
return o, nil
default:
return b, TypeError{Method: IntType, Encoded: typ}
}
}
// MarshalMsg implements msgp.Marshaler
func (n *Number) MarshalMsg(b []byte) ([]byte, error) {
switch n.typ {
case IntType:
return AppendInt64(b, int64(n.bits)), nil
case UintType:
return AppendUint64(b, uint64(n.bits)), nil
case Float64Type:
return AppendFloat64(b, math.Float64frombits(n.bits)), nil
case Float32Type:
return AppendFloat32(b, math.Float32frombits(uint32(n.bits))), nil
default:
return AppendInt64(b, 0), nil
}
}
// EncodeMsg implements msgp.Encodable
func (n *Number) EncodeMsg(w *Writer) error {
switch n.typ {
case IntType:
return w.WriteInt64(int64(n.bits))
case UintType:
return w.WriteUint64(n.bits)
case Float64Type:
return w.WriteFloat64(math.Float64frombits(n.bits))
case Float32Type:
return w.WriteFloat32(math.Float32frombits(uint32(n.bits)))
default:
return w.WriteInt64(0)
}
}
// Msgsize implements msgp.Sizer
func (n *Number) Msgsize() int {
switch n.typ {
case Float32Type:
return Float32Size
case Float64Type:
return Float64Size
case IntType:
return Int64Size
case UintType:
return Uint64Size
default:
return 1 // fixint(0)
}
}
// MarshalJSON implements json.Marshaler
func (n *Number) MarshalJSON() ([]byte, error) {
t := n.Type()
if t == InvalidType {
return []byte{'0'}, nil
}
out := make([]byte, 0, 32)
switch t {
case Float32Type, Float64Type:
f, _ := n.Float()
return strconv.AppendFloat(out, f, 'f', -1, 64), nil
case IntType:
i, _ := n.Int()
return strconv.AppendInt(out, i, 10), nil
case UintType:
u, _ := n.Uint()
return strconv.AppendUint(out, u, 10), nil
default:
panic("(*Number).typ is invalid")
}
}
// String implements fmt.Stringer
func (n *Number) String() string {
switch n.typ {
case InvalidType:
return "0"
case Float32Type, Float64Type:
f, _ := n.Float()
return strconv.FormatFloat(f, 'f', -1, 64)
case IntType:
i, _ := n.Int()
return strconv.FormatInt(i, 10)
case UintType:
u, _ := n.Uint()
return strconv.FormatUint(u, 10)
default:
panic("(*Number).typ is invalid")
}
}
// +build purego appengine
package msgp
// let's just assume appengine
// uses 64-bit hardware...
const smallint = false
func UnsafeString(b []byte) string {
return string(b)
}
func UnsafeBytes(s string) []byte {
return []byte(s)
}
package msgp
import (
"io"
"math"
"sync"
"time"
"github.com/philhofer/fwd"
)
// where we keep old *Readers
var readerPool = sync.Pool{New: func() interface{} { return &Reader{} }}
// Type is a MessagePack wire type,
// including this package's built-in
// extension types.
type Type byte
// MessagePack Types
//
// The zero value of Type
// is InvalidType.
const (
InvalidType Type = iota
// MessagePack built-in types
StrType
BinType
MapType
ArrayType
Float64Type
Float32Type
BoolType
IntType
UintType
NilType
ExtensionType
// pseudo-types provided
// by extensions
Complex64Type
Complex128Type
TimeType
_maxtype
)
// String implements fmt.Stringer
func (t Type) String() string {
switch t {
case StrType:
return "str"
case BinType:
return "bin"
case MapType:
return "map"
case ArrayType:
return "array"
case Float64Type:
return "float64"
case Float32Type:
return "float32"
case BoolType:
return "bool"
case UintType:
return "uint"
case IntType:
return "int"
case ExtensionType:
return "ext"
case NilType:
return "nil"
default:
return "<invalid>"
}
}
func freeR(m *Reader) {
readerPool.Put(m)
}
// Unmarshaler is the interface fulfilled
// by objects that know how to unmarshal
// themselves from MessagePack.
// UnmarshalMsg unmarshals the object
// from binary, returing any leftover
// bytes and any errors encountered.
type Unmarshaler interface {
UnmarshalMsg([]byte) ([]byte, error)
}
// Decodable is the interface fulfilled
// by objects that know how to read
// themselves from a *Reader.
type Decodable interface {
DecodeMsg(*Reader) error
}
// Decode decodes 'd' from 'r'.
func Decode(r io.Reader, d Decodable) error {
rd := NewReader(r)
err := d.DecodeMsg(rd)
freeR(rd)
return err
}
// NewReader returns a *Reader that
// reads from the provided reader. The
// reader will be buffered.
func NewReader(r io.Reader) *Reader {
p := readerPool.Get().(*Reader)
if p.R == nil {
p.R = fwd.NewReader(r)
} else {
p.R.Reset(r)
}
return p
}
// NewReaderSize returns a *Reader with a buffer of the given size.
// (This is vastly preferable to passing the decoder a reader that is already buffered.)
func NewReaderSize(r io.Reader, sz int) *Reader {
return &Reader{R: fwd.NewReaderSize(r, sz)}
}
// Reader wraps an io.Reader and provides
// methods to read MessagePack-encoded values
// from it. Readers are buffered.
type Reader struct {
// R is the buffered reader
// that the Reader uses
// to decode MessagePack.
// The Reader itself
// is stateless; all the
// buffering is done
// within R.
R *fwd.Reader
scratch []byte
}
// Read implements `io.Reader`
func (m *Reader) Read(p []byte) (int, error) {
return m.R.Read(p)
}
// CopyNext reads the next object from m without decoding it and writes it to w.
// It avoids unnecessary copies internally.
func (m *Reader) CopyNext(w io.Writer) (int64, error) {
sz, o, err := getNextSize(m.R)
if err != nil {
return 0, err
}
var n int64
// Opportunistic optimization: if we can fit the whole thing in the m.R
// buffer, then just get a pointer to that, and pass it to w.Write,
// avoiding an allocation.
if int(sz) <= m.R.BufferSize() {
var nn int
var buf []byte
buf, err = m.R.Next(int(sz))
if err != nil {
if err == io.ErrUnexpectedEOF {
err = ErrShortBytes
}
return 0, err
}
nn, err = w.Write(buf)
n += int64(nn)
} else {
// Fall back to io.CopyN.
// May avoid allocating if w is a ReaderFrom (e.g. bytes.Buffer)
n, err = io.CopyN(w, m.R, int64(sz))
if err == io.ErrUnexpectedEOF {
err = ErrShortBytes
}
}
if err != nil {
return n, err
} else if n < int64(sz) {
return n, io.ErrShortWrite
}
// for maps and slices, read elements
for x := uintptr(0); x < o; x++ {
var n2 int64
n2, err = m.CopyNext(w)
if err != nil {
return n, err
}
n += n2
}
return n, nil
}
// ReadFull implements `io.ReadFull`
func (m *Reader) ReadFull(p []byte) (int, error) {
return m.R.ReadFull(p)
}
// Reset resets the underlying reader.
func (m *Reader) Reset(r io.Reader) { m.R.Reset(r) }
// Buffered returns the number of bytes currently in the read buffer.
func (m *Reader) Buffered() int { return m.R.Buffered() }
// BufferSize returns the capacity of the read buffer.
func (m *Reader) BufferSize() int { return m.R.BufferSize() }
// NextType returns the next object type to be decoded.
func (m *Reader) NextType() (Type, error) {
p, err := m.R.Peek(1)
if err != nil {
return InvalidType, err
}
t := getType(p[0])
if t == InvalidType {
return t, InvalidPrefixError(p[0])
}
if t == ExtensionType {
v, err := m.peekExtensionType()
if err != nil {
return InvalidType, err
}
switch v {
case Complex64Extension:
return Complex64Type, nil
case Complex128Extension:
return Complex128Type, nil
case TimeExtension:
return TimeType, nil
}
}
return t, nil
}
// IsNil returns whether or not
// the next byte is a null messagepack byte
func (m *Reader) IsNil() bool {
p, err := m.R.Peek(1)
return err == nil && p[0] == mnil
}
// getNextSize returns the size of the next object on the wire.
// returns (obj size, obj elements, error)
// only maps and arrays have non-zero obj elements
// for maps and arrays, obj size does not include elements
//
// use uintptr b/c it's guaranteed to be large enough
// to hold whatever we can fit in memory.
func getNextSize(r *fwd.Reader) (uintptr, uintptr, error) {
b, err := r.Peek(1)
if err != nil {
return 0, 0, err
}
lead := b[0]
spec := &sizes[lead]
size, mode := spec.size, spec.extra
if size == 0 {
return 0, 0, InvalidPrefixError(lead)
}
if mode >= 0 {
return uintptr(size), uintptr(mode), nil
}
b, err = r.Peek(int(size))
if err != nil {
return 0, 0, err
}
switch mode {
case extra8:
return uintptr(size) + uintptr(b[1]), 0, nil
case extra16:
return uintptr(size) + uintptr(big.Uint16(b[1:])), 0, nil
case extra32:
return uintptr(size) + uintptr(big.Uint32(b[1:])), 0, nil
case map16v:
return uintptr(size), 2 * uintptr(big.Uint16(b[1:])), nil
case map32v:
return uintptr(size), 2 * uintptr(big.Uint32(b[1:])), nil
case array16v:
return uintptr(size), uintptr(big.Uint16(b[1:])), nil
case array32v:
return uintptr(size), uintptr(big.Uint32(b[1:])), nil
default:
return 0, 0, fatal
}
}
// Skip skips over the next object, regardless of
// its type. If it is an array or map, the whole array
// or map will be skipped.
func (m *Reader) Skip() error {
var (
v uintptr // bytes
o uintptr // objects
err error
p []byte
)
// we can use the faster
// method if we have enough
// buffered data
if m.R.Buffered() >= 5 {
p, err = m.R.Peek(5)
if err != nil {
return err
}
v, o, err = getSize(p)
if err != nil {
return err
}
} else {
v, o, err = getNextSize(m.R)
if err != nil {
return err
}
}
// 'v' is always non-zero
// if err == nil
_, err = m.R.Skip(int(v))
if err != nil {
return err
}
// for maps and slices, skip elements
for x := uintptr(0); x < o; x++ {
err = m.Skip()
if err != nil {
return err
}
}
return nil
}
// ReadMapHeader reads the next object
// as a map header and returns the size
// of the map and the number of bytes written.
// It will return a TypeError{} if the next
// object is not a map.
func (m *Reader) ReadMapHeader() (sz uint32, err error) {
var p []byte
var lead byte
p, err = m.R.Peek(1)
if err != nil {
return
}
lead = p[0]
if isfixmap(lead) {
sz = uint32(rfixmap(lead))
_, err = m.R.Skip(1)
return
}
switch lead {
case mmap16:
p, err = m.R.Next(3)
if err != nil {
return
}
sz = uint32(big.Uint16(p[1:]))
return
case mmap32:
p, err = m.R.Next(5)
if err != nil {
return
}
sz = big.Uint32(p[1:])
return
default:
err = badPrefix(MapType, lead)
return
}
}
// ReadMapKey reads either a 'str' or 'bin' field from
// the reader and returns the value as a []byte. It uses
// scratch for storage if it is large enough.
func (m *Reader) ReadMapKey(scratch []byte) ([]byte, error) {
out, err := m.ReadStringAsBytes(scratch)
if err != nil {
if tperr, ok := err.(TypeError); ok && tperr.Encoded == BinType {
return m.ReadBytes(scratch)
}
return nil, err
}
return out, nil
}
// MapKeyPtr returns a []byte pointing to the contents
// of a valid map key. The key cannot be empty, and it
// must be shorter than the total buffer size of the
// *Reader. Additionally, the returned slice is only
// valid until the next *Reader method call. Users
// should exercise extreme care when using this
// method; writing into the returned slice may
// corrupt future reads.
func (m *Reader) ReadMapKeyPtr() ([]byte, error) {
p, err := m.R.Peek(1)
if err != nil {
return nil, err
}
lead := p[0]
var read int
if isfixstr(lead) {
read = int(rfixstr(lead))
m.R.Skip(1)
goto fill
}
switch lead {
case mstr8, mbin8:
p, err = m.R.Next(2)
if err != nil {
return nil, err
}
read = int(p[1])
case mstr16, mbin16:
p, err = m.R.Next(3)
if err != nil {
return nil, err
}
read = int(big.Uint16(p[1:]))
case mstr32, mbin32:
p, err = m.R.Next(5)
if err != nil {
return nil, err
}
read = int(big.Uint32(p[1:]))
default:
return nil, badPrefix(StrType, lead)
}
fill:
if read == 0 {
return nil, ErrShortBytes
}
return m.R.Next(read)
}
// ReadArrayHeader reads the next object as an
// array header and returns the size of the array
// and the number of bytes read.
func (m *Reader) ReadArrayHeader() (sz uint32, err error) {
var lead byte
var p []byte
p, err = m.R.Peek(1)
if err != nil {
return
}
lead = p[0]
if isfixarray(lead) {
sz = uint32(rfixarray(lead))
_, err = m.R.Skip(1)
return
}
switch lead {
case marray16:
p, err = m.R.Next(3)
if err != nil {
return
}
sz = uint32(big.Uint16(p[1:]))
return
case marray32:
p, err = m.R.Next(5)
if err != nil {
return
}
sz = big.Uint32(p[1:])
return
default:
err = badPrefix(ArrayType, lead)
return
}
}
// ReadNil reads a 'nil' MessagePack byte from the reader
func (m *Reader) ReadNil() error {
p, err := m.R.Peek(1)
if err != nil {
return err
}
if p[0] != mnil {
return badPrefix(NilType, p[0])
}
_, err = m.R.Skip(1)
return err
}
// ReadFloat64 reads a float64 from the reader.
// (If the value on the wire is encoded as a float32,
// it will be up-cast to a float64.)
func (m *Reader) ReadFloat64() (f float64, err error) {
var p []byte
p, err = m.R.Peek(9)
if err != nil {
// we'll allow a coversion from float32 to float64,
// since we don't lose any precision
if err == io.EOF && len(p) > 0 && p[0] == mfloat32 {
ef, err := m.ReadFloat32()
return float64(ef), err
}
return
}
if p[0] != mfloat64 {
// see above
if p[0] == mfloat32 {
ef, err := m.ReadFloat32()
return float64(ef), err
}
err = badPrefix(Float64Type, p[0])
return
}
f = math.Float64frombits(getMuint64(p))
_, err = m.R.Skip(9)
return
}
// ReadFloat32 reads a float32 from the reader
func (m *Reader) ReadFloat32() (f float32, err error) {
var p []byte
p, err = m.R.Peek(5)
if err != nil {
return
}
if p[0] != mfloat32 {
err = badPrefix(Float32Type, p[0])
return
}
f = math.Float32frombits(getMuint32(p))
_, err = m.R.Skip(5)
return
}
// ReadBool reads a bool from the reader
func (m *Reader) ReadBool() (b bool, err error) {
var p []byte
p, err = m.R.Peek(1)
if err != nil {
return
}
switch p[0] {
case mtrue:
b = true
case mfalse:
default:
err = badPrefix(BoolType, p[0])
return
}
_, err = m.R.Skip(1)
return
}
// ReadInt64 reads an int64 from the reader
func (m *Reader) ReadInt64() (i int64, err error) {
var p []byte
var lead byte
p, err = m.R.Peek(1)
if err != nil {
return
}
lead = p[0]
if isfixint(lead) {
i = int64(rfixint(lead))
_, err = m.R.Skip(1)
return
} else if isnfixint(lead) {
i = int64(rnfixint(lead))
_, err = m.R.Skip(1)
return
}
switch lead {
case mint8:
p, err = m.R.Next(2)
if err != nil {
return
}
i = int64(getMint8(p))
return
case muint8:
p, err = m.R.Next(2)
if err != nil {
return
}
i = int64(getMuint8(p))
return
case mint16:
p, err = m.R.Next(3)
if err != nil {
return
}
i = int64(getMint16(p))
return
case muint16:
p, err = m.R.Next(3)
if err != nil {
return
}
i = int64(getMuint16(p))
return
case mint32:
p, err = m.R.Next(5)
if err != nil {
return
}
i = int64(getMint32(p))
return
case muint32:
p, err = m.R.Next(5)
if err != nil {
return
}
i = int64(getMuint32(p))
return
case mint64:
p, err = m.R.Next(9)
if err != nil {
return
}
i = getMint64(p)
return
case muint64:
p, err = m.R.Next(9)
if err != nil {
return
}
u := getMuint64(p)
if u > math.MaxInt64 {
err = UintOverflow{Value: u, FailedBitsize: 64}
return
}
i = int64(u)
return
default:
err = badPrefix(IntType, lead)
return
}
}
// ReadInt32 reads an int32 from the reader
func (m *Reader) ReadInt32() (i int32, err error) {
var in int64
in, err = m.ReadInt64()
if in > math.MaxInt32 || in < math.MinInt32 {
err = IntOverflow{Value: in, FailedBitsize: 32}
return
}
i = int32(in)
return
}
// ReadInt16 reads an int16 from the reader
func (m *Reader) ReadInt16() (i int16, err error) {
var in int64
in, err = m.ReadInt64()
if in > math.MaxInt16 || in < math.MinInt16 {
err = IntOverflow{Value: in, FailedBitsize: 16}
return
}
i = int16(in)
return
}
// ReadInt8 reads an int8 from the reader
func (m *Reader) ReadInt8() (i int8, err error) {
var in int64
in, err = m.ReadInt64()
if in > math.MaxInt8 || in < math.MinInt8 {
err = IntOverflow{Value: in, FailedBitsize: 8}
return
}
i = int8(in)
return
}
// ReadInt reads an int from the reader
func (m *Reader) ReadInt() (i int, err error) {
if smallint {
var in int32
in, err = m.ReadInt32()
i = int(in)
return
}
var in int64
in, err = m.ReadInt64()
i = int(in)
return
}
// ReadUint64 reads a uint64 from the reader
func (m *Reader) ReadUint64() (u uint64, err error) {
var p []byte
var lead byte
p, err = m.R.Peek(1)
if err != nil {
return
}
lead = p[0]
if isfixint(lead) {
u = uint64(rfixint(lead))
_, err = m.R.Skip(1)
return
}
switch lead {
case mint8:
p, err = m.R.Next(2)
if err != nil {
return
}
v := int64(getMint8(p))
if v < 0 {
err = UintBelowZero{Value: v}
return
}
u = uint64(v)
return
case muint8:
p, err = m.R.Next(2)
if err != nil {
return
}
u = uint64(getMuint8(p))
return
case mint16:
p, err = m.R.Next(3)
if err != nil {
return
}
v := int64(getMint16(p))
if v < 0 {
err = UintBelowZero{Value: v}
return
}
u = uint64(v)
return
case muint16:
p, err = m.R.Next(3)
if err != nil {
return
}
u = uint64(getMuint16(p))
return
case mint32:
p, err = m.R.Next(5)
if err != nil {
return
}
v := int64(getMint32(p))
if v < 0 {
err = UintBelowZero{Value: v}
return
}
u = uint64(v)
return
case muint32:
p, err = m.R.Next(5)
if err != nil {
return
}
u = uint64(getMuint32(p))
return
case mint64:
p, err = m.R.Next(9)
if err != nil {
return
}
v := int64(getMint64(p))
if v < 0 {
err = UintBelowZero{Value: v}
return
}
u = uint64(v)
return
case muint64:
p, err = m.R.Next(9)
if err != nil {
return
}
u = getMuint64(p)
return
default:
if isnfixint(lead) {
err = UintBelowZero{Value: int64(rnfixint(lead))}
} else {
err = badPrefix(UintType, lead)
}
return
}
}
// ReadUint32 reads a uint32 from the reader
func (m *Reader) ReadUint32() (u uint32, err error) {
var in uint64
in, err = m.ReadUint64()
if in > math.MaxUint32 {
err = UintOverflow{Value: in, FailedBitsize: 32}
return
}
u = uint32(in)
return
}
// ReadUint16 reads a uint16 from the reader
func (m *Reader) ReadUint16() (u uint16, err error) {
var in uint64
in, err = m.ReadUint64()
if in > math.MaxUint16 {
err = UintOverflow{Value: in, FailedBitsize: 16}
return
}
u = uint16(in)
return
}
// ReadUint8 reads a uint8 from the reader
func (m *Reader) ReadUint8() (u uint8, err error) {
var in uint64
in, err = m.ReadUint64()
if in > math.MaxUint8 {
err = UintOverflow{Value: in, FailedBitsize: 8}
return
}
u = uint8(in)
return
}
// ReadUint reads a uint from the reader
func (m *Reader) ReadUint() (u uint, err error) {
if smallint {
var un uint32
un, err = m.ReadUint32()
u = uint(un)
return
}
var un uint64
un, err = m.ReadUint64()
u = uint(un)
return
}
// ReadByte is analogous to ReadUint8.
//
// NOTE: this is *not* an implementation
// of io.ByteReader.
func (m *Reader) ReadByte() (b byte, err error) {
var in uint64
in, err = m.ReadUint64()
if in > math.MaxUint8 {
err = UintOverflow{Value: in, FailedBitsize: 8}
return
}
b = byte(in)
return
}
// ReadBytes reads a MessagePack 'bin' object
// from the reader and returns its value. It may
// use 'scratch' for storage if it is non-nil.
func (m *Reader) ReadBytes(scratch []byte) (b []byte, err error) {
var p []byte
var lead byte
p, err = m.R.Peek(2)
if err != nil {
return
}
lead = p[0]
var read int64
switch lead {
case mbin8:
read = int64(p[1])
m.R.Skip(2)
case mbin16:
p, err = m.R.Next(3)
if err != nil {
return
}
read = int64(big.Uint16(p[1:]))
case mbin32:
p, err = m.R.Next(5)
if err != nil {
return
}
read = int64(big.Uint32(p[1:]))
default:
err = badPrefix(BinType, lead)
return
}
if int64(cap(scratch)) < read {
b = make([]byte, read)
} else {
b = scratch[0:read]
}
_, err = m.R.ReadFull(b)
return
}
// ReadBytesHeader reads the size header
// of a MessagePack 'bin' object. The user
// is responsible for dealing with the next
// 'sz' bytes from the reader in an application-specific
// way.
func (m *Reader) ReadBytesHeader() (sz uint32, err error) {
var p []byte
p, err = m.R.Peek(1)
if err != nil {
return
}
switch p[0] {
case mbin8:
p, err = m.R.Next(2)
if err != nil {
return
}
sz = uint32(p[1])
return
case mbin16:
p, err = m.R.Next(3)
if err != nil {
return
}
sz = uint32(big.Uint16(p[1:]))
return
case mbin32:
p, err = m.R.Next(5)
if err != nil {
return
}
sz = uint32(big.Uint32(p[1:]))
return
default:
err = badPrefix(BinType, p[0])
return
}
}
// ReadExactBytes reads a MessagePack 'bin'-encoded
// object off of the wire into the provided slice. An
// ArrayError will be returned if the object is not
// exactly the length of the input slice.
func (m *Reader) ReadExactBytes(into []byte) error {
p, err := m.R.Peek(2)
if err != nil {
return err
}
lead := p[0]
var read int64 // bytes to read
var skip int // prefix size to skip
switch lead {
case mbin8:
read = int64(p[1])
skip = 2
case mbin16:
p, err = m.R.Peek(3)
if err != nil {
return err
}
read = int64(big.Uint16(p[1:]))
skip = 3
case mbin32:
p, err = m.R.Peek(5)
if err != nil {
return err
}
read = int64(big.Uint32(p[1:]))
skip = 5
default:
return badPrefix(BinType, lead)
}
if read != int64(len(into)) {
return ArrayError{Wanted: uint32(len(into)), Got: uint32(read)}
}
m.R.Skip(skip)
_, err = m.R.ReadFull(into)
return err
}
// ReadStringAsBytes reads a MessagePack 'str' (utf-8) string
// and returns its value as bytes. It may use 'scratch' for storage
// if it is non-nil.
func (m *Reader) ReadStringAsBytes(scratch []byte) (b []byte, err error) {
var p []byte
var lead byte
p, err = m.R.Peek(1)
if err != nil {
return
}
lead = p[0]
var read int64
if isfixstr(lead) {
read = int64(rfixstr(lead))
m.R.Skip(1)
goto fill
}
switch lead {
case mstr8:
p, err = m.R.Next(2)
if err != nil {
return
}
read = int64(uint8(p[1]))
case mstr16:
p, err = m.R.Next(3)
if err != nil {
return
}
read = int64(big.Uint16(p[1:]))
case mstr32:
p, err = m.R.Next(5)
if err != nil {
return
}
read = int64(big.Uint32(p[1:]))
default:
err = badPrefix(StrType, lead)
return
}
fill:
if int64(cap(scratch)) < read {
b = make([]byte, read)
} else {
b = scratch[0:read]
}
_, err = m.R.ReadFull(b)
return
}
// ReadStringHeader reads a string header
// off of the wire. The user is then responsible
// for dealing with the next 'sz' bytes from
// the reader in an application-specific manner.
func (m *Reader) ReadStringHeader() (sz uint32, err error) {
var p []byte
p, err = m.R.Peek(1)
if err != nil {
return
}
lead := p[0]
if isfixstr(lead) {
sz = uint32(rfixstr(lead))
m.R.Skip(1)
return
}
switch lead {
case mstr8:
p, err = m.R.Next(2)
if err != nil {
return
}
sz = uint32(p[1])
return
case mstr16:
p, err = m.R.Next(3)
if err != nil {
return
}
sz = uint32(big.Uint16(p[1:]))
return
case mstr32:
p, err = m.R.Next(5)
if err != nil {
return
}
sz = big.Uint32(p[1:])
return
default:
err = badPrefix(StrType, lead)
return
}
}
// ReadString reads a utf-8 string from the reader
func (m *Reader) ReadString() (s string, err error) {
var p []byte
var lead byte
var read int64
p, err = m.R.Peek(1)
if err != nil {
return
}
lead = p[0]
if isfixstr(lead) {
read = int64(rfixstr(lead))
m.R.Skip(1)
goto fill
}
switch lead {
case mstr8:
p, err = m.R.Next(2)
if err != nil {
return
}
read = int64(uint8(p[1]))
case mstr16:
p, err = m.R.Next(3)
if err != nil {
return
}
read = int64(big.Uint16(p[1:]))
case mstr32:
p, err = m.R.Next(5)
if err != nil {
return
}
read = int64(big.Uint32(p[1:]))
default:
err = badPrefix(StrType, lead)
return
}
fill:
if read == 0 {
s, err = "", nil
return
}
// reading into the memory
// that will become the string
// itself has vastly superior
// worst-case performance, because
// the reader buffer doesn't have
// to be large enough to hold the string.
// the idea here is to make it more
// difficult for someone malicious
// to cause the system to run out of
// memory by sending very large strings.
//
// NOTE: this works because the argument
// passed to (*fwd.Reader).ReadFull escapes
// to the heap; its argument may, in turn,
// be passed to the underlying reader, and
// thus escape analysis *must* conclude that
// 'out' escapes.
out := make([]byte, read)
_, err = m.R.ReadFull(out)
if err != nil {
return
}
s = UnsafeString(out)
return
}
// ReadComplex64 reads a complex64 from the reader
func (m *Reader) ReadComplex64() (f complex64, err error) {
var p []byte
p, err = m.R.Peek(10)
if err != nil {
return
}
if p[0] != mfixext8 {
err = badPrefix(Complex64Type, p[0])
return
}
if int8(p[1]) != Complex64Extension {
err = errExt(int8(p[1]), Complex64Extension)
return
}
f = complex(math.Float32frombits(big.Uint32(p[2:])),
math.Float32frombits(big.Uint32(p[6:])))
_, err = m.R.Skip(10)
return
}
// ReadComplex128 reads a complex128 from the reader
func (m *Reader) ReadComplex128() (f complex128, err error) {
var p []byte
p, err = m.R.Peek(18)
if err != nil {
return
}
if p[0] != mfixext16 {
err = badPrefix(Complex128Type, p[0])
return
}
if int8(p[1]) != Complex128Extension {
err = errExt(int8(p[1]), Complex128Extension)
return
}
f = complex(math.Float64frombits(big.Uint64(p[2:])),
math.Float64frombits(big.Uint64(p[10:])))
_, err = m.R.Skip(18)
return
}
// ReadMapStrIntf reads a MessagePack map into a map[string]interface{}.
// (You must pass a non-nil map into the function.)
func (m *Reader) ReadMapStrIntf(mp map[string]interface{}) (err error) {
var sz uint32
sz, err = m.ReadMapHeader()
if err != nil {
return
}
for key := range mp {
delete(mp, key)
}
for i := uint32(0); i < sz; i++ {
var key string
var val interface{}
key, err = m.ReadString()
if err != nil {
return
}
val, err = m.ReadIntf()
if err != nil {
return
}
mp[key] = val
}
return
}
// ReadTime reads a time.Time object from the reader.
// The returned time's location will be set to time.Local.
func (m *Reader) ReadTime() (t time.Time, err error) {
var p []byte
p, err = m.R.Peek(15)
if err != nil {
return
}
if p[0] != mext8 || p[1] != 12 {
err = badPrefix(TimeType, p[0])
return
}
if int8(p[2]) != TimeExtension {
err = errExt(int8(p[2]), TimeExtension)
return
}
sec, nsec := getUnix(p[3:])
t = time.Unix(sec, int64(nsec)).Local()
_, err = m.R.Skip(15)
return
}
// ReadIntf reads out the next object as a raw interface{}.
// Arrays are decoded as []interface{}, and maps are decoded
// as map[string]interface{}. Integers are decoded as int64
// and unsigned integers are decoded as uint64.
func (m *Reader) ReadIntf() (i interface{}, err error) {
var t Type
t, err = m.NextType()
if err != nil {
return
}
switch t {
case BoolType:
i, err = m.ReadBool()
return
case IntType:
i, err = m.ReadInt64()
return
case UintType:
i, err = m.ReadUint64()
return
case BinType:
i, err = m.ReadBytes(nil)
return
case StrType:
i, err = m.ReadString()
return
case Complex64Type:
i, err = m.ReadComplex64()
return
case Complex128Type:
i, err = m.ReadComplex128()
return
case TimeType:
i, err = m.ReadTime()
return
case ExtensionType:
var t int8
t, err = m.peekExtensionType()
if err != nil {
return
}
f, ok := extensionReg[t]
if ok {
e := f()
err = m.ReadExtension(e)
i = e
return
}
var e RawExtension
e.Type = t
err = m.ReadExtension(&e)
i = &e
return
case MapType:
mp := make(map[string]interface{})
err = m.ReadMapStrIntf(mp)
i = mp
return
case NilType:
err = m.ReadNil()
i = nil
return
case Float32Type:
i, err = m.ReadFloat32()
return
case Float64Type:
i, err = m.ReadFloat64()
return
case ArrayType:
var sz uint32
sz, err = m.ReadArrayHeader()
if err != nil {
return
}
out := make([]interface{}, int(sz))
for j := range out {
out[j], err = m.ReadIntf()
if err != nil {
return
}
}
i = out
return
default:
return nil, fatal // unreachable
}
}
package msgp
import (
"bytes"
"encoding/binary"
"math"
"time"
)
var big = binary.BigEndian
// NextType returns the type of the next
// object in the slice. If the length
// of the input is zero, it returns
// InvalidType.
func NextType(b []byte) Type {
if len(b) == 0 {
return InvalidType
}
spec := sizes[b[0]]
t := spec.typ
if t == ExtensionType && len(b) > int(spec.size) {
var tp int8
if spec.extra == constsize {
tp = int8(b[1])
} else {
tp = int8(b[spec.size-1])
}
switch tp {
case TimeExtension:
return TimeType
case Complex128Extension:
return Complex128Type
case Complex64Extension:
return Complex64Type
default:
return ExtensionType
}
}
return t
}
// IsNil returns true if len(b)>0 and
// the leading byte is a 'nil' MessagePack
// byte; false otherwise
func IsNil(b []byte) bool {
if len(b) != 0 && b[0] == mnil {
return true
}
return false
}
// Raw is raw MessagePack.
// Raw allows you to read and write
// data without interpreting its contents.
type Raw []byte
// MarshalMsg implements msgp.Marshaler.
// It appends the raw contents of 'raw'
// to the provided byte slice. If 'raw'
// is 0 bytes, 'nil' will be appended instead.
func (r Raw) MarshalMsg(b []byte) ([]byte, error) {
i := len(r)
if i == 0 {
return AppendNil(b), nil
}
o, l := ensure(b, i)
copy(o[l:], []byte(r))
return o, nil
}
// UnmarshalMsg implements msgp.Unmarshaler.
// It sets the contents of *Raw to be the next
// object in the provided byte slice.
func (r *Raw) UnmarshalMsg(b []byte) ([]byte, error) {
l := len(b)
out, err := Skip(b)
if err != nil {
return b, err
}
rlen := l - len(out)
if IsNil(b[:rlen]) {
rlen = 0
}
if cap(*r) < rlen {
*r = make(Raw, rlen)
} else {
*r = (*r)[0:rlen]
}
copy(*r, b[:rlen])
return out, nil
}
// EncodeMsg implements msgp.Encodable.
// It writes the raw bytes to the writer.
// If r is empty, it writes 'nil' instead.
func (r Raw) EncodeMsg(w *Writer) error {
if len(r) == 0 {
return w.WriteNil()
}
_, err := w.Write([]byte(r))
return err
}
// DecodeMsg implements msgp.Decodable.
// It sets the value of *Raw to be the
// next object on the wire.
func (r *Raw) DecodeMsg(f *Reader) error {
*r = (*r)[:0]
err := appendNext(f, (*[]byte)(r))
if IsNil(*r) {
*r = (*r)[:0]
}
return err
}
// Msgsize implements msgp.Sizer
func (r Raw) Msgsize() int {
l := len(r)
if l == 0 {
return 1 // for 'nil'
}
return l
}
func appendNext(f *Reader, d *[]byte) error {
amt, o, err := getNextSize(f.R)
if err != nil {
return err
}
var i int
*d, i = ensure(*d, int(amt))
_, err = f.R.ReadFull((*d)[i:])
if err != nil {
return err
}
for o > 0 {
err = appendNext(f, d)
if err != nil {
return err
}
o--
}
return nil
}
// MarshalJSON implements json.Marshaler
func (r *Raw) MarshalJSON() ([]byte, error) {
var buf bytes.Buffer
_, err := UnmarshalAsJSON(&buf, []byte(*r))
return buf.Bytes(), err
}
// ReadMapHeaderBytes reads a map header size
// from 'b' and returns the remaining bytes.
// Possible errors:
// - ErrShortBytes (too few bytes)
// - TypeError{} (not a map)
func ReadMapHeaderBytes(b []byte) (sz uint32, o []byte, err error) {
l := len(b)
if l < 1 {
err = ErrShortBytes
return
}
lead := b[0]
if isfixmap(lead) {
sz = uint32(rfixmap(lead))
o = b[1:]
return
}
switch lead {
case mmap16:
if l < 3 {
err = ErrShortBytes
return
}
sz = uint32(big.Uint16(b[1:]))
o = b[3:]
return
case mmap32:
if l < 5 {
err = ErrShortBytes
return
}
sz = big.Uint32(b[1:])
o = b[5:]
return
default:
err = badPrefix(MapType, lead)
return
}
}
// ReadMapKeyZC attempts to read a map key
// from 'b' and returns the key bytes and the remaining bytes
// Possible errors:
// - ErrShortBytes (too few bytes)
// - TypeError{} (not a str or bin)
func ReadMapKeyZC(b []byte) ([]byte, []byte, error) {
o, x, err := ReadStringZC(b)
if err != nil {
if tperr, ok := err.(TypeError); ok && tperr.Encoded == BinType {
return ReadBytesZC(b)
}
return nil, b, err
}
return o, x, nil
}
// ReadArrayHeaderBytes attempts to read
// the array header size off of 'b' and return
// the size and remaining bytes.
// Possible errors:
// - ErrShortBytes (too few bytes)
// - TypeError{} (not an array)
func ReadArrayHeaderBytes(b []byte) (sz uint32, o []byte, err error) {
if len(b) < 1 {
return 0, nil, ErrShortBytes
}
lead := b[0]
if isfixarray(lead) {
sz = uint32(rfixarray(lead))
o = b[1:]
return
}
switch lead {
case marray16:
if len(b) < 3 {
err = ErrShortBytes
return
}
sz = uint32(big.Uint16(b[1:]))
o = b[3:]
return
case marray32:
if len(b) < 5 {
err = ErrShortBytes
return
}
sz = big.Uint32(b[1:])
o = b[5:]
return
default:
err = badPrefix(ArrayType, lead)
return
}
}
// ReadNilBytes tries to read a "nil" byte
// off of 'b' and return the remaining bytes.
// Possible errors:
// - ErrShortBytes (too few bytes)
// - TypeError{} (not a 'nil')
// - InvalidPrefixError
func ReadNilBytes(b []byte) ([]byte, error) {
if len(b) < 1 {
return nil, ErrShortBytes
}
if b[0] != mnil {
return b, badPrefix(NilType, b[0])
}
return b[1:], nil
}
// ReadFloat64Bytes tries to read a float64
// from 'b' and return the value and the remaining bytes.
// Possible errors:
// - ErrShortBytes (too few bytes)
// - TypeError{} (not a float64)
func ReadFloat64Bytes(b []byte) (f float64, o []byte, err error) {
if len(b) < 9 {
if len(b) >= 5 && b[0] == mfloat32 {
var tf float32
tf, o, err = ReadFloat32Bytes(b)
f = float64(tf)
return
}
err = ErrShortBytes
return
}
if b[0] != mfloat64 {
if b[0] == mfloat32 {
var tf float32
tf, o, err = ReadFloat32Bytes(b)
f = float64(tf)
return
}
err = badPrefix(Float64Type, b[0])
return
}
f = math.Float64frombits(getMuint64(b))
o = b[9:]
return
}
// ReadFloat32Bytes tries to read a float64
// from 'b' and return the value and the remaining bytes.
// Possible errors:
// - ErrShortBytes (too few bytes)
// - TypeError{} (not a float32)
func ReadFloat32Bytes(b []byte) (f float32, o []byte, err error) {
if len(b) < 5 {
err = ErrShortBytes
return
}
if b[0] != mfloat32 {
err = TypeError{Method: Float32Type, Encoded: getType(b[0])}
return
}
f = math.Float32frombits(getMuint32(b))
o = b[5:]
return
}
// ReadBoolBytes tries to read a float64
// from 'b' and return the value and the remaining bytes.
// Possible errors:
// - ErrShortBytes (too few bytes)
// - TypeError{} (not a bool)
func ReadBoolBytes(b []byte) (bool, []byte, error) {
if len(b) < 1 {
return false, b, ErrShortBytes
}
switch b[0] {
case mtrue:
return true, b[1:], nil
case mfalse:
return false, b[1:], nil
default:
return false, b, badPrefix(BoolType, b[0])
}
}
// ReadInt64Bytes tries to read an int64
// from 'b' and return the value and the remaining bytes.
// Possible errors:
// - ErrShortBytes (too few bytes)
// - TypeError (not a int)
func ReadInt64Bytes(b []byte) (i int64, o []byte, err error) {
l := len(b)
if l < 1 {
return 0, nil, ErrShortBytes
}
lead := b[0]
if isfixint(lead) {
i = int64(rfixint(lead))
o = b[1:]
return
}
if isnfixint(lead) {
i = int64(rnfixint(lead))
o = b[1:]
return
}
switch lead {
case mint8:
if l < 2 {
err = ErrShortBytes
return
}
i = int64(getMint8(b))
o = b[2:]
return
case muint8:
if l < 2 {
err = ErrShortBytes
return
}
i = int64(getMuint8(b))
o = b[2:]
return
case mint16:
if l < 3 {
err = ErrShortBytes
return
}
i = int64(getMint16(b))
o = b[3:]
return
case muint16:
if l < 3 {
err = ErrShortBytes
return
}
i = int64(getMuint16(b))
o = b[3:]
return
case mint32:
if l < 5 {
err = ErrShortBytes
return
}
i = int64(getMint32(b))
o = b[5:]
return
case muint32:
if l < 5 {
err = ErrShortBytes
return
}
i = int64(getMuint32(b))
o = b[5:]
return
case mint64:
if l < 9 {
err = ErrShortBytes
return
}
i = int64(getMint64(b))
o = b[9:]
return
case muint64:
if l < 9 {
err = ErrShortBytes
return
}
u := getMuint64(b)
if u > math.MaxInt64 {
err = UintOverflow{Value: u, FailedBitsize: 64}
return
}
i = int64(u)
o = b[9:]
return
default:
err = badPrefix(IntType, lead)
return
}
}
// ReadInt32Bytes tries to read an int32
// from 'b' and return the value and the remaining bytes.
// Possible errors:
// - ErrShortBytes (too few bytes)
// - TypeError{} (not a int)
// - IntOverflow{} (value doesn't fit in int32)
func ReadInt32Bytes(b []byte) (int32, []byte, error) {
i, o, err := ReadInt64Bytes(b)
if i > math.MaxInt32 || i < math.MinInt32 {
return 0, o, IntOverflow{Value: i, FailedBitsize: 32}
}
return int32(i), o, err
}
// ReadInt16Bytes tries to read an int16
// from 'b' and return the value and the remaining bytes.
// Possible errors:
// - ErrShortBytes (too few bytes)
// - TypeError{} (not a int)
// - IntOverflow{} (value doesn't fit in int16)
func ReadInt16Bytes(b []byte) (int16, []byte, error) {
i, o, err := ReadInt64Bytes(b)
if i > math.MaxInt16 || i < math.MinInt16 {
return 0, o, IntOverflow{Value: i, FailedBitsize: 16}
}
return int16(i), o, err
}
// ReadInt8Bytes tries to read an int16
// from 'b' and return the value and the remaining bytes.
// Possible errors:
// - ErrShortBytes (too few bytes)
// - TypeError{} (not a int)
// - IntOverflow{} (value doesn't fit in int8)
func ReadInt8Bytes(b []byte) (int8, []byte, error) {
i, o, err := ReadInt64Bytes(b)
if i > math.MaxInt8 || i < math.MinInt8 {
return 0, o, IntOverflow{Value: i, FailedBitsize: 8}
}
return int8(i), o, err
}
// ReadIntBytes tries to read an int
// from 'b' and return the value and the remaining bytes.
// Possible errors:
// - ErrShortBytes (too few bytes)
// - TypeError{} (not a int)
// - IntOverflow{} (value doesn't fit in int; 32-bit platforms only)
func ReadIntBytes(b []byte) (int, []byte, error) {
if smallint {
i, b, err := ReadInt32Bytes(b)
return int(i), b, err
}
i, b, err := ReadInt64Bytes(b)
return int(i), b, err
}
// ReadUint64Bytes tries to read a uint64
// from 'b' and return the value and the remaining bytes.
// Possible errors:
// - ErrShortBytes (too few bytes)
// - TypeError{} (not a uint)
func ReadUint64Bytes(b []byte) (u uint64, o []byte, err error) {
l := len(b)
if l < 1 {
return 0, nil, ErrShortBytes
}
lead := b[0]
if isfixint(lead) {
u = uint64(rfixint(lead))
o = b[1:]
return
}
switch lead {
case mint8:
if l < 2 {
err = ErrShortBytes
return
}
v := int64(getMint8(b))
if v < 0 {
err = UintBelowZero{Value: v}
return
}
u = uint64(v)
o = b[2:]
return
case muint8:
if l < 2 {
err = ErrShortBytes
return
}
u = uint64(getMuint8(b))
o = b[2:]
return
case mint16:
if l < 3 {
err = ErrShortBytes
return
}
v := int64(getMint16(b))
if v < 0 {
err = UintBelowZero{Value: v}
return
}
u = uint64(v)
o = b[3:]
return
case muint16:
if l < 3 {
err = ErrShortBytes
return
}
u = uint64(getMuint16(b))
o = b[3:]
return
case mint32:
if l < 5 {
err = ErrShortBytes
return
}
v := int64(getMint32(b))
if v < 0 {
err = UintBelowZero{Value: v}
return
}
u = uint64(v)
o = b[5:]
return
case muint32:
if l < 5 {
err = ErrShortBytes
return
}
u = uint64(getMuint32(b))
o = b[5:]
return
case mint64:
if l < 9 {
err = ErrShortBytes
return
}
v := int64(getMint64(b))
if v < 0 {
err = UintBelowZero{Value: v}
return
}
u = uint64(v)
o = b[9:]
return
case muint64:
if l < 9 {
err = ErrShortBytes
return
}
u = getMuint64(b)
o = b[9:]
return
default:
if isnfixint(lead) {
err = UintBelowZero{Value: int64(rnfixint(lead))}
} else {
err = badPrefix(UintType, lead)
}
return
}
}
// ReadUint32Bytes tries to read a uint32
// from 'b' and return the value and the remaining bytes.
// Possible errors:
// - ErrShortBytes (too few bytes)
// - TypeError{} (not a uint)
// - UintOverflow{} (value too large for uint32)
func ReadUint32Bytes(b []byte) (uint32, []byte, error) {
v, o, err := ReadUint64Bytes(b)
if v > math.MaxUint32 {
return 0, nil, UintOverflow{Value: v, FailedBitsize: 32}
}
return uint32(v), o, err
}
// ReadUint16Bytes tries to read a uint16
// from 'b' and return the value and the remaining bytes.
// Possible errors:
// - ErrShortBytes (too few bytes)
// - TypeError{} (not a uint)
// - UintOverflow{} (value too large for uint16)
func ReadUint16Bytes(b []byte) (uint16, []byte, error) {
v, o, err := ReadUint64Bytes(b)
if v > math.MaxUint16 {
return 0, nil, UintOverflow{Value: v, FailedBitsize: 16}
}
return uint16(v), o, err
}
// ReadUint8Bytes tries to read a uint8
// from 'b' and return the value and the remaining bytes.
// Possible errors:
// - ErrShortBytes (too few bytes)
// - TypeError{} (not a uint)
// - UintOverflow{} (value too large for uint8)
func ReadUint8Bytes(b []byte) (uint8, []byte, error) {
v, o, err := ReadUint64Bytes(b)
if v > math.MaxUint8 {
return 0, nil, UintOverflow{Value: v, FailedBitsize: 8}
}
return uint8(v), o, err
}
// ReadUintBytes tries to read a uint
// from 'b' and return the value and the remaining bytes.
// Possible errors:
// - ErrShortBytes (too few bytes)
// - TypeError{} (not a uint)
// - UintOverflow{} (value too large for uint; 32-bit platforms only)
func ReadUintBytes(b []byte) (uint, []byte, error) {
if smallint {
u, b, err := ReadUint32Bytes(b)
return uint(u), b, err
}
u, b, err := ReadUint64Bytes(b)
return uint(u), b, err
}
// ReadByteBytes is analogous to ReadUint8Bytes
func ReadByteBytes(b []byte) (byte, []byte, error) {
return ReadUint8Bytes(b)
}
// ReadBytesBytes reads a 'bin' object
// from 'b' and returns its vaue and
// the remaining bytes in 'b'.
// Possible errors:
// - ErrShortBytes (too few bytes)
// - TypeError{} (not a 'bin' object)
func ReadBytesBytes(b []byte, scratch []byte) (v []byte, o []byte, err error) {
return readBytesBytes(b, scratch, false)
}
func readBytesBytes(b []byte, scratch []byte, zc bool) (v []byte, o []byte, err error) {
l := len(b)
if l < 1 {
return nil, nil, ErrShortBytes
}
lead := b[0]
var read int
switch lead {
case mbin8:
if l < 2 {
err = ErrShortBytes
return
}
read = int(b[1])
b = b[2:]
case mbin16:
if l < 3 {
err = ErrShortBytes
return
}
read = int(big.Uint16(b[1:]))
b = b[3:]
case mbin32:
if l < 5 {
err = ErrShortBytes
return
}
read = int(big.Uint32(b[1:]))
b = b[5:]
default:
err = badPrefix(BinType, lead)
return
}
if len(b) < read {
err = ErrShortBytes
return
}
// zero-copy
if zc {
v = b[0:read]
o = b[read:]
return
}
if cap(scratch) >= read {
v = scratch[0:read]
} else {
v = make([]byte, read)
}
o = b[copy(v, b):]
return
}
// ReadBytesZC extracts the messagepack-encoded
// binary field without copying. The returned []byte
// points to the same memory as the input slice.
// Possible errors:
// - ErrShortBytes (b not long enough)
// - TypeError{} (object not 'bin')
func ReadBytesZC(b []byte) (v []byte, o []byte, err error) {
return readBytesBytes(b, nil, true)
}
func ReadExactBytes(b []byte, into []byte) (o []byte, err error) {
l := len(b)
if l < 1 {
err = ErrShortBytes
return
}
lead := b[0]
var read uint32
var skip int
switch lead {
case mbin8:
if l < 2 {
err = ErrShortBytes
return
}
read = uint32(b[1])
skip = 2
case mbin16:
if l < 3 {
err = ErrShortBytes
return
}
read = uint32(big.Uint16(b[1:]))
skip = 3
case mbin32:
if l < 5 {
err = ErrShortBytes
return
}
read = uint32(big.Uint32(b[1:]))
skip = 5
default:
err = badPrefix(BinType, lead)
return
}
if read != uint32(len(into)) {
err = ArrayError{Wanted: uint32(len(into)), Got: read}
return
}
o = b[skip+copy(into, b[skip:]):]
return
}
// ReadStringZC reads a messagepack string field
// without copying. The returned []byte points
// to the same memory as the input slice.
// Possible errors:
// - ErrShortBytes (b not long enough)
// - TypeError{} (object not 'str')
func ReadStringZC(b []byte) (v []byte, o []byte, err error) {
l := len(b)
if l < 1 {
return nil, nil, ErrShortBytes
}
lead := b[0]
var read int
if isfixstr(lead) {
read = int(rfixstr(lead))
b = b[1:]
} else {
switch lead {
case mstr8:
if l < 2 {
err = ErrShortBytes
return
}
read = int(b[1])
b = b[2:]
case mstr16:
if l < 3 {
err = ErrShortBytes
return
}
read = int(big.Uint16(b[1:]))
b = b[3:]
case mstr32:
if l < 5 {
err = ErrShortBytes
return
}
read = int(big.Uint32(b[1:]))
b = b[5:]
default:
err = TypeError{Method: StrType, Encoded: getType(lead)}
return
}
}
if len(b) < read {
err = ErrShortBytes
return
}
v = b[0:read]
o = b[read:]
return
}
// ReadStringBytes reads a 'str' object
// from 'b' and returns its value and the
// remaining bytes in 'b'.
// Possible errors:
// - ErrShortBytes (b not long enough)
// - TypeError{} (not 'str' type)
// - InvalidPrefixError
func ReadStringBytes(b []byte) (string, []byte, error) {
v, o, err := ReadStringZC(b)
return string(v), o, err
}
// ReadStringAsBytes reads a 'str' object
// into a slice of bytes. 'v' is the value of
// the 'str' object, which may reside in memory
// pointed to by 'scratch.' 'o' is the remaining bytes
// in 'b.''
// Possible errors:
// - ErrShortBytes (b not long enough)
// - TypeError{} (not 'str' type)
// - InvalidPrefixError (unknown type marker)
func ReadStringAsBytes(b []byte, scratch []byte) (v []byte, o []byte, err error) {
var tmp []byte
tmp, o, err = ReadStringZC(b)
v = append(scratch[:0], tmp...)
return
}
// ReadComplex128Bytes reads a complex128
// extension object from 'b' and returns the
// remaining bytes.
// Possible errors:
// - ErrShortBytes (not enough bytes in 'b')
// - TypeError{} (object not a complex128)
// - InvalidPrefixError
// - ExtensionTypeError{} (object an extension of the correct size, but not a complex128)
func ReadComplex128Bytes(b []byte) (c complex128, o []byte, err error) {
if len(b) < 18 {
err = ErrShortBytes
return
}
if b[0] != mfixext16 {
err = badPrefix(Complex128Type, b[0])
return
}
if int8(b[1]) != Complex128Extension {
err = errExt(int8(b[1]), Complex128Extension)
return
}
c = complex(math.Float64frombits(big.Uint64(b[2:])),
math.Float64frombits(big.Uint64(b[10:])))
o = b[18:]
return
}
// ReadComplex64Bytes reads a complex64
// extension object from 'b' and returns the
// remaining bytes.
// Possible errors:
// - ErrShortBytes (not enough bytes in 'b')
// - TypeError{} (object not a complex64)
// - ExtensionTypeError{} (object an extension of the correct size, but not a complex64)
func ReadComplex64Bytes(b []byte) (c complex64, o []byte, err error) {
if len(b) < 10 {
err = ErrShortBytes
return
}
if b[0] != mfixext8 {
err = badPrefix(Complex64Type, b[0])
return
}
if b[1] != Complex64Extension {
err = errExt(int8(b[1]), Complex64Extension)
return
}
c = complex(math.Float32frombits(big.Uint32(b[2:])),
math.Float32frombits(big.Uint32(b[6:])))
o = b[10:]
return
}
// ReadTimeBytes reads a time.Time
// extension object from 'b' and returns the
// remaining bytes.
// Possible errors:
// - ErrShortBytes (not enough bytes in 'b')
// - TypeError{} (object not a complex64)
// - ExtensionTypeError{} (object an extension of the correct size, but not a time.Time)
func ReadTimeBytes(b []byte) (t time.Time, o []byte, err error) {
if len(b) < 15 {
err = ErrShortBytes
return
}
if b[0] != mext8 || b[1] != 12 {
err = badPrefix(TimeType, b[0])
return
}
if int8(b[2]) != TimeExtension {
err = errExt(int8(b[2]), TimeExtension)
return
}
sec, nsec := getUnix(b[3:])
t = time.Unix(sec, int64(nsec)).Local()
o = b[15:]
return
}
// ReadMapStrIntfBytes reads a map[string]interface{}
// out of 'b' and returns the map and remaining bytes.
// If 'old' is non-nil, the values will be read into that map.
func ReadMapStrIntfBytes(b []byte, old map[string]interface{}) (v map[string]interface{}, o []byte, err error) {
var sz uint32
o = b
sz, o, err = ReadMapHeaderBytes(o)
if err != nil {
return
}
if old != nil {
for key := range old {
delete(old, key)
}
v = old
} else {
v = make(map[string]interface{}, int(sz))
}
for z := uint32(0); z < sz; z++ {
if len(o) < 1 {
err = ErrShortBytes
return
}
var key []byte
key, o, err = ReadMapKeyZC(o)
if err != nil {
return
}
var val interface{}
val, o, err = ReadIntfBytes(o)
if err != nil {
return
}
v[string(key)] = val
}
return
}
// ReadIntfBytes attempts to read
// the next object out of 'b' as a raw interface{} and
// return the remaining bytes.
func ReadIntfBytes(b []byte) (i interface{}, o []byte, err error) {
if len(b) < 1 {
err = ErrShortBytes
return
}
k := NextType(b)
switch k {
case MapType:
i, o, err = ReadMapStrIntfBytes(b, nil)
return
case ArrayType:
var sz uint32
sz, o, err = ReadArrayHeaderBytes(b)
if err != nil {
return
}
j := make([]interface{}, int(sz))
i = j
for d := range j {
j[d], o, err = ReadIntfBytes(o)
if err != nil {
return
}
}
return
case Float32Type:
i, o, err = ReadFloat32Bytes(b)
return
case Float64Type:
i, o, err = ReadFloat64Bytes(b)
return
case IntType:
i, o, err = ReadInt64Bytes(b)
return
case UintType:
i, o, err = ReadUint64Bytes(b)
return
case BoolType:
i, o, err = ReadBoolBytes(b)
return
case TimeType:
i, o, err = ReadTimeBytes(b)
return
case Complex64Type:
i, o, err = ReadComplex64Bytes(b)
return
case Complex128Type:
i, o, err = ReadComplex128Bytes(b)
return
case ExtensionType:
var t int8
t, err = peekExtension(b)
if err != nil {
return
}
// use a user-defined extension,
// if it's been registered
f, ok := extensionReg[t]
if ok {
e := f()
o, err = ReadExtensionBytes(b, e)
i = e
return
}
// last resort is a raw extension
e := RawExtension{}
e.Type = int8(t)
o, err = ReadExtensionBytes(b, &e)
i = &e
return
case NilType:
o, err = ReadNilBytes(b)
return
case BinType:
i, o, err = ReadBytesBytes(b, nil)
return
case StrType:
i, o, err = ReadStringBytes(b)
return
default:
err = InvalidPrefixError(b[0])
return
}
}
// Skip skips the next object in 'b' and
// returns the remaining bytes. If the object
// is a map or array, all of its elements
// will be skipped.
// Possible Errors:
// - ErrShortBytes (not enough bytes in b)
// - InvalidPrefixError (bad encoding)
func Skip(b []byte) ([]byte, error) {
sz, asz, err := getSize(b)
if err != nil {
return b, err
}
if uintptr(len(b)) < sz {
return b, ErrShortBytes
}
b = b[sz:]
for asz > 0 {
b, err = Skip(b)
if err != nil {
return b, err
}
asz--
}
return b, nil
}
// returns (skip N bytes, skip M objects, error)
func getSize(b []byte) (uintptr, uintptr, error) {
l := len(b)
if l == 0 {
return 0, 0, ErrShortBytes
}
lead := b[0]
spec := &sizes[lead] // get type information
size, mode := spec.size, spec.extra
if size == 0 {
return 0, 0, InvalidPrefixError(lead)
}
if mode >= 0 { // fixed composites
return uintptr(size), uintptr(mode), nil
}
if l < int(size) {
return 0, 0, ErrShortBytes
}
switch mode {
case extra8:
return uintptr(size) + uintptr(b[1]), 0, nil
case extra16:
return uintptr(size) + uintptr(big.Uint16(b[1:])), 0, nil
case extra32:
return uintptr(size) + uintptr(big.Uint32(b[1:])), 0, nil
case map16v:
return uintptr(size), 2 * uintptr(big.Uint16(b[1:])), nil
case map32v:
return uintptr(size), 2 * uintptr(big.Uint32(b[1:])), nil
case array16v:
return uintptr(size), uintptr(big.Uint16(b[1:])), nil
case array32v:
return uintptr(size), uintptr(big.Uint32(b[1:])), nil
default:
return 0, 0, fatal
}
}
package msgp
// The sizes provided
// are the worst-case
// encoded sizes for
// each type. For variable-
// length types ([]byte, string),
// the total encoded size is
// the prefix size plus the
// length of the object.
const (
Int64Size = 9
IntSize = Int64Size
UintSize = Int64Size
Int8Size = 2
Int16Size = 3
Int32Size = 5
Uint8Size = 2
ByteSize = Uint8Size
Uint16Size = 3
Uint32Size = 5
Uint64Size = Int64Size
Float64Size = 9
Float32Size = 5
Complex64Size = 10
Complex128Size = 18
TimeSize = 15
BoolSize = 1
NilSize = 1
MapHeaderSize = 5
ArrayHeaderSize = 5
BytesPrefixSize = 5
StringPrefixSize = 5
ExtensionPrefixSize = 6
)
// +build !purego,!appengine
package msgp
import (
"reflect"
"unsafe"
)
// NOTE:
// all of the definition in this file
// should be repeated in appengine.go,
// but without using unsafe
const (
// spec says int and uint are always
// the same size, but that int/uint
// size may not be machine word size
smallint = unsafe.Sizeof(int(0)) == 4
)
// UnsafeString returns the byte slice as a volatile string
// THIS SHOULD ONLY BE USED BY THE CODE GENERATOR.
// THIS IS EVIL CODE.
// YOU HAVE BEEN WARNED.
func UnsafeString(b []byte) string {
sh := (*reflect.SliceHeader)(unsafe.Pointer(&b))
return *(*string)(unsafe.Pointer(&reflect.StringHeader{Data: sh.Data, Len: sh.Len}))
}
// UnsafeBytes returns the string as a byte slice
// THIS SHOULD ONLY BE USED BY THE CODE GENERATOR.
// THIS IS EVIL CODE.
// YOU HAVE BEEN WARNED.
func UnsafeBytes(s string) []byte {
return *(*[]byte)(unsafe.Pointer(&reflect.SliceHeader{
Len: len(s),
Cap: len(s),
Data: (*(*reflect.StringHeader)(unsafe.Pointer(&s))).Data,
}))
}
package msgp
import (
"errors"
"fmt"
"io"
"math"
"reflect"
"sync"
"time"
)
// Sizer is an interface implemented
// by types that can estimate their
// size when MessagePack encoded.
// This interface is optional, but
// encoding/marshaling implementations
// may use this as a way to pre-allocate
// memory for serialization.
type Sizer interface {
Msgsize() int
}
var (
// Nowhere is an io.Writer to nowhere
Nowhere io.Writer = nwhere{}
btsType = reflect.TypeOf(([]byte)(nil))
writerPool = sync.Pool{
New: func() interface{} {
return &Writer{buf: make([]byte, 2048)}
},
}
)
func popWriter(w io.Writer) *Writer {
wr := writerPool.Get().(*Writer)
wr.Reset(w)
return wr
}
func pushWriter(wr *Writer) {
wr.w = nil
wr.wloc = 0
writerPool.Put(wr)
}
// freeW frees a writer for use
// by other processes. It is not necessary
// to call freeW on a writer. However, maintaining
// a reference to a *Writer after calling freeW on
// it will cause undefined behavior.
func freeW(w *Writer) { pushWriter(w) }
// Require ensures that cap(old)-len(old) >= extra.
func Require(old []byte, extra int) []byte {
l := len(old)
c := cap(old)
r := l + extra
if c >= r {
return old
} else if l == 0 {
return make([]byte, 0, extra)
}
// the new size is the greater
// of double the old capacity
// and the sum of the old length
// and the number of new bytes
// necessary.
c <<= 1
if c < r {
c = r
}
n := make([]byte, l, c)
copy(n, old)
return n
}
// nowhere writer
type nwhere struct{}
func (n nwhere) Write(p []byte) (int, error) { return len(p), nil }
// Marshaler is the interface implemented
// by types that know how to marshal themselves
// as MessagePack. MarshalMsg appends the marshalled
// form of the object to the provided
// byte slice, returning the extended
// slice and any errors encountered.
type Marshaler interface {
MarshalMsg([]byte) ([]byte, error)
}
// Encodable is the interface implemented
// by types that know how to write themselves
// as MessagePack using a *msgp.Writer.
type Encodable interface {
EncodeMsg(*Writer) error
}
// Writer is a buffered writer
// that can be used to write
// MessagePack objects to an io.Writer.
// You must call *Writer.Flush() in order
// to flush all of the buffered data
// to the underlying writer.
type Writer struct {
w io.Writer
buf []byte
wloc int
}
// NewWriter returns a new *Writer.
func NewWriter(w io.Writer) *Writer {
if wr, ok := w.(*Writer); ok {
return wr
}
return popWriter(w)
}
// NewWriterSize returns a writer with a custom buffer size.
func NewWriterSize(w io.Writer, sz int) *Writer {
// we must be able to require() 18
// contiguous bytes, so that is the
// practical minimum buffer size
if sz < 18 {
sz = 18
}
return &Writer{
w: w,
buf: make([]byte, sz),
}
}
// Encode encodes an Encodable to an io.Writer.
func Encode(w io.Writer, e Encodable) error {
wr := NewWriter(w)
err := e.EncodeMsg(wr)
if err == nil {
err = wr.Flush()
}
freeW(wr)
return err
}
func (mw *Writer) flush() error {
if mw.wloc == 0 {
return nil
}
n, err := mw.w.Write(mw.buf[:mw.wloc])
if err != nil {
if n > 0 {
mw.wloc = copy(mw.buf, mw.buf[n:mw.wloc])
}
return err
}
mw.wloc = 0
return nil
}
// Flush flushes all of the buffered
// data to the underlying writer.
func (mw *Writer) Flush() error { return mw.flush() }
// Buffered returns the number bytes in the write buffer
func (mw *Writer) Buffered() int { return len(mw.buf) - mw.wloc }
func (mw *Writer) avail() int { return len(mw.buf) - mw.wloc }
func (mw *Writer) bufsize() int { return len(mw.buf) }
// NOTE: this should only be called with
// a number that is guaranteed to be less than
// len(mw.buf). typically, it is called with a constant.
//
// NOTE: this is a hot code path
func (mw *Writer) require(n int) (int, error) {
c := len(mw.buf)
wl := mw.wloc
if c-wl < n {
if err := mw.flush(); err != nil {
return 0, err
}
wl = mw.wloc
}
mw.wloc += n
return wl, nil
}
func (mw *Writer) Append(b ...byte) error {
if mw.avail() < len(b) {
err := mw.flush()
if err != nil {
return err
}
}
mw.wloc += copy(mw.buf[mw.wloc:], b)
return nil
}
// push one byte onto the buffer
//
// NOTE: this is a hot code path
func (mw *Writer) push(b byte) error {
if mw.wloc == len(mw.buf) {
if err := mw.flush(); err != nil {
return err
}
}
mw.buf[mw.wloc] = b
mw.wloc++
return nil
}
func (mw *Writer) prefix8(b byte, u uint8) error {
const need = 2
if len(mw.buf)-mw.wloc < need {
if err := mw.flush(); err != nil {
return err
}
}
prefixu8(mw.buf[mw.wloc:], b, u)
mw.wloc += need
return nil
}
func (mw *Writer) prefix16(b byte, u uint16) error {
const need = 3
if len(mw.buf)-mw.wloc < need {
if err := mw.flush(); err != nil {
return err
}
}
prefixu16(mw.buf[mw.wloc:], b, u)
mw.wloc += need
return nil
}
func (mw *Writer) prefix32(b byte, u uint32) error {
const need = 5
if len(mw.buf)-mw.wloc < need {
if err := mw.flush(); err != nil {
return err
}
}
prefixu32(mw.buf[mw.wloc:], b, u)
mw.wloc += need
return nil
}
func (mw *Writer) prefix64(b byte, u uint64) error {
const need = 9
if len(mw.buf)-mw.wloc < need {
if err := mw.flush(); err != nil {
return err
}
}
prefixu64(mw.buf[mw.wloc:], b, u)
mw.wloc += need
return nil
}
// Write implements io.Writer, and writes
// data directly to the buffer.
func (mw *Writer) Write(p []byte) (int, error) {
l := len(p)
if mw.avail() < l {
if err := mw.flush(); err != nil {
return 0, err
}
if l > len(mw.buf) {
return mw.w.Write(p)
}
}
mw.wloc += copy(mw.buf[mw.wloc:], p)
return l, nil
}
// implements io.WriteString
func (mw *Writer) writeString(s string) error {
l := len(s)
if mw.avail() < l {
if err := mw.flush(); err != nil {
return err
}
if l > len(mw.buf) {
_, err := io.WriteString(mw.w, s)
return err
}
}
mw.wloc += copy(mw.buf[mw.wloc:], s)
return nil
}
// Reset changes the underlying writer used by the Writer
func (mw *Writer) Reset(w io.Writer) {
mw.buf = mw.buf[:cap(mw.buf)]
mw.w = w
mw.wloc = 0
}
// WriteMapHeader writes a map header of the given
// size to the writer
func (mw *Writer) WriteMapHeader(sz uint32) error {
switch {
case sz <= 15:
return mw.push(wfixmap(uint8(sz)))
case sz <= math.MaxUint16:
return mw.prefix16(mmap16, uint16(sz))
default:
return mw.prefix32(mmap32, sz)
}
}
// WriteArrayHeader writes an array header of the
// given size to the writer
func (mw *Writer) WriteArrayHeader(sz uint32) error {
switch {
case sz <= 15:
return mw.push(wfixarray(uint8(sz)))
case sz <= math.MaxUint16:
return mw.prefix16(marray16, uint16(sz))
default:
return mw.prefix32(marray32, sz)
}
}
// WriteNil writes a nil byte to the buffer
func (mw *Writer) WriteNil() error {
return mw.push(mnil)
}
// WriteFloat64 writes a float64 to the writer
func (mw *Writer) WriteFloat64(f float64) error {
return mw.prefix64(mfloat64, math.Float64bits(f))
}
// WriteFloat32 writes a float32 to the writer
func (mw *Writer) WriteFloat32(f float32) error {
return mw.prefix32(mfloat32, math.Float32bits(f))
}
// WriteInt64 writes an int64 to the writer
func (mw *Writer) WriteInt64(i int64) error {
if i >= 0 {
switch {
case i <= math.MaxInt8:
return mw.push(wfixint(uint8(i)))
case i <= math.MaxInt16:
return mw.prefix16(mint16, uint16(i))
case i <= math.MaxInt32:
return mw.prefix32(mint32, uint32(i))
default:
return mw.prefix64(mint64, uint64(i))
}
}
switch {
case i >= -32:
return mw.push(wnfixint(int8(i)))
case i >= math.MinInt8:
return mw.prefix8(mint8, uint8(i))
case i >= math.MinInt16:
return mw.prefix16(mint16, uint16(i))
case i >= math.MinInt32:
return mw.prefix32(mint32, uint32(i))
default:
return mw.prefix64(mint64, uint64(i))
}
}
// WriteInt8 writes an int8 to the writer
func (mw *Writer) WriteInt8(i int8) error { return mw.WriteInt64(int64(i)) }
// WriteInt16 writes an int16 to the writer
func (mw *Writer) WriteInt16(i int16) error { return mw.WriteInt64(int64(i)) }
// WriteInt32 writes an int32 to the writer
func (mw *Writer) WriteInt32(i int32) error { return mw.WriteInt64(int64(i)) }
// WriteInt writes an int to the writer
func (mw *Writer) WriteInt(i int) error { return mw.WriteInt64(int64(i)) }
// WriteUint64 writes a uint64 to the writer
func (mw *Writer) WriteUint64(u uint64) error {
switch {
case u <= (1<<7)-1:
return mw.push(wfixint(uint8(u)))
case u <= math.MaxUint8:
return mw.prefix8(muint8, uint8(u))
case u <= math.MaxUint16:
return mw.prefix16(muint16, uint16(u))
case u <= math.MaxUint32:
return mw.prefix32(muint32, uint32(u))
default:
return mw.prefix64(muint64, u)
}
}
// WriteByte is analogous to WriteUint8
func (mw *Writer) WriteByte(u byte) error { return mw.WriteUint8(uint8(u)) }
// WriteUint8 writes a uint8 to the writer
func (mw *Writer) WriteUint8(u uint8) error { return mw.WriteUint64(uint64(u)) }
// WriteUint16 writes a uint16 to the writer
func (mw *Writer) WriteUint16(u uint16) error { return mw.WriteUint64(uint64(u)) }
// WriteUint32 writes a uint32 to the writer
func (mw *Writer) WriteUint32(u uint32) error { return mw.WriteUint64(uint64(u)) }
// WriteUint writes a uint to the writer
func (mw *Writer) WriteUint(u uint) error { return mw.WriteUint64(uint64(u)) }
// WriteBytes writes binary as 'bin' to the writer
func (mw *Writer) WriteBytes(b []byte) error {
sz := uint32(len(b))
var err error
switch {
case sz <= math.MaxUint8:
err = mw.prefix8(mbin8, uint8(sz))
case sz <= math.MaxUint16:
err = mw.prefix16(mbin16, uint16(sz))
default:
err = mw.prefix32(mbin32, sz)
}
if err != nil {
return err
}
_, err = mw.Write(b)
return err
}
// WriteBytesHeader writes just the size header
// of a MessagePack 'bin' object. The user is responsible
// for then writing 'sz' more bytes into the stream.
func (mw *Writer) WriteBytesHeader(sz uint32) error {
switch {
case sz <= math.MaxUint8:
return mw.prefix8(mbin8, uint8(sz))
case sz <= math.MaxUint16:
return mw.prefix16(mbin16, uint16(sz))
default:
return mw.prefix32(mbin32, sz)
}
}
// WriteBool writes a bool to the writer
func (mw *Writer) WriteBool(b bool) error {
if b {
return mw.push(mtrue)
}
return mw.push(mfalse)
}
// WriteString writes a messagepack string to the writer.
// (This is NOT an implementation of io.StringWriter)
func (mw *Writer) WriteString(s string) error {
sz := uint32(len(s))
var err error
switch {
case sz <= 31:
err = mw.push(wfixstr(uint8(sz)))
case sz <= math.MaxUint8:
err = mw.prefix8(mstr8, uint8(sz))
case sz <= math.MaxUint16:
err = mw.prefix16(mstr16, uint16(sz))
default:
err = mw.prefix32(mstr32, sz)
}
if err != nil {
return err
}
return mw.writeString(s)
}
// WriteStringHeader writes just the string size
// header of a MessagePack 'str' object. The user
// is responsible for writing 'sz' more valid UTF-8
// bytes to the stream.
func (mw *Writer) WriteStringHeader(sz uint32) error {
switch {
case sz <= 31:
return mw.push(wfixstr(uint8(sz)))
case sz <= math.MaxUint8:
return mw.prefix8(mstr8, uint8(sz))
case sz <= math.MaxUint16:
return mw.prefix16(mstr16, uint16(sz))
default:
return mw.prefix32(mstr32, sz)
}
}
// WriteStringFromBytes writes a 'str' object
// from a []byte.
func (mw *Writer) WriteStringFromBytes(str []byte) error {
sz := uint32(len(str))
var err error
switch {
case sz <= 31:
err = mw.push(wfixstr(uint8(sz)))
case sz <= math.MaxUint8:
err = mw.prefix8(mstr8, uint8(sz))
case sz <= math.MaxUint16:
err = mw.prefix16(mstr16, uint16(sz))
default:
err = mw.prefix32(mstr32, sz)
}
if err != nil {
return err
}
_, err = mw.Write(str)
return err
}
// WriteComplex64 writes a complex64 to the writer
func (mw *Writer) WriteComplex64(f complex64) error {
o, err := mw.require(10)
if err != nil {
return err
}
mw.buf[o] = mfixext8
mw.buf[o+1] = Complex64Extension
big.PutUint32(mw.buf[o+2:], math.Float32bits(real(f)))
big.PutUint32(mw.buf[o+6:], math.Float32bits(imag(f)))
return nil
}
// WriteComplex128 writes a complex128 to the writer
func (mw *Writer) WriteComplex128(f complex128) error {
o, err := mw.require(18)
if err != nil {
return err
}
mw.buf[o] = mfixext16
mw.buf[o+1] = Complex128Extension
big.PutUint64(mw.buf[o+2:], math.Float64bits(real(f)))
big.PutUint64(mw.buf[o+10:], math.Float64bits(imag(f)))
return nil
}
// WriteMapStrStr writes a map[string]string to the writer
func (mw *Writer) WriteMapStrStr(mp map[string]string) (err error) {
err = mw.WriteMapHeader(uint32(len(mp)))
if err != nil {
return
}
for key, val := range mp {
err = mw.WriteString(key)
if err != nil {
return
}
err = mw.WriteString(val)
if err != nil {
return
}
}
return nil
}
// WriteMapStrIntf writes a map[string]interface to the writer
func (mw *Writer) WriteMapStrIntf(mp map[string]interface{}) (err error) {
err = mw.WriteMapHeader(uint32(len(mp)))
if err != nil {
return
}
for key, val := range mp {
err = mw.WriteString(key)
if err != nil {
return
}
err = mw.WriteIntf(val)
if err != nil {
return
}
}
return
}
// WriteTime writes a time.Time object to the wire.
//
// Time is encoded as Unix time, which means that
// location (time zone) data is removed from the object.
// The encoded object itself is 12 bytes: 8 bytes for
// a big-endian 64-bit integer denoting seconds
// elapsed since "zero" Unix time, followed by 4 bytes
// for a big-endian 32-bit signed integer denoting
// the nanosecond offset of the time. This encoding
// is intended to ease portability across languages.
// (Note that this is *not* the standard time.Time
// binary encoding, because its implementation relies
// heavily on the internal representation used by the
// time package.)
func (mw *Writer) WriteTime(t time.Time) error {
t = t.UTC()
o, err := mw.require(15)
if err != nil {
return err
}
mw.buf[o] = mext8
mw.buf[o+1] = 12
mw.buf[o+2] = TimeExtension
putUnix(mw.buf[o+3:], t.Unix(), int32(t.Nanosecond()))
return nil
}
// WriteIntf writes the concrete type of 'v'.
// WriteIntf will error if 'v' is not one of the following:
// - A bool, float, string, []byte, int, uint, or complex
// - A map of supported types (with string keys)
// - An array or slice of supported types
// - A pointer to a supported type
// - A type that satisfies the msgp.Encodable interface
// - A type that satisfies the msgp.Extension interface
func (mw *Writer) WriteIntf(v interface{}) error {
if v == nil {
return mw.WriteNil()
}
switch v := v.(type) {
// preferred interfaces
case Encodable:
return v.EncodeMsg(mw)
case Extension:
return mw.WriteExtension(v)
// concrete types
case bool:
return mw.WriteBool(v)
case float32:
return mw.WriteFloat32(v)
case float64:
return mw.WriteFloat64(v)
case complex64:
return mw.WriteComplex64(v)
case complex128:
return mw.WriteComplex128(v)
case uint8:
return mw.WriteUint8(v)
case uint16:
return mw.WriteUint16(v)
case uint32:
return mw.WriteUint32(v)
case uint64:
return mw.WriteUint64(v)
case uint:
return mw.WriteUint(v)
case int8:
return mw.WriteInt8(v)
case int16:
return mw.WriteInt16(v)
case int32:
return mw.WriteInt32(v)
case int64:
return mw.WriteInt64(v)
case int:
return mw.WriteInt(v)
case string:
return mw.WriteString(v)
case []byte:
return mw.WriteBytes(v)
case map[string]string:
return mw.WriteMapStrStr(v)
case map[string]interface{}:
return mw.WriteMapStrIntf(v)
case time.Time:
return mw.WriteTime(v)
}
val := reflect.ValueOf(v)
if !isSupported(val.Kind()) || !val.IsValid() {
return fmt.Errorf("msgp: type %s not supported", val)
}
switch val.Kind() {
case reflect.Ptr:
if val.IsNil() {
return mw.WriteNil()
}
return mw.WriteIntf(val.Elem().Interface())
case reflect.Slice:
return mw.writeSlice(val)
case reflect.Map:
return mw.writeMap(val)
}
return &ErrUnsupportedType{T: val.Type()}
}
func (mw *Writer) writeMap(v reflect.Value) (err error) {
if v.Type().Key().Kind() != reflect.String {
return errors.New("msgp: map keys must be strings")
}
ks := v.MapKeys()
err = mw.WriteMapHeader(uint32(len(ks)))
if err != nil {
return
}
for _, key := range ks {
val := v.MapIndex(key)
err = mw.WriteString(key.String())
if err != nil {
return
}
err = mw.WriteIntf(val.Interface())
if err != nil {
return
}
}
return
}
func (mw *Writer) writeSlice(v reflect.Value) (err error) {
// is []byte
if v.Type().ConvertibleTo(btsType) {
return mw.WriteBytes(v.Bytes())
}
sz := uint32(v.Len())
err = mw.WriteArrayHeader(sz)
if err != nil {
return
}
for i := uint32(0); i < sz; i++ {
err = mw.WriteIntf(v.Index(int(i)).Interface())
if err != nil {
return
}
}
return
}
func (mw *Writer) writeStruct(v reflect.Value) error {
if enc, ok := v.Interface().(Encodable); ok {
return enc.EncodeMsg(mw)
}
return fmt.Errorf("msgp: unsupported type: %s", v.Type())
}
func (mw *Writer) writeVal(v reflect.Value) error {
if !isSupported(v.Kind()) {
return fmt.Errorf("msgp: msgp/enc: type %q not supported", v.Type())
}
// shortcut for nil values
if v.IsNil() {
return mw.WriteNil()
}
switch v.Kind() {
case reflect.Bool:
return mw.WriteBool(v.Bool())
case reflect.Float32, reflect.Float64:
return mw.WriteFloat64(v.Float())
case reflect.Complex64, reflect.Complex128:
return mw.WriteComplex128(v.Complex())
case reflect.Int, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Int8:
return mw.WriteInt64(v.Int())
case reflect.Interface, reflect.Ptr:
if v.IsNil() {
mw.WriteNil()
}
return mw.writeVal(v.Elem())
case reflect.Map:
return mw.writeMap(v)
case reflect.Uint, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uint8:
return mw.WriteUint64(v.Uint())
case reflect.String:
return mw.WriteString(v.String())
case reflect.Slice, reflect.Array:
return mw.writeSlice(v)
case reflect.Struct:
return mw.writeStruct(v)
}
return fmt.Errorf("msgp: msgp/enc: type %q not supported", v.Type())
}
// is the reflect.Kind encodable?
func isSupported(k reflect.Kind) bool {
switch k {
case reflect.Func, reflect.Chan, reflect.Invalid, reflect.UnsafePointer:
return false
default:
return true
}
}
// GuessSize guesses the size of the underlying
// value of 'i'. If the underlying value is not
// a simple builtin (or []byte), GuessSize defaults
// to 512.
func GuessSize(i interface{}) int {
if i == nil {
return NilSize
}
switch i := i.(type) {
case Sizer:
return i.Msgsize()
case Extension:
return ExtensionPrefixSize + i.Len()
case float64:
return Float64Size
case float32:
return Float32Size
case uint8, uint16, uint32, uint64, uint:
return UintSize
case int8, int16, int32, int64, int:
return IntSize
case []byte:
return BytesPrefixSize + len(i)
case string:
return StringPrefixSize + len(i)
case complex64:
return Complex64Size
case complex128:
return Complex128Size
case bool:
return BoolSize
case map[string]interface{}:
s := MapHeaderSize
for key, val := range i {
s += StringPrefixSize + len(key) + GuessSize(val)
}
return s
case map[string]string:
s := MapHeaderSize
for key, val := range i {
s += 2*StringPrefixSize + len(key) + len(val)
}
return s
default:
return 512
}
}
package msgp
import (
"math"
"reflect"
"time"
)
// ensure 'sz' extra bytes in 'b' btw len(b) and cap(b)
func ensure(b []byte, sz int) ([]byte, int) {
l := len(b)
c := cap(b)
if c-l < sz {
o := make([]byte, (2*c)+sz) // exponential growth
n := copy(o, b)
return o[:n+sz], n
}
return b[:l+sz], l
}
// AppendMapHeader appends a map header with the
// given size to the slice
func AppendMapHeader(b []byte, sz uint32) []byte {
switch {
case sz <= 15:
return append(b, wfixmap(uint8(sz)))
case sz <= math.MaxUint16:
o, n := ensure(b, 3)
prefixu16(o[n:], mmap16, uint16(sz))
return o
default:
o, n := ensure(b, 5)
prefixu32(o[n:], mmap32, sz)
return o
}
}
// AppendArrayHeader appends an array header with
// the given size to the slice
func AppendArrayHeader(b []byte, sz uint32) []byte {
switch {
case sz <= 15:
return append(b, wfixarray(uint8(sz)))
case sz <= math.MaxUint16:
o, n := ensure(b, 3)
prefixu16(o[n:], marray16, uint16(sz))
return o
default:
o, n := ensure(b, 5)
prefixu32(o[n:], marray32, sz)
return o
}
}
// AppendNil appends a 'nil' byte to the slice
func AppendNil(b []byte) []byte { return append(b, mnil) }
// AppendFloat64 appends a float64 to the slice
func AppendFloat64(b []byte, f float64) []byte {
o, n := ensure(b, Float64Size)
prefixu64(o[n:], mfloat64, math.Float64bits(f))
return o
}
// AppendFloat32 appends a float32 to the slice
func AppendFloat32(b []byte, f float32) []byte {
o, n := ensure(b, Float32Size)
prefixu32(o[n:], mfloat32, math.Float32bits(f))
return o
}
// AppendInt64 appends an int64 to the slice
func AppendInt64(b []byte, i int64) []byte {
if i >= 0 {
switch {
case i <= math.MaxInt8:
return append(b, wfixint(uint8(i)))
case i <= math.MaxInt16:
o, n := ensure(b, 3)
putMint16(o[n:], int16(i))
return o
case i <= math.MaxInt32:
o, n := ensure(b, 5)
putMint32(o[n:], int32(i))
return o
default:
o, n := ensure(b, 9)
putMint64(o[n:], i)
return o
}
}
switch {
case i >= -32:
return append(b, wnfixint(int8(i)))
case i >= math.MinInt8:
o, n := ensure(b, 2)
putMint8(o[n:], int8(i))
return o
case i >= math.MinInt16:
o, n := ensure(b, 3)
putMint16(o[n:], int16(i))
return o
case i >= math.MinInt32:
o, n := ensure(b, 5)
putMint32(o[n:], int32(i))
return o
default:
o, n := ensure(b, 9)
putMint64(o[n:], i)
return o
}
}
// AppendInt appends an int to the slice
func AppendInt(b []byte, i int) []byte { return AppendInt64(b, int64(i)) }
// AppendInt8 appends an int8 to the slice
func AppendInt8(b []byte, i int8) []byte { return AppendInt64(b, int64(i)) }
// AppendInt16 appends an int16 to the slice
func AppendInt16(b []byte, i int16) []byte { return AppendInt64(b, int64(i)) }
// AppendInt32 appends an int32 to the slice
func AppendInt32(b []byte, i int32) []byte { return AppendInt64(b, int64(i)) }
// AppendUint64 appends a uint64 to the slice
func AppendUint64(b []byte, u uint64) []byte {
switch {
case u <= (1<<7)-1:
return append(b, wfixint(uint8(u)))
case u <= math.MaxUint8:
o, n := ensure(b, 2)
putMuint8(o[n:], uint8(u))
return o
case u <= math.MaxUint16:
o, n := ensure(b, 3)
putMuint16(o[n:], uint16(u))
return o
case u <= math.MaxUint32:
o, n := ensure(b, 5)
putMuint32(o[n:], uint32(u))
return o
default:
o, n := ensure(b, 9)
putMuint64(o[n:], u)
return o
}
}
// AppendUint appends a uint to the slice
func AppendUint(b []byte, u uint) []byte { return AppendUint64(b, uint64(u)) }
// AppendUint8 appends a uint8 to the slice
func AppendUint8(b []byte, u uint8) []byte { return AppendUint64(b, uint64(u)) }
// AppendByte is analogous to AppendUint8
func AppendByte(b []byte, u byte) []byte { return AppendUint8(b, uint8(u)) }
// AppendUint16 appends a uint16 to the slice
func AppendUint16(b []byte, u uint16) []byte { return AppendUint64(b, uint64(u)) }
// AppendUint32 appends a uint32 to the slice
func AppendUint32(b []byte, u uint32) []byte { return AppendUint64(b, uint64(u)) }
// AppendBytes appends bytes to the slice as MessagePack 'bin' data
func AppendBytes(b []byte, bts []byte) []byte {
sz := len(bts)
var o []byte
var n int
switch {
case sz <= math.MaxUint8:
o, n = ensure(b, 2+sz)
prefixu8(o[n:], mbin8, uint8(sz))
n += 2
case sz <= math.MaxUint16:
o, n = ensure(b, 3+sz)
prefixu16(o[n:], mbin16, uint16(sz))
n += 3
default:
o, n = ensure(b, 5+sz)
prefixu32(o[n:], mbin32, uint32(sz))
n += 5
}
return o[:n+copy(o[n:], bts)]
}
// AppendBool appends a bool to the slice
func AppendBool(b []byte, t bool) []byte {
if t {
return append(b, mtrue)
}
return append(b, mfalse)
}
// AppendString appends a string as a MessagePack 'str' to the slice
func AppendString(b []byte, s string) []byte {
sz := len(s)
var n int
var o []byte
switch {
case sz <= 31:
o, n = ensure(b, 1+sz)
o[n] = wfixstr(uint8(sz))
n++
case sz <= math.MaxUint8:
o, n = ensure(b, 2+sz)
prefixu8(o[n:], mstr8, uint8(sz))
n += 2
case sz <= math.MaxUint16:
o, n = ensure(b, 3+sz)
prefixu16(o[n:], mstr16, uint16(sz))
n += 3
default:
o, n = ensure(b, 5+sz)
prefixu32(o[n:], mstr32, uint32(sz))
n += 5
}
return o[:n+copy(o[n:], s)]
}
// AppendStringFromBytes appends a []byte
// as a MessagePack 'str' to the slice 'b.'
func AppendStringFromBytes(b []byte, str []byte) []byte {
sz := len(str)
var n int
var o []byte
switch {
case sz <= 31:
o, n = ensure(b, 1+sz)
o[n] = wfixstr(uint8(sz))
n++
case sz <= math.MaxUint8:
o, n = ensure(b, 2+sz)
prefixu8(o[n:], mstr8, uint8(sz))
n += 2
case sz <= math.MaxUint16:
o, n = ensure(b, 3+sz)
prefixu16(o[n:], mstr16, uint16(sz))
n += 3
default:
o, n = ensure(b, 5+sz)
prefixu32(o[n:], mstr32, uint32(sz))
n += 5
}
return o[:n+copy(o[n:], str)]
}
// AppendComplex64 appends a complex64 to the slice as a MessagePack extension
func AppendComplex64(b []byte, c complex64) []byte {
o, n := ensure(b, Complex64Size)
o[n] = mfixext8
o[n+1] = Complex64Extension
big.PutUint32(o[n+2:], math.Float32bits(real(c)))
big.PutUint32(o[n+6:], math.Float32bits(imag(c)))
return o
}
// AppendComplex128 appends a complex128 to the slice as a MessagePack extension
func AppendComplex128(b []byte, c complex128) []byte {
o, n := ensure(b, Complex128Size)
o[n] = mfixext16
o[n+1] = Complex128Extension
big.PutUint64(o[n+2:], math.Float64bits(real(c)))
big.PutUint64(o[n+10:], math.Float64bits(imag(c)))
return o
}
// AppendTime appends a time.Time to the slice as a MessagePack extension
func AppendTime(b []byte, t time.Time) []byte {
o, n := ensure(b, TimeSize)
t = t.UTC()
o[n] = mext8
o[n+1] = 12
o[n+2] = TimeExtension
putUnix(o[n+3:], t.Unix(), int32(t.Nanosecond()))
return o
}
// AppendMapStrStr appends a map[string]string to the slice
// as a MessagePack map with 'str'-type keys and values
func AppendMapStrStr(b []byte, m map[string]string) []byte {
sz := uint32(len(m))
b = AppendMapHeader(b, sz)
for key, val := range m {
b = AppendString(b, key)
b = AppendString(b, val)
}
return b
}
// AppendMapStrIntf appends a map[string]interface{} to the slice
// as a MessagePack map with 'str'-type keys.
func AppendMapStrIntf(b []byte, m map[string]interface{}) ([]byte, error) {
sz := uint32(len(m))
b = AppendMapHeader(b, sz)
var err error
for key, val := range m {
b = AppendString(b, key)
b, err = AppendIntf(b, val)
if err != nil {
return b, err
}
}
return b, nil
}
// AppendIntf appends the concrete type of 'i' to the
// provided []byte. 'i' must be one of the following:
// - 'nil'
// - A bool, float, string, []byte, int, uint, or complex
// - A map[string]interface{} or map[string]string
// - A []T, where T is another supported type
// - A *T, where T is another supported type
// - A type that satisfieds the msgp.Marshaler interface
// - A type that satisfies the msgp.Extension interface
func AppendIntf(b []byte, i interface{}) ([]byte, error) {
if i == nil {
return AppendNil(b), nil
}
// all the concrete types
// for which we have methods
switch i := i.(type) {
case Marshaler:
return i.MarshalMsg(b)
case Extension:
return AppendExtension(b, i)
case bool:
return AppendBool(b, i), nil
case float32:
return AppendFloat32(b, i), nil
case float64:
return AppendFloat64(b, i), nil
case complex64:
return AppendComplex64(b, i), nil
case complex128:
return AppendComplex128(b, i), nil
case string:
return AppendString(b, i), nil
case []byte:
return AppendBytes(b, i), nil
case int8:
return AppendInt8(b, i), nil
case int16:
return AppendInt16(b, i), nil
case int32:
return AppendInt32(b, i), nil
case int64:
return AppendInt64(b, i), nil
case int:
return AppendInt64(b, int64(i)), nil
case uint:
return AppendUint64(b, uint64(i)), nil
case uint8:
return AppendUint8(b, i), nil
case uint16:
return AppendUint16(b, i), nil
case uint32:
return AppendUint32(b, i), nil
case uint64:
return AppendUint64(b, i), nil
case time.Time:
return AppendTime(b, i), nil
case map[string]interface{}:
return AppendMapStrIntf(b, i)
case map[string]string:
return AppendMapStrStr(b, i), nil
case []interface{}:
b = AppendArrayHeader(b, uint32(len(i)))
var err error
for _, k := range i {
b, err = AppendIntf(b, k)
if err != nil {
return b, err
}
}
return b, nil
}
var err error
v := reflect.ValueOf(i)
switch v.Kind() {
case reflect.Array, reflect.Slice:
l := v.Len()
b = AppendArrayHeader(b, uint32(l))
for i := 0; i < l; i++ {
b, err = AppendIntf(b, v.Index(i).Interface())
if err != nil {
return b, err
}
}
return b, nil
case reflect.Ptr:
if v.IsNil() {
return AppendNil(b), err
}
b, err = AppendIntf(b, v.Elem().Interface())
return b, err
default:
return b, &ErrUnsupportedType{T: v.Type()}
}
}
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
// Copyright (c) 2017-2018 Uber Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package config
import (
"errors"
"fmt"
"io"
"strings"
"time"
"github.com/opentracing/opentracing-go"
"github.com/uber/jaeger-client-go"
"github.com/uber/jaeger-client-go/internal/baggage/remote"
throttler "github.com/uber/jaeger-client-go/internal/throttler/remote"
"github.com/uber/jaeger-client-go/rpcmetrics"
"github.com/uber/jaeger-client-go/transport"
"github.com/uber/jaeger-lib/metrics"
)
const defaultSamplingProbability = 0.001
// Configuration configures and creates Jaeger Tracer
type Configuration struct {
// ServiceName specifies the service name to use on the tracer.
// Can be provided via environment variable named JAEGER_SERVICE_NAME
ServiceName string `yaml:"serviceName"`
// Disabled can be provided via environment variable named JAEGER_DISABLED
Disabled bool `yaml:"disabled"`
// RPCMetrics can be provided via environment variable named JAEGER_RPC_METRICS
RPCMetrics bool `yaml:"rpc_metrics"`
// Tags can be provided via environment variable named JAEGER_TAGS
Tags []opentracing.Tag `yaml:"tags"`
Sampler *SamplerConfig `yaml:"sampler"`
Reporter *ReporterConfig `yaml:"reporter"`
Headers *jaeger.HeadersConfig `yaml:"headers"`
BaggageRestrictions *BaggageRestrictionsConfig `yaml:"baggage_restrictions"`
Throttler *ThrottlerConfig `yaml:"throttler"`
}
// SamplerConfig allows initializing a non-default sampler. All fields are optional.
type SamplerConfig struct {
// Type specifies the type of the sampler: const, probabilistic, rateLimiting, or remote
// Can be set by exporting an environment variable named JAEGER_SAMPLER_TYPE
Type string `yaml:"type"`
// Param is a value passed to the sampler.
// Valid values for Param field are:
// - for "const" sampler, 0 or 1 for always false/true respectively
// - for "probabilistic" sampler, a probability between 0 and 1
// - for "rateLimiting" sampler, the number of spans per second
// - for "remote" sampler, param is the same as for "probabilistic"
// and indicates the initial sampling rate before the actual one
// is received from the mothership.
// Can be set by exporting an environment variable named JAEGER_SAMPLER_PARAM
Param float64 `yaml:"param"`
// SamplingServerURL is the address of jaeger-agent's HTTP sampling server
// Can be set by exporting an environment variable named JAEGER_SAMPLER_MANAGER_HOST_PORT
SamplingServerURL string `yaml:"samplingServerURL"`
// MaxOperations is the maximum number of operations that the sampler
// will keep track of. If an operation is not tracked, a default probabilistic
// sampler will be used rather than the per operation specific sampler.
// Can be set by exporting an environment variable named JAEGER_SAMPLER_MAX_OPERATIONS
MaxOperations int `yaml:"maxOperations"`
// SamplingRefreshInterval controls how often the remotely controlled sampler will poll
// jaeger-agent for the appropriate sampling strategy.
// Can be set by exporting an environment variable named JAEGER_SAMPLER_REFRESH_INTERVAL
SamplingRefreshInterval time.Duration `yaml:"samplingRefreshInterval"`
}
// ReporterConfig configures the reporter. All fields are optional.
type ReporterConfig struct {
// QueueSize controls how many spans the reporter can keep in memory before it starts dropping
// new spans. The queue is continuously drained by a background go-routine, as fast as spans
// can be sent out of process.
// Can be set by exporting an environment variable named JAEGER_REPORTER_MAX_QUEUE_SIZE
QueueSize int `yaml:"queueSize"`
// BufferFlushInterval controls how often the buffer is force-flushed, even if it's not full.
// It is generally not useful, as it only matters for very low traffic services.
// Can be set by exporting an environment variable named JAEGER_REPORTER_FLUSH_INTERVAL
BufferFlushInterval time.Duration
// LogSpans, when true, enables LoggingReporter that runs in parallel with the main reporter
// and logs all submitted spans. Main Configuration.Logger must be initialized in the code
// for this option to have any effect.
// Can be set by exporting an environment variable named JAEGER_REPORTER_LOG_SPANS
LogSpans bool `yaml:"logSpans"`
// LocalAgentHostPort instructs reporter to send spans to jaeger-agent at this address
// Can be set by exporting an environment variable named JAEGER_AGENT_HOST / JAEGER_AGENT_PORT
LocalAgentHostPort string `yaml:"localAgentHostPort"`
// CollectorEndpoint instructs reporter to send spans to jaeger-collector at this URL
// Can be set by exporting an environment variable named JAEGER_ENDPOINT
CollectorEndpoint string `yaml:"collectorEndpoint"`
// User instructs reporter to include a user for basic http authentication when sending spans to jaeger-collector.
// Can be set by exporting an environment variable named JAEGER_USER
User string `yaml:"user"`
// Password instructs reporter to include a password for basic http authentication when sending spans to
// jaeger-collector. Can be set by exporting an environment variable named JAEGER_PASSWORD
Password string `yaml:"password"`
}
// BaggageRestrictionsConfig configures the baggage restrictions manager which can be used to whitelist
// certain baggage keys. All fields are optional.
type BaggageRestrictionsConfig struct {
// DenyBaggageOnInitializationFailure controls the startup failure mode of the baggage restriction
// manager. If true, the manager will not allow any baggage to be written until baggage restrictions have
// been retrieved from jaeger-agent. If false, the manager wil allow any baggage to be written until baggage
// restrictions have been retrieved from jaeger-agent.
DenyBaggageOnInitializationFailure bool `yaml:"denyBaggageOnInitializationFailure"`
// HostPort is the hostPort of jaeger-agent's baggage restrictions server
HostPort string `yaml:"hostPort"`
// RefreshInterval controls how often the baggage restriction manager will poll
// jaeger-agent for the most recent baggage restrictions.
RefreshInterval time.Duration `yaml:"refreshInterval"`
}
// ThrottlerConfig configures the throttler which can be used to throttle the
// rate at which the client may send debug requests.
type ThrottlerConfig struct {
// HostPort of jaeger-agent's credit server.
HostPort string `yaml:"hostPort"`
// RefreshInterval controls how often the throttler will poll jaeger-agent
// for more throttling credits.
RefreshInterval time.Duration `yaml:"refreshInterval"`
// SynchronousInitialization determines whether or not the throttler should
// synchronously fetch credits from the agent when an operation is seen for
// the first time. This should be set to true if the client will be used by
// a short lived service that needs to ensure that credits are fetched
// upfront such that sampling or throttling occurs.
SynchronousInitialization bool `yaml:"synchronousInitialization"`
}
type nullCloser struct{}
func (*nullCloser) Close() error { return nil }
// New creates a new Jaeger Tracer, and a closer func that can be used to flush buffers
// before shutdown.
//
// Deprecated: use NewTracer() function
func (c Configuration) New(
serviceName string,
options ...Option,
) (opentracing.Tracer, io.Closer, error) {
if serviceName != "" {
c.ServiceName = serviceName
}
return c.NewTracer(options...)
}
// NewTracer returns a new tracer based on the current configuration, using the given options,
// and a closer func that can be used to flush buffers before shutdown.
func (c Configuration) NewTracer(options ...Option) (opentracing.Tracer, io.Closer, error) {
if c.ServiceName == "" {
return nil, nil, errors.New("no service name provided")
}
if c.Disabled {
return &opentracing.NoopTracer{}, &nullCloser{}, nil
}
opts := applyOptions(options...)
tracerMetrics := jaeger.NewMetrics(opts.metrics, nil)
if c.RPCMetrics {
Observer(
rpcmetrics.NewObserver(
opts.metrics.Namespace(metrics.NSOptions{Name: "jaeger-rpc", Tags: map[string]string{"component": "jaeger"}}),
rpcmetrics.DefaultNameNormalizer,
),
)(&opts) // adds to c.observers
}
if c.Sampler == nil {
c.Sampler = &SamplerConfig{
Type: jaeger.SamplerTypeRemote,
Param: defaultSamplingProbability,
}
}
if c.Reporter == nil {
c.Reporter = &ReporterConfig{}
}
sampler := opts.sampler
if sampler == nil {
s, err := c.Sampler.NewSampler(c.ServiceName, tracerMetrics)
if err != nil {
return nil, nil, err
}
sampler = s
}
reporter := opts.reporter
if reporter == nil {
r, err := c.Reporter.NewReporter(c.ServiceName, tracerMetrics, opts.logger)
if err != nil {
return nil, nil, err
}
reporter = r
}
tracerOptions := []jaeger.TracerOption{
jaeger.TracerOptions.Metrics(tracerMetrics),
jaeger.TracerOptions.Logger(opts.logger),
jaeger.TracerOptions.CustomHeaderKeys(c.Headers),
jaeger.TracerOptions.Gen128Bit(opts.gen128Bit),
jaeger.TracerOptions.ZipkinSharedRPCSpan(opts.zipkinSharedRPCSpan),
jaeger.TracerOptions.MaxTagValueLength(opts.maxTagValueLength),
}
for _, tag := range opts.tags {
tracerOptions = append(tracerOptions, jaeger.TracerOptions.Tag(tag.Key, tag.Value))
}
for _, tag := range c.Tags {
tracerOptions = append(tracerOptions, jaeger.TracerOptions.Tag(tag.Key, tag.Value))
}
for _, obs := range opts.observers {
tracerOptions = append(tracerOptions, jaeger.TracerOptions.Observer(obs))
}
for _, cobs := range opts.contribObservers {
tracerOptions = append(tracerOptions, jaeger.TracerOptions.ContribObserver(cobs))
}
for format, injector := range opts.injectors {
tracerOptions = append(tracerOptions, jaeger.TracerOptions.Injector(format, injector))
}
for format, extractor := range opts.extractors {
tracerOptions = append(tracerOptions, jaeger.TracerOptions.Extractor(format, extractor))
}
if c.BaggageRestrictions != nil {
mgr := remote.NewRestrictionManager(
c.ServiceName,
remote.Options.Metrics(tracerMetrics),
remote.Options.Logger(opts.logger),
remote.Options.HostPort(c.BaggageRestrictions.HostPort),
remote.Options.RefreshInterval(c.BaggageRestrictions.RefreshInterval),
remote.Options.DenyBaggageOnInitializationFailure(
c.BaggageRestrictions.DenyBaggageOnInitializationFailure,
),
)
tracerOptions = append(tracerOptions, jaeger.TracerOptions.BaggageRestrictionManager(mgr))
}
if c.Throttler != nil {
debugThrottler := throttler.NewThrottler(
c.ServiceName,
throttler.Options.Metrics(tracerMetrics),
throttler.Options.Logger(opts.logger),
throttler.Options.HostPort(c.Throttler.HostPort),
throttler.Options.RefreshInterval(c.Throttler.RefreshInterval),
throttler.Options.SynchronousInitialization(
c.Throttler.SynchronousInitialization,
),
)
tracerOptions = append(tracerOptions, jaeger.TracerOptions.DebugThrottler(debugThrottler))
}
tracer, closer := jaeger.NewTracer(
c.ServiceName,
sampler,
reporter,
tracerOptions...,
)
return tracer, closer, nil
}
// InitGlobalTracer creates a new Jaeger Tracer, and sets it as global OpenTracing Tracer.
// It returns a closer func that can be used to flush buffers before shutdown.
func (c Configuration) InitGlobalTracer(
serviceName string,
options ...Option,
) (io.Closer, error) {
if c.Disabled {
return &nullCloser{}, nil
}
tracer, closer, err := c.New(serviceName, options...)
if err != nil {
return nil, err
}
opentracing.SetGlobalTracer(tracer)
return closer, nil
}
// NewSampler creates a new sampler based on the configuration
func (sc *SamplerConfig) NewSampler(
serviceName string,
metrics *jaeger.Metrics,
) (jaeger.Sampler, error) {
samplerType := strings.ToLower(sc.Type)
if samplerType == jaeger.SamplerTypeConst {
return jaeger.NewConstSampler(sc.Param != 0), nil
}
if samplerType == jaeger.SamplerTypeProbabilistic {
if sc.Param >= 0 && sc.Param <= 1.0 {
return jaeger.NewProbabilisticSampler(sc.Param)
}
return nil, fmt.Errorf(
"Invalid Param for probabilistic sampler: %v. Expecting value between 0 and 1",
sc.Param,
)
}
if samplerType == jaeger.SamplerTypeRateLimiting {
return jaeger.NewRateLimitingSampler(sc.Param), nil
}
if samplerType == jaeger.SamplerTypeRemote || sc.Type == "" {
sc2 := *sc
sc2.Type = jaeger.SamplerTypeProbabilistic
initSampler, err := sc2.NewSampler(serviceName, nil)
if err != nil {
return nil, err
}
options := []jaeger.SamplerOption{
jaeger.SamplerOptions.Metrics(metrics),
jaeger.SamplerOptions.InitialSampler(initSampler),
jaeger.SamplerOptions.SamplingServerURL(sc.SamplingServerURL),
}
if sc.MaxOperations != 0 {
options = append(options, jaeger.SamplerOptions.MaxOperations(sc.MaxOperations))
}
if sc.SamplingRefreshInterval != 0 {
options = append(options, jaeger.SamplerOptions.SamplingRefreshInterval(sc.SamplingRefreshInterval))
}
return jaeger.NewRemotelyControlledSampler(serviceName, options...), nil
}
return nil, fmt.Errorf("Unknown sampler type %v", sc.Type)
}
// NewReporter instantiates a new reporter that submits spans to the collector
func (rc *ReporterConfig) NewReporter(
serviceName string,
metrics *jaeger.Metrics,
logger jaeger.Logger,
) (jaeger.Reporter, error) {
sender, err := rc.newTransport()
if err != nil {
return nil, err
}
reporter := jaeger.NewRemoteReporter(
sender,
jaeger.ReporterOptions.QueueSize(rc.QueueSize),
jaeger.ReporterOptions.BufferFlushInterval(rc.BufferFlushInterval),
jaeger.ReporterOptions.Logger(logger),
jaeger.ReporterOptions.Metrics(metrics))
if rc.LogSpans && logger != nil {
logger.Infof("Initializing logging reporter\n")
reporter = jaeger.NewCompositeReporter(jaeger.NewLoggingReporter(logger), reporter)
}
return reporter, err
}
func (rc *ReporterConfig) newTransport() (jaeger.Transport, error) {
switch {
case rc.CollectorEndpoint != "" && rc.User != "" && rc.Password != "":
return transport.NewHTTPTransport(rc.CollectorEndpoint, transport.HTTPBatchSize(1),
transport.HTTPBasicAuth(rc.User, rc.Password)), nil
case rc.CollectorEndpoint != "":
return transport.NewHTTPTransport(rc.CollectorEndpoint, transport.HTTPBatchSize(1)), nil
default:
return jaeger.NewUDPTransport(rc.LocalAgentHostPort, 0)
}
}
// Copyright (c) 2018 The Jaeger Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package config
import (
"fmt"
"net/url"
"os"
"strconv"
"strings"
"time"
"github.com/opentracing/opentracing-go"
"github.com/pkg/errors"
"github.com/uber/jaeger-client-go"
)
const (
// environment variable names
envServiceName = "JAEGER_SERVICE_NAME"
envDisabled = "JAEGER_DISABLED"
envRPCMetrics = "JAEGER_RPC_METRICS"
envTags = "JAEGER_TAGS"
envSamplerType = "JAEGER_SAMPLER_TYPE"
envSamplerParam = "JAEGER_SAMPLER_PARAM"
envSamplerManagerHostPort = "JAEGER_SAMPLER_MANAGER_HOST_PORT"
envSamplerMaxOperations = "JAEGER_SAMPLER_MAX_OPERATIONS"
envSamplerRefreshInterval = "JAEGER_SAMPLER_REFRESH_INTERVAL"
envReporterMaxQueueSize = "JAEGER_REPORTER_MAX_QUEUE_SIZE"
envReporterFlushInterval = "JAEGER_REPORTER_FLUSH_INTERVAL"
envReporterLogSpans = "JAEGER_REPORTER_LOG_SPANS"
envEndpoint = "JAEGER_ENDPOINT"
envUser = "JAEGER_USER"
envPassword = "JAEGER_PASSWORD"
envAgentHost = "JAEGER_AGENT_HOST"
envAgentPort = "JAEGER_AGENT_PORT"
)
// FromEnv uses environment variables to set the tracer's Configuration
func FromEnv() (*Configuration, error) {
c := &Configuration{}
if e := os.Getenv(envServiceName); e != "" {
c.ServiceName = e
}
if e := os.Getenv(envRPCMetrics); e != "" {
if value, err := strconv.ParseBool(e); err == nil {
c.RPCMetrics = value
} else {
return nil, errors.Wrapf(err, "cannot parse env var %s=%s", envRPCMetrics, e)
}
}
if e := os.Getenv(envDisabled); e != "" {
if value, err := strconv.ParseBool(e); err == nil {
c.Disabled = value
} else {
return nil, errors.Wrapf(err, "cannot parse env var %s=%s", envDisabled, e)
}
}
if e := os.Getenv(envTags); e != "" {
c.Tags = parseTags(e)
}
if s, err := samplerConfigFromEnv(); err == nil {
c.Sampler = s
} else {
return nil, errors.Wrap(err, "cannot obtain sampler config from env")
}
if r, err := reporterConfigFromEnv(); err == nil {
c.Reporter = r
} else {
return nil, errors.Wrap(err, "cannot obtain reporter config from env")
}
return c, nil
}
// samplerConfigFromEnv creates a new SamplerConfig based on the environment variables
func samplerConfigFromEnv() (*SamplerConfig, error) {
sc := &SamplerConfig{}
if e := os.Getenv(envSamplerType); e != "" {
sc.Type = e
}
if e := os.Getenv(envSamplerParam); e != "" {
if value, err := strconv.ParseFloat(e, 64); err == nil {
sc.Param = value
} else {
return nil, errors.Wrapf(err, "cannot parse env var %s=%s", envSamplerParam, e)
}
}
if e := os.Getenv(envSamplerManagerHostPort); e != "" {
sc.SamplingServerURL = e
}
if e := os.Getenv(envSamplerMaxOperations); e != "" {
if value, err := strconv.ParseInt(e, 10, 0); err == nil {
sc.MaxOperations = int(value)
} else {
return nil, errors.Wrapf(err, "cannot parse env var %s=%s", envSamplerMaxOperations, e)
}
}
if e := os.Getenv(envSamplerRefreshInterval); e != "" {
if value, err := time.ParseDuration(e); err == nil {
sc.SamplingRefreshInterval = value
} else {
return nil, errors.Wrapf(err, "cannot parse env var %s=%s", envSamplerRefreshInterval, e)
}
}
return sc, nil
}
// reporterConfigFromEnv creates a new ReporterConfig based on the environment variables
func reporterConfigFromEnv() (*ReporterConfig, error) {
rc := &ReporterConfig{}
if e := os.Getenv(envReporterMaxQueueSize); e != "" {
if value, err := strconv.ParseInt(e, 10, 0); err == nil {
rc.QueueSize = int(value)
} else {
return nil, errors.Wrapf(err, "cannot parse env var %s=%s", envReporterMaxQueueSize, e)
}
}
if e := os.Getenv(envReporterFlushInterval); e != "" {
if value, err := time.ParseDuration(e); err == nil {
rc.BufferFlushInterval = value
} else {
return nil, errors.Wrapf(err, "cannot parse env var %s=%s", envReporterFlushInterval, e)
}
}
if e := os.Getenv(envReporterLogSpans); e != "" {
if value, err := strconv.ParseBool(e); err == nil {
rc.LogSpans = value
} else {
return nil, errors.Wrapf(err, "cannot parse env var %s=%s", envReporterLogSpans, e)
}
}
if e := os.Getenv(envEndpoint); e != "" {
u, err := url.ParseRequestURI(e)
if err != nil {
return nil, errors.Wrapf(err, "cannot parse env var %s=%s", envEndpoint, e)
}
rc.CollectorEndpoint = u.String()
user := os.Getenv(envUser)
pswd := os.Getenv(envPassword)
if user != "" && pswd == "" || user == "" && pswd != "" {
return nil, errors.Errorf("you must set %s and %s env vars together", envUser, envPassword)
}
rc.User = user
rc.Password = pswd
} else {
host := jaeger.DefaultUDPSpanServerHost
if e := os.Getenv(envAgentHost); e != "" {
host = e
}
port := jaeger.DefaultUDPSpanServerPort
if e := os.Getenv(envAgentPort); e != "" {
if value, err := strconv.ParseInt(e, 10, 0); err == nil {
port = int(value)
} else {
return nil, errors.Wrapf(err, "cannot parse env var %s=%s", envAgentPort, e)
}
}
rc.LocalAgentHostPort = fmt.Sprintf("%s:%d", host, port)
}
return rc, nil
}
// parseTags parses the given string into a collection of Tags.
// Spec for this value:
// - comma separated list of key=value
// - value can be specified using the notation ${envVar:defaultValue}, where `envVar`
// is an environment variable and `defaultValue` is the value to use in case the env var is not set
func parseTags(sTags string) []opentracing.Tag {
pairs := strings.Split(sTags, ",")
tags := make([]opentracing.Tag, 0)
for _, p := range pairs {
kv := strings.SplitN(p, "=", 2)
k, v := strings.TrimSpace(kv[0]), strings.TrimSpace(kv[1])
if strings.HasPrefix(v, "${") && strings.HasSuffix(v, "}") {
ed := strings.SplitN(v[2:len(v)-1], ":", 2)
e, d := ed[0], ed[1]
v = os.Getenv(e)
if v == "" && d != "" {
v = d
}
}
tag := opentracing.Tag{Key: k, Value: v}
tags = append(tags, tag)
}
return tags
}
// Copyright (c) 2017 Uber Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package config
import (
opentracing "github.com/opentracing/opentracing-go"
"github.com/uber/jaeger-lib/metrics"
"github.com/uber/jaeger-client-go"
)
// Option is a function that sets some option on the client.
type Option func(c *Options)
// Options control behavior of the client.
type Options struct {
metrics metrics.Factory
logger jaeger.Logger
reporter jaeger.Reporter
sampler jaeger.Sampler
contribObservers []jaeger.ContribObserver
observers []jaeger.Observer
gen128Bit bool
zipkinSharedRPCSpan bool
maxTagValueLength int
tags []opentracing.Tag
injectors map[interface{}]jaeger.Injector
extractors map[interface{}]jaeger.Extractor
}
// Metrics creates an Option that initializes Metrics in the tracer,
// which is used to emit statistics about spans.
func Metrics(factory metrics.Factory) Option {
return func(c *Options) {
c.metrics = factory
}
}
// Logger can be provided to log Reporter errors, as well as to log spans
// if Reporter.LogSpans is set to true.
func Logger(logger jaeger.Logger) Option {
return func(c *Options) {
c.logger = logger
}
}
// Reporter can be provided explicitly to override the configuration.
// Useful for testing, e.g. by passing InMemoryReporter.
func Reporter(reporter jaeger.Reporter) Option {
return func(c *Options) {
c.reporter = reporter
}
}
// Sampler can be provided explicitly to override the configuration.
func Sampler(sampler jaeger.Sampler) Option {
return func(c *Options) {
c.sampler = sampler
}
}
// Observer can be registered with the Tracer to receive notifications about new Spans.
func Observer(observer jaeger.Observer) Option {
return func(c *Options) {
c.observers = append(c.observers, observer)
}
}
// ContribObserver can be registered with the Tracer to receive notifications
// about new spans.
func ContribObserver(observer jaeger.ContribObserver) Option {
return func(c *Options) {
c.contribObservers = append(c.contribObservers, observer)
}
}
// Gen128Bit specifies whether to generate 128bit trace IDs.
func Gen128Bit(gen128Bit bool) Option {
return func(c *Options) {
c.gen128Bit = gen128Bit
}
}
// ZipkinSharedRPCSpan creates an option that enables sharing span ID between client
// and server spans a la zipkin. If false, client and server spans will be assigned
// different IDs.
func ZipkinSharedRPCSpan(zipkinSharedRPCSpan bool) Option {
return func(c *Options) {
c.zipkinSharedRPCSpan = zipkinSharedRPCSpan
}
}
// MaxTagValueLength can be provided to override the default max tag value length.
func MaxTagValueLength(maxTagValueLength int) Option {
return func(c *Options) {
c.maxTagValueLength = maxTagValueLength
}
}
// Tag creates an option that adds a tracer-level tag.
func Tag(key string, value interface{}) Option {
return func(c *Options) {
c.tags = append(c.tags, opentracing.Tag{Key: key, Value: value})
}
}
// Injector registers an Injector with the given format.
func Injector(format interface{}, injector jaeger.Injector) Option {
return func(c *Options) {
c.injectors[format] = injector
}
}
// Extractor registers an Extractor with the given format.
func Extractor(format interface{}, extractor jaeger.Extractor) Option {
return func(c *Options) {
c.extractors[format] = extractor
}
}
func applyOptions(options ...Option) Options {
opts := Options{
injectors: make(map[interface{}]jaeger.Injector),
extractors: make(map[interface{}]jaeger.Extractor),
}
for _, option := range options {
option(&opts)
}
if opts.metrics == nil {
opts.metrics = metrics.NullFactory
}
if opts.logger == nil {
opts.logger = jaeger.NullLogger
}
return opts
}
// Copyright (c) 2017 Uber Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package log
import (
"bytes"
"fmt"
"log"
"sync"
)
// Logger provides an abstract interface for logging from Reporters.
// Applications can provide their own implementation of this interface to adapt
// reporters logging to whatever logging library they prefer (stdlib log,
// logrus, go-logging, etc).
type Logger interface {
// Error logs a message at error priority
Error(msg string)
// Infof logs a message at info priority
Infof(msg string, args ...interface{})
}
// StdLogger is implementation of the Logger interface that delegates to default `log` package
var StdLogger = &stdLogger{}
type stdLogger struct{}
func (l *stdLogger) Error(msg string) {
log.Printf("ERROR: %s", msg)
}
// Infof logs a message at info priority
func (l *stdLogger) Infof(msg string, args ...interface{}) {
log.Printf(msg, args...)
}
// NullLogger is implementation of the Logger interface that is no-op
var NullLogger = &nullLogger{}
type nullLogger struct{}
func (l *nullLogger) Error(msg string) {}
func (l *nullLogger) Infof(msg string, args ...interface{}) {}
// BytesBufferLogger implements Logger backed by a bytes.Buffer.
type BytesBufferLogger struct {
mux sync.Mutex
buf bytes.Buffer
}
// Error implements Logger.
func (l *BytesBufferLogger) Error(msg string) {
l.mux.Lock()
l.buf.WriteString(fmt.Sprintf("ERROR: %s\n", msg))
l.mux.Unlock()
}
// Infof implements Logger.
func (l *BytesBufferLogger) Infof(msg string, args ...interface{}) {
l.mux.Lock()
l.buf.WriteString("INFO: " + fmt.Sprintf(msg, args...) + "\n")
l.mux.Unlock()
}
// String returns string representation of the underlying buffer.
func (l *BytesBufferLogger) String() string {
l.mux.Lock()
defer l.mux.Unlock()
return l.buf.String()
}
// Flush empties the underlying buffer.
func (l *BytesBufferLogger) Flush() {
l.mux.Lock()
defer l.mux.Unlock()
l.buf.Reset()
}
The MIT License (MIT)
Copyright (c) 2016-2017 GitLab B.V.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
package correlation
import "bytes"
const base62Chars string = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
// encodeReverseBase62 encodes num into its Base62 reversed representation.
// The most significant value is at the end of the string.
//
// Appending is faster than prepending and this is enough for the purpose of a random ID
func encodeReverseBase62(num int64) string {
if num == 0 {
return "0"
}
encoded := bytes.Buffer{}
for q := num; q > 0; q /= 62 {
encoded.Write([]byte{base62Chars[q%62]})
}
return encoded.String()
}
package correlation
import (
"context"
)
type ctxKey int
const keyCorrelationID ctxKey = iota
// ExtractFromContext extracts the CollectionID from the provided context
// Returns an empty string if it's unable to extract the CorrelationID for
// any reason.
func ExtractFromContext(ctx context.Context) string {
if ctx == nil {
return ""
}
id := ctx.Value(keyCorrelationID)
str, ok := id.(string)
if !ok {
return ""
}
return str
}
// ContextWithCorrelation will create a new context containing the provided Correlation-ID value
// This can be extracted using ExtractFromContext
func ContextWithCorrelation(ctx context.Context, correlationID string) context.Context {
return context.WithValue(ctx, keyCorrelationID, correlationID)
}
/*
Package correlation is the primary entrypoint into LabKit's correlation utilities.
Provided Functionality
Provides http middlewares for general purpose use cases:
Generating a correlation-id for an incoming HTTP request using .
Obtaining the correlation-id from the context.
Injecting the correlation-id from the context into an outgoing request.
Still to be implemented
Extracting a correlation-id from an incoming HTTP request into the context.
*/
package correlation
package correlation
import (
"crypto/rand"
"fmt"
"log"
"math"
"math/big"
"net/http"
"time"
)
var (
randMax = big.NewInt(math.MaxInt64)
randSource = rand.Reader
)
// generateRandomCorrelationID will attempt to generate a correlationid randomly
// or raise an error
func generateRandomCorrelationID() (string, error) {
id, err := rand.Int(randSource, randMax)
if err != nil {
return "", err
}
base62 := encodeReverseBase62(id.Int64())
return base62, nil
}
func generatePseudorandomCorrelationID(req *http.Request) string {
return fmt.Sprintf("E:%s:%s", req.RemoteAddr, encodeReverseBase62(time.Now().UnixNano()))
}
// generateRandomCorrelationID will attempt to generate a correlationid randomly
// if this fails, will log a message and fallback to a pseudorandom approach
func generateRandomCorrelationIDWithFallback(req *http.Request) string {
correlationID, err := generateRandomCorrelationID()
if err == nil {
return correlationID
}
log.Printf("can't generate random correlation-id: %v", err)
return generatePseudorandomCorrelationID(req)
}
package correlation
import (
"net/http"
)
// InjectCorrelationID is an HTTP middleware to generate an Correlation-ID for the incoming request,
// or extract the existing Correlation-ID from the incoming request. By default, any upstream Correlation-ID,
// passed in via the `X-Request-ID` header will be ignored. To enable this behaviour, the `WithPropagation`
// option should be passed into the options.
// Whether the Correlation-ID is generated or propagated, once inside this handler the request context
// will have a Correlation-ID associated with it.
func InjectCorrelationID(h http.Handler, opts ...InboundHandlerOption) http.Handler {
config := applyInboundHandlerOptions(opts)
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
parent := r.Context()
correlationID := ""
if config.propagation {
correlationID = extractFromRequest(r)
}
if correlationID == "" {
correlationID = generateRandomCorrelationIDWithFallback(r)
}
if config.sendResponseHeader {
setResponseHeader(w, correlationID)
}
ctx := ContextWithCorrelation(parent, correlationID)
h.ServeHTTP(w, r.WithContext(ctx))
})
}
func extractFromRequest(r *http.Request) string {
return r.Header.Get(propagationHeader)
}
func setResponseHeader(w http.ResponseWriter, correlationID string) {
w.Header().Set(propagationHeader, correlationID)
}
package correlation
// The configuration for InjectCorrelationID
type inboundHandlerConfig struct {
propagation bool
sendResponseHeader bool
}
// InboundHandlerOption will configure a correlation handler
// currently there are no options, but this gives us the option
// to extend the interface in a backwards compatible way
type InboundHandlerOption func(*inboundHandlerConfig)
func applyInboundHandlerOptions(opts []InboundHandlerOption) inboundHandlerConfig {
config := inboundHandlerConfig{
propagation: false,
}
for _, v := range opts {
v(&config)
}
return config
}
// WithPropagation will configure the handler to propagate existing correlation_ids
// passed in from upstream services.
// This is not the default behaviour.
func WithPropagation() InboundHandlerOption {
return func(config *inboundHandlerConfig) {
config.propagation = true
}
}
// WithSetResponseHeader will configure the handler to set the correlation_id
// in the http response headers
func WithSetResponseHeader() InboundHandlerOption {
return func(config *inboundHandlerConfig) {
config.sendResponseHeader = true
}
}
package correlation
import (
"net/http"
)
const propagationHeader = "X-Request-ID"
// injectRequest will pass the CorrelationId through to a downstream http request
// for propagation
func injectRequest(req *http.Request) {
correlationID := ExtractFromContext(req.Context())
if correlationID != "" {
req.Header.Set(propagationHeader, correlationID)
}
}
type instrumentedRoundTripper struct {
delegate http.RoundTripper
}
func (c instrumentedRoundTripper) RoundTrip(req *http.Request) (res *http.Response, e error) {
injectRequest(req)
return c.delegate.RoundTrip(req)
}
// NewInstrumentedRoundTripper acts as a "client-middleware" for outbound http requests
// adding instrumentation to the outbound request and then delegating to the underlying
// transport.
//
// If will extract the current Correlation-ID from the request context and pass this via
// the X-Request-ID request header to downstream services.
func NewInstrumentedRoundTripper(delegate http.RoundTripper, opts ...InstrumentedRoundTripperOption) http.RoundTripper {
// Currently we don't use any of the options available
applyInstrumentedRoundTripperOptions(opts)
return &instrumentedRoundTripper{delegate: delegate}
}
package correlation
// The configuration for InjectCorrelationID
type instrumentedRoundTripperConfig struct {
}
// InstrumentedRoundTripperOption will configure a correlation handler
// currently there are no options, but this gives us the option
// to extend the interface in a backwards compatible way
type InstrumentedRoundTripperOption func(*instrumentedRoundTripperConfig)
func applyInstrumentedRoundTripperOptions(opts []InstrumentedRoundTripperOption) instrumentedRoundTripperConfig {
config := instrumentedRoundTripperConfig{}
for _, v := range opts {
v(&config)
}
return config
}
package connstr
import (
"fmt"
"net/url"
"regexp"
)
// Connection strings:
// * opentracing://jaeger
// * opentracing://datadog
// * opentracing://lightstep?access_key=12345
var errInvalidConnection = fmt.Errorf("invalid connection string")
// Parse parses a opentracing connection string into a driverName and options map.
func Parse(connectionString string) (driverName string, options map[string]string, err error) {
if connectionString == "" {
return "", nil, errInvalidConnection
}
URL, err := url.Parse(connectionString)
if err != nil {
return "", nil, errInvalidConnection
}
if URL.Scheme != "opentracing" {
return "", nil, errInvalidConnection
}
driverName = URL.Host
if driverName == "" {
return "", nil, errInvalidConnection
}
// Connection strings should not have a path
if URL.Path != "" {
return "", nil, errInvalidConnection
}
matched, err := regexp.MatchString("^[a-z0-9_]+$", driverName)
if err != nil || !matched {
return "", nil, errInvalidConnection
}
query := URL.Query()
driverName = URL.Host
options = make(map[string]string, len(query))
for k := range query {
options[k] = query.Get(k)
}
return driverName, options, nil
}
/*
Package tracing is the primary entrypoint into LabKit's distributed tracing functionality.
(This documentation assumes some minimal knowledge of Distributed Tracing, and uses
tracing terminology without providing definitions. Please review
https://opentracing.io/docs/overview/what-is-tracing/ for an broad overview of distributed
tracing if you are not familiar with the technology)
Internally the `tracing` package relies on Opentracing, but avoids leaking this abstraction.
In theory, LabKit could replace Opentracing with another distributed tracing interface, such
as Zipkin or OpenCensus, without needing to make changes to the application (other than vendoring
in a new version of LabKit, of course).
This design decision is deliberate: the package should not leak the underlying tracing implementation.
The package provides three primary exports:
* `tracing.Initialize()` for initializing the global tracer using the `GITLAB_TRACING` environment variable.
* An HTTP Handler middleware, `tracing.Handler()`, for instrumenting incoming HTTP requests.
* An HTTP RoundTripper, `tracing.NewRoundTripper()` for instrumenting outbound HTTP requests to other services.
The provided example in `example_test.go` demonstrates usage of both the HTTP Middleware and the HTTP RoundTripper.
*Initializing the global tracer*
Opentracing makes use of a global tracer. Opentracing ships with a default NoOp tracer which does
nothing at all. This is always configured, meaning that, without initialization, Opentracing does nothing and
has a very low overhead.
LabKit's tracing is configured through an environment variable, `GITLAB_TRACING`. This environment variable contains
a "connection string"-like configuration, such as:
* `opentracing://jaeger?udp_endpoint=localhost:6831`
* `opentracing://datadog`
* `opentracing://lightstep`
The parameters for these connection-strings are implementation specific.
This configuration is identical to the one used to configure GitLab's ruby tracing libraries in the `Gitlab::Tracing`
package. Having a consistent configuration makes it easy to configure multiple processes at the same time. For example,
in GitLab Development Kit, tracing can be configured with a single environment variable, `GITLAB_TRACING=... gdk run`,
since `GITLAB_TRACING` will configure Workhorse (written in Go), Gitaly (written in Go) and GitLab's rails components,
using the same configuration.
*Compiling applications with Tracing support*
Go's Opentracing interface does not allow tracing implementations to be loaded dynamically; implementations need to be
compiled into the application. With LabKit, this is done conditionally, using build tags. Two build tags need to be
specified:
* `tracer_static` - this compiles in the static plugin registry support
* `tracer_static_[DRIVER_NAME]` - this compile in support for the given driver.
For example, to compile support for Jaeger, compile your Go app with `tracer_static,tracer_static_jaeger`
Note that multiple (or all) drivers can be compiled in alongside one another: using the tags:
`tracer_static,tracer_static_jaeger,tracer_static_lightstep,tracer_static_datadog`
If the `GITLAB_TRACING` environment variable references an unknown or unregistered driver, it will log a message
and continue without tracing. This is a deliberate decision: the risk of bringing down a cluster during a rollout
with a misconfigured tracer configuration is greater than the risk of an operator loosing some time because
their application was not compiled with the correct tracers.
*Using the HTTP Handler middleware to instrument incoming HTTP requests*
When an incoming HTTP request arrives on the server, it may already include Distributed Tracing headers,
propagated from an upstream service.
The tracing middleware will attempt to extract the tracing information from the headers (the exact headers used are
tracing implementation specific), set up a span and pass the information through the request context.
It is up to the Opentracing implementation to decide whether the span will be sent to the tracing infrastructure.
This will be implementation-specific, but generally relies on server load, sampler configuration, whether an
error occurred, whether certain spans took an anomalous amount of time, etc.
*Using the HTTP RoundTripper to instrument outgoing HTTP requests*
The RoundTripper should be added to the HTTP client RoundTripper stack (see the example). When an outbound
HTTP request is sent from the HTTP client, the RoundTripper will determine whether there is an active span
and if so, will inject headers into the outgoing HTTP request identifying the span. The details of these
headers is implementation specific.
It is important to ensure that the context is passed into the outgoing request, using `req.WithContext(ctx)`
so that the correct span information can be injected into the request headers.
*Propagating tracing information to child processes*
Sometimes we want a trace to continue from a parent process to a spawned child process. For this,
the tracing package provides `tracing.NewEnvInjector()` and `tracing.ExtractFromEnv()`, for the
parent and child processes respectively.
NewEnvInjector() will configure a []string array of environment variables, ensuring they have the
correct tracing configuration and any trace and span identifiers. NewEnvInjector() should be called
in the child process and will extract the trace and span information from the environment.
Please review the examples in the godocs for details of how to implement both approaches.
*/
package tracing
package tracing
import (
"context"
"os"
"strings"
opentracing "github.com/opentracing/opentracing-go"
"github.com/opentracing/opentracing-go/ext"
"gitlab.com/gitlab-org/labkit/correlation"
)
// ExtractFromEnv will extract a span from the environment after it has been passed in
// from the parent process. Returns a new context, and a defer'able function, which
// should be called on process termination
func ExtractFromEnv(ctx context.Context, opts ...ExtractFromEnvOption) (context.Context, func()) {
/* config not yet used */ applyExtractFromEnvOptions(opts)
tracer := opentracing.GlobalTracer()
// Extract the Correlation-ID
envMap := environAsMap(os.Environ())
correlationID := envMap[envCorrelationIDKey]
if correlationID != "" {
ctx = correlation.ContextWithCorrelation(ctx, correlationID)
}
// Attempt to deserialize tracing identifiers
wireContext, err := tracer.Extract(
opentracing.TextMap,
opentracing.TextMapCarrier(envMap))
if err != nil {
/* Clients could send bad data, in which case we simply ignore it */
return ctx, func() {}
}
// Create the span referring to the RPC client if available.
// If wireContext == nil, a root span will be created.
additionalStartSpanOpts := []opentracing.StartSpanOption{
ext.RPCServerOption(wireContext),
}
if correlationID != "" {
additionalStartSpanOpts = append(additionalStartSpanOpts, opentracing.Tag{Key: "correlation_id", Value: correlationID})
}
serverSpan := opentracing.StartSpan(
"execute",
additionalStartSpanOpts...,
)
ctx = opentracing.ContextWithSpan(ctx, serverSpan)
return ctx, func() { serverSpan.Finish() }
}
func environAsMap(env []string) map[string]string {
envMap := make(map[string]string, len(env))
for _, v := range env {
s := strings.SplitN(v, "=", 2)
envMap[s[0]] = s[1]
}
return envMap
}
package tracing
type extractFromEnvConfig struct{}
// ExtractFromEnvOption will configure an environment injector
type ExtractFromEnvOption func(*extractFromEnvConfig)
func applyExtractFromEnvOptions(opts []ExtractFromEnvOption) extractFromEnvConfig {
config := extractFromEnvConfig{}
for _, v := range opts {
v(&config)
}
return config
}
package tracing
import (
"context"
"fmt"
"log"
"os"
"sort"
opentracing "github.com/opentracing/opentracing-go"
"gitlab.com/gitlab-org/labkit/correlation"
)
// envCorrelationIDKey is used to pass the current correlation-id over to the child process
const envCorrelationIDKey = "CORRELATION_ID"
// EnvInjector will inject tracing information into an environment in preparation for
// spawning a child process. This includes trace and span identifiers, as well
// as the GITLAB_TRACING configuration. Will gracefully degrade if tracing is
// not configured, or an active span is not currently available.
type EnvInjector func(ctx context.Context, env []string) []string
// NewEnvInjector will create a new environment injector
func NewEnvInjector(opts ...EnvInjectorOption) EnvInjector {
/* config not yet used */ applyEnvInjectorOptions(opts)
return func(ctx context.Context, env []string) []string {
envMap := map[string]string{}
// Pass the Correlation-ID through the environment if set
correlationID := correlation.ExtractFromContext(ctx)
if correlationID != "" {
envMap[envCorrelationIDKey] = correlationID
}
// Also include the GITLAB_TRACING configuration so that
// the child process knows how to configure itself
v, ok := os.LookupEnv(tracingEnvKey)
if ok {
envMap[tracingEnvKey] = v
}
span := opentracing.SpanFromContext(ctx)
if span == nil {
// If no active span, short circuit
return appendMapToEnv(env, envMap)
}
carrier := opentracing.TextMapCarrier(envMap)
err := span.Tracer().Inject(span.Context(), opentracing.TextMap, carrier)
if err != nil {
log.Printf("tracing span injection failed: %v", err)
}
return appendMapToEnv(env, envMap)
}
}
// appendMapToEnv takes a map of key,value pairs and appends it to an
// array of environment variable pairs in `K=V` string pairs
func appendMapToEnv(env []string, envMap map[string]string) []string {
additionalItems := []string{}
for k, v := range envMap {
additionalItems = append(additionalItems, fmt.Sprintf("%s=%s", k, v))
}
sort.Strings(additionalItems)
return append(env, additionalItems...)
}
package tracing
type envInjectorConfig struct{}
// EnvInjectorOption will configure an environment injector
type EnvInjectorOption func(*envInjectorConfig)
func applyEnvInjectorOptions(opts []EnvInjectorOption) envInjectorConfig {
config := envInjectorConfig{}
for _, v := range opts {
v(&config)
}
return config
}
package tracing
import (
"fmt"
)
// ErrConfiguration is returned when the tracer is not properly configured.
var ErrConfiguration = fmt.Errorf("Tracing is not properly configured")
package impl
const keyStrictConnectionParsing = "strict_parsing"
// +build tracer_static,tracer_static_datadog
package impl
import (
"io"
opentracing "github.com/opentracing/opentracing-go"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace/opentracer"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer"
)
func tracerFactory(config map[string]string) (opentracing.Tracer, io.Closer, error) {
opts := []tracer.StartOption{}
if config["ServiceName"] != "" {
opts = append(opts, tracer.WithServiceName(config["ServiceName"]))
}
return opentracer.New(opts...), nil, nil
}
func init() {
registerTracer("datadog", tracerFactory)
}
// +build tracer_static,tracer_static_jaeger
package impl
import (
"fmt"
"io"
"log"
"strconv"
opentracing "github.com/opentracing/opentracing-go"
jaegercfg "github.com/uber/jaeger-client-go/config"
jaegerlog "github.com/uber/jaeger-client-go/log"
)
type traceConfigMapper func(traceCfg *jaegercfg.Configuration, value string) ([]jaegercfg.Option, error)
var configMapper = map[string]traceConfigMapper{
"ServiceName": func(traceCfg *jaegercfg.Configuration, value string) ([]jaegercfg.Option, error) {
traceCfg.ServiceName = value
return nil, nil
},
"debug": func(traceCfg *jaegercfg.Configuration, value string) ([]jaegercfg.Option, error) {
return []jaegercfg.Option{jaegercfg.Logger(jaegerlog.StdLogger)}, nil
},
"sampler": func(traceCfg *jaegercfg.Configuration, value string) ([]jaegercfg.Option, error) {
if traceCfg.Sampler == nil {
traceCfg.Sampler = &jaegercfg.SamplerConfig{}
}
traceCfg.Sampler.Type = value
return nil, nil
},
"sampler_param": func(traceCfg *jaegercfg.Configuration, value string) ([]jaegercfg.Option, error) {
if traceCfg.Sampler == nil {
traceCfg.Sampler = &jaegercfg.SamplerConfig{}
}
valuef, err := strconv.ParseFloat(value, 64)
if err != nil {
return nil, fmt.Errorf("jaeger tracer: sampler_param must be a float")
}
traceCfg.Sampler.Param = valuef
return nil, nil
},
"http_endpoint": func(traceCfg *jaegercfg.Configuration, value string) ([]jaegercfg.Option, error) {
if traceCfg.Reporter == nil {
traceCfg.Reporter = &jaegercfg.ReporterConfig{}
}
traceCfg.Reporter.CollectorEndpoint = value
return nil, nil
},
"udp_endpoint": func(traceCfg *jaegercfg.Configuration, value string) ([]jaegercfg.Option, error) {
if traceCfg.Reporter == nil {
traceCfg.Reporter = &jaegercfg.ReporterConfig{}
}
traceCfg.Reporter.LocalAgentHostPort = value
return nil, nil
},
}
func jaegerTracerFactory(config map[string]string) (opentracing.Tracer, io.Closer, error) {
traceCfg, err := jaegercfg.FromEnv()
if err != nil {
return nil, nil, err
}
options := []jaegercfg.Option{}
// Convert the configuration map into a jaeger configuration
for k, v := range config {
mapper := configMapper[k]
if k == keyStrictConnectionParsing {
continue
}
if mapper != nil {
o, err := mapper(traceCfg, v)
if err != nil {
return nil, nil, err
}
options = append(options, o...)
} else {
if config[keyStrictConnectionParsing] != "" {
return nil, nil, fmt.Errorf("jaeger tracer: invalid option: %s", k)
}
log.Printf("jaeger tracer: warning: ignoring unknown configuration option: %s", k)
}
}
return traceCfg.NewTracer(options...)
}
func init() {
registerTracer("jaeger", jaegerTracerFactory)
}
// +build tracer_static,tracer_static_lightstep
package impl
import (
"context"
"fmt"
"io"
"log"
lightstep "github.com/lightstep/lightstep-tracer-go"
opentracing "github.com/opentracing/opentracing-go"
)
type lightstepCloser struct {
tracer lightstep.Tracer
}
func (c *lightstepCloser) Close() error {
lightstep.Close(context.Background(), c.tracer)
return nil
}
var lightstepConfigMapper = map[string]func(traceCfg *lightstep.Options, value string) error{
"ServiceName": func(options *lightstep.Options, value string) error {
options.Tags[lightstep.ComponentNameKey] = value
return nil
},
"access_token": func(options *lightstep.Options, value string) error {
options.AccessToken = value
return nil
},
}
func lightstepTracerFactory(config map[string]string) (opentracing.Tracer, io.Closer, error) {
options := lightstep.Options{
Tags: map[string]interface{}{},
}
// Convert the configuration map into a jaeger configuration
for k, v := range config {
mapper := lightstepConfigMapper[k]
if k == keyStrictConnectionParsing {
continue
}
if mapper != nil {
err := mapper(&options, v)
if err != nil {
return nil, nil, err
}
} else {
if config[keyStrictConnectionParsing] != "" {
return nil, nil, fmt.Errorf("lightstep tracer: invalid option: %s", k)
}
log.Printf("lightstep tracer: warning: ignoring unknown configuration option: %s", k)
}
}
tracer := lightstep.NewTracer(options)
if tracer == nil {
return nil, nil, fmt.Errorf("lightstep tracer: unable to create tracer, review log messages")
}
return tracer, &lightstepCloser{tracer}, nil
}
func init() {
registerTracer("lightstep", lightstepTracerFactory)
}
// +build !tracer_static
package impl
import (
"fmt"
"io"
opentracing "github.com/opentracing/opentracing-go"
)
// New will instantiate a new instance of the tracer, given the driver and configuration
func New(driverName string, config map[string]string) (opentracing.Tracer, io.Closer, error) {
return nil, nil, fmt.Errorf("tracer: binary compiled without tracer support: cannot load driver %s", driverName)
}
// +build tracer_static
package impl
import (
"fmt"
"io"
opentracing "github.com/opentracing/opentracing-go"
)
// New will instantiate a new instance of the tracer, given the driver and configuration
func New(driverName string, config map[string]string) (opentracing.Tracer, io.Closer, error) {
factory := registry[driverName]
if factory == nil {
return nil, nil, fmt.Errorf("tracer: unable to load driver %s", driverName)
}
return factory(config)
}
package impl
import (
"io"
opentracing "github.com/opentracing/opentracing-go"
)
type tracerFactoryFunc func(config map[string]string) (opentracing.Tracer, io.Closer, error)
var registry = map[string]tracerFactoryFunc{}
func registerTracer(name string, factory tracerFactoryFunc) {
registry[name] = factory
}
package tracing
import (
"net/http"
opentracing "github.com/opentracing/opentracing-go"
"github.com/opentracing/opentracing-go/ext"
"gitlab.com/gitlab-org/labkit/correlation"
)
// Handler will extract tracing from inbound request
func Handler(h http.Handler, opts ...HandlerOption) http.Handler {
config := applyHandlerOptions(opts)
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
tracer := opentracing.GlobalTracer()
if tracer == nil {
h.ServeHTTP(w, r)
return
}
wireContext, _ := tracer.Extract(
opentracing.HTTPHeaders,
opentracing.HTTPHeadersCarrier(r.Header))
// Create the span referring to the RPC client if available.
// If wireContext == nil, a root span will be created.
additionalStartSpanOpts := []opentracing.StartSpanOption{
ext.RPCServerOption(wireContext),
}
correlationID := correlation.ExtractFromContext(r.Context())
if correlationID != "" {
additionalStartSpanOpts = append(additionalStartSpanOpts, opentracing.Tag{Key: "correlation_id", Value: correlationID})
}
serverSpan := opentracing.StartSpan(
config.getOperationName(r),
additionalStartSpanOpts...,
)
defer serverSpan.Finish()
ctx := opentracing.ContextWithSpan(r.Context(), serverSpan)
h.ServeHTTP(w, r.WithContext(ctx))
})
}
package tracing
import (
"fmt"
"net/http"
)
// OperationNamer will return an operation name given an HTTP request
type OperationNamer func(*http.Request) string
// The configuration for InjectCorrelationID
type handlerConfig struct {
getOperationName OperationNamer
}
// HandlerOption will configure a correlation handler
type HandlerOption func(*handlerConfig)
func applyHandlerOptions(opts []HandlerOption) handlerConfig {
config := handlerConfig{
getOperationName: func(req *http.Request) string {
// By default use `GET /x/y/z` for operation names
return fmt.Sprintf("%s %s", req.Method, req.URL.Path)
},
}
for _, v := range opts {
v(&config)
}
return config
}
// WithRouteIdentifier allows a RouteIdentifier attribute to be set in the handler.
// This value will appear in the traces
func WithRouteIdentifier(routeIdentifier string) HandlerOption {
return func(config *handlerConfig) {
config.getOperationName = func(req *http.Request) string {
// Use `GET routeIdentifier` for operation names
return fmt.Sprintf("%s %s", req.Method, routeIdentifier)
}
}
}
package tracing
import (
"io"
"log"
opentracing "github.com/opentracing/opentracing-go"
"gitlab.com/gitlab-org/labkit/tracing/connstr"
"gitlab.com/gitlab-org/labkit/tracing/impl"
)
type nopCloser struct {
}
func (nopCloser) Close() error { return nil }
// Initialize will initialize distributed tracing
func Initialize(opts ...InitializationOption) io.Closer {
config := applyInitializationOptions(opts)
if config.connectionString == "" {
// No opentracing connection has been set
return &nopCloser{}
}
driverName, options, err := connstr.Parse(config.connectionString)
if err != nil {
log.Printf("unable to parse connection: %v", err)
return &nopCloser{}
}
if config.serviceName != "" {
options["ServiceName"] = config.serviceName
}
tracer, closer, err := impl.New(driverName, options)
if err != nil {
log.Printf("skipping tracing configuration step: %v", err)
return &nopCloser{}
}
if tracer == nil {
log.Printf("no tracer provided, tracing will be disabled")
} else {
log.Printf("Tracing enabled")
opentracing.SetGlobalTracer(tracer)
}
if closer == nil {
return &nopCloser{}
}
return closer
}
package tracing
import (
"os"
"path"
)
const tracingEnvKey = "GITLAB_TRACING"
// The configuration for InjectCorrelationID
type initializationConfig struct {
serviceName string
connectionString string
}
// InitializationOption will configure a correlation handler
type InitializationOption func(*initializationConfig)
func applyInitializationOptions(opts []InitializationOption) initializationConfig {
config := initializationConfig{
serviceName: path.Base(os.Args[0]),
connectionString: os.Getenv(tracingEnvKey),
}
for _, v := range opts {
v(&config)
}
return config
}
// WithServiceName allows the service name to be configured for the tracer
// this will appear in traces.
func WithServiceName(serviceName string) InitializationOption {
return func(config *initializationConfig) {
config.serviceName = serviceName
}
}
// WithConnectionString allows the opentracing connection string to be overridden. By default
// this will be retrieved from the GITLAB_TRACING environment variable.
func WithConnectionString(connectionString string) InitializationOption {
return func(config *initializationConfig) {
config.connectionString = connectionString
}
}
package tracing
import (
"crypto/tls"
"log"
"net/http"
"net/http/httptrace"
opentracing "github.com/opentracing/opentracing-go"
"github.com/opentracing/opentracing-go/ext"
otlog "github.com/opentracing/opentracing-go/log"
)
type tracingRoundTripper struct {
delegate http.RoundTripper
config roundTripperConfig
}
func (c tracingRoundTripper) RoundTrip(req *http.Request) (res *http.Response, e error) {
tracer := opentracing.GlobalTracer()
if tracer == nil {
return c.delegate.RoundTrip(req)
}
ctx := req.Context()
var parentCtx opentracing.SpanContext
parentSpan := opentracing.SpanFromContext(ctx)
if parentSpan != nil {
parentCtx = parentSpan.Context()
}
// start a new Span to wrap HTTP request
span := opentracing.StartSpan(
c.config.getOperationName(req),
opentracing.ChildOf(parentCtx),
)
defer span.Finish()
ctx = opentracing.ContextWithSpan(ctx, span)
// attach ClientTrace to the Context, and Context to request
trace := newClientTrace(span)
ctx = httptrace.WithClientTrace(ctx, trace)
req = req.WithContext(ctx)
ext.SpanKindRPCClient.Set(span)
ext.HTTPUrl.Set(span, req.URL.String())
ext.HTTPMethod.Set(span, req.Method)
carrier := opentracing.HTTPHeadersCarrier(req.Header)
err := span.Tracer().Inject(span.Context(), opentracing.HTTPHeaders, carrier)
if err != nil {
log.Printf("tracing span injection failed: %v", err)
}
response, err := c.delegate.RoundTrip(req)
if err != nil {
span.LogFields(
otlog.String("event", "roundtrip error"),
otlog.Object("error", err),
)
} else {
span.LogFields(
otlog.String("event", "roundtrip complete"),
otlog.Int("status", response.StatusCode),
)
}
return response, err
}
func newClientTrace(span opentracing.Span) *httptrace.ClientTrace {
trace := &clientTrace{span: span}
return &httptrace.ClientTrace{
GotFirstResponseByte: trace.gotFirstResponseByte,
ConnectStart: trace.connectStart,
ConnectDone: trace.connectDone,
TLSHandshakeStart: trace.tlsHandshakeStart,
TLSHandshakeDone: trace.tlsHandshakeDone,
WroteHeaders: trace.wroteHeaders,
WroteRequest: trace.wroteRequest,
}
}
// clientTrace holds a reference to the Span and
// provides methods used as ClientTrace callbacks
type clientTrace struct {
span opentracing.Span
}
func (h *clientTrace) gotFirstResponseByte() {
h.span.LogFields(otlog.String("event", "got first response byte"))
}
func (h *clientTrace) connectStart(network, addr string) {
h.span.LogFields(
otlog.String("event", "connect started"),
otlog.String("network", network),
otlog.String("addr", addr),
)
}
func (h *clientTrace) connectDone(network, addr string, err error) {
h.span.LogFields(
otlog.String("event", "connect done"),
otlog.String("network", network),
otlog.String("addr", addr),
otlog.Object("error", err),
)
}
func (h *clientTrace) tlsHandshakeStart() {
h.span.LogFields(otlog.String("event", "tls handshake started"))
}
func (h *clientTrace) tlsHandshakeDone(state tls.ConnectionState, err error) {
h.span.LogFields(
otlog.String("event", "tls handshake done"),
otlog.Object("error", err),
)
}
func (h *clientTrace) wroteHeaders() {
h.span.LogFields(otlog.String("event", "headers written"))
}
func (h *clientTrace) wroteRequest(info httptrace.WroteRequestInfo) {
h.span.LogFields(
otlog.String("event", "request written"),
otlog.Object("error", info.Err),
)
}
// NewRoundTripper acts as a "client-middleware" for outbound http requests
// adding instrumentation to the outbound request and then delegating to the underlying
// transport
func NewRoundTripper(delegate http.RoundTripper, opts ...RoundTripperOption) http.RoundTripper {
config := applyRoundTripperOptions(opts)
return &tracingRoundTripper{delegate: delegate, config: config}
}
package tracing
import (
"fmt"
"net/http"
)
// The configuration for InjectCorrelationID
type roundTripperConfig struct {
getOperationName OperationNamer
}
// RoundTripperOption will configure a correlation handler
type RoundTripperOption func(*roundTripperConfig)
func applyRoundTripperOptions(opts []RoundTripperOption) roundTripperConfig {
config := roundTripperConfig{
getOperationName: func(req *http.Request) string {
// By default use `GET https://localhost` for operation names
return fmt.Sprintf("%s %s://%s", req.Method, req.URL.Scheme, req.URL.Host)
},
}
for _, v := range opts {
v(&config)
}
return config
}
Copyright (c) 2016, Datadog <info@datadoghq.com>
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of Datadog nor the
names of its contributors may be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL <COPYRIGHT HOLDER> BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Component,Origin,License,Copyright
import,io.opentracing,Apache-2.0,Copyright 2016-2017 The OpenTracing Authors
\ No newline at end of file
ignored = [
# From these libraries we should always ensure compatibility with the latest:
"github.com/opentracing/*",
"golang.org/x/*",
]
[[constraint]]
name = "github.com/tinylib/msgp"
revision = "3b5c87ab5fb00c660bf85b888445d9a01db64db4" # Feb 15, 2018
// Package ddtrace contains the interfaces that specify the implementations of Datadog's
// tracing library, as well as a set of sub-packages containing various implementations:
// our native implementation ("tracer"), a wrapper that can be used with Opentracing
// ("opentracer") and a mock tracer to be used for testing ("mocktracer"). Additionally,
// package "ext" provides a set of tag names and values specific to Datadog's APM product.
//
// To get started, visit the documentation for any of the packages you'd like to begin
// with by accessing the subdirectories of this package: https://godoc.org/gopkg.in/DataDog/dd-trace-go.v1/ddtrace#pkg-subdirectories.
package ddtrace // import "gopkg.in/DataDog/dd-trace-go.v1/ddtrace"
import "time"
// Tracer specifies an implementation of the Datadog tracer which allows starting
// and propagating spans. The official implementation if exposed as functions
// within the "tracer" package.
type Tracer interface {
// StartSpan starts a span with the given operation name and options.
StartSpan(operationName string, opts ...StartSpanOption) Span
// Extract extracts a span context from a given carrier. Note that baggage item
// keys will always be lower-cased to maintain consistency. It is impossible to
// maintain the original casing due to MIME header canonicalization standards.
Extract(carrier interface{}) (SpanContext, error)
// Inject injects a span context into the given carrier.
Inject(context SpanContext, carrier interface{}) error
// Stop stops the active tracer and sets the global tracer to a no-op. Calls to
// Stop should be idempotent.
Stop()
}
// Span represents a chunk of computation time. Spans have names, durations,
// timestamps and other metadata. A Tracer is used to create hierarchies of
// spans in a request, buffer and submit them to the server.
type Span interface {
// SetTag sets a key/value pair as metadata on the span.
SetTag(key string, value interface{})
// SetOperationName sets the operation name for this span. An operation name should be
// a representative name for a group of spans (e.g. "grpc.server" or "http.request").
SetOperationName(operationName string)
// BaggageItem returns the baggage item held by the given key.
BaggageItem(key string) string
// SetBaggageItem sets a new baggage item at the given key. The baggage
// item should propagate to all descendant spans, both in- and cross-process.
SetBaggageItem(key, val string)
// Finish finishes the current span with the given options. Finish calls should be idempotent.
Finish(opts ...FinishOption)
// Context returns the SpanContext of this Span.
Context() SpanContext
}
// SpanContext represents a span state that can propagate to descendant spans
// and across process boundaries. It contains all the information needed to
// spawn a direct descendant of the span that it belongs to. It can be used
// to create distributed tracing by propagating it using the provided interfaces.
type SpanContext interface {
// SpanID returns the span ID that this context is carrying.
SpanID() uint64
// TraceID returns the trace ID that this context is carrying.
TraceID() uint64
// ForeachBaggageItem provides an iterator over the key/value pairs set as
// baggage within this context. Iteration stops when the handler returns
// false.
ForeachBaggageItem(handler func(k, v string) bool)
}
// StartSpanOption is a configuration option that can be used with a Tracer's StartSpan method.
type StartSpanOption func(cfg *StartSpanConfig)
// FinishOption is a configuration option that can be used with a Span's Finish method.
type FinishOption func(cfg *FinishConfig)
// FinishConfig holds the configuration for finishing a span. It is usually passed around by
// reference to one or more FinishOption functions which shape it into its final form.
type FinishConfig struct {
// FinishTime represents the time that should be set as finishing time for the
// span. Implementations should use the current time when FinishTime.IsZero().
FinishTime time.Time
// Error holds an optional error that should be set on the span before
// finishing.
Error error
// NoDebugStack will prevent any set errors from generating an attached stack trace tag.
NoDebugStack bool
}
// StartSpanConfig holds the configuration for starting a new span. It is usually passed
// around by reference to one or more StartSpanOption functions which shape it into its
// final form.
type StartSpanConfig struct {
// Parent holds the SpanContext that should be used as a parent for the
// new span. If nil, implementations should return a root span.
Parent SpanContext
// StartTime holds the time that should be used as the start time of the span.
// Implementations should use the current time when StartTime.IsZero().
StartTime time.Time
// Tags holds a set of key/value pairs that should be set as metadata on the
// new span.
Tags map[string]interface{}
}
package ext // import "gopkg.in/DataDog/dd-trace-go.v1/ddtrace/ext"
// App types determine how to categorize a trace in the Datadog application.
// For more fine-grained behaviour, use the SpanType* constants.
const (
// DEPRECATED: Use SpanTypeWeb
// AppTypeWeb specifies the Web span type and can be used as a tag value
// for a span's SpanType tag.
AppTypeWeb = "web"
// AppTypeDB specifies the DB span type and can be used as a tag value
// for a span's SpanType tag. If possible, use one of the SpanType*
// constants for a more accurate indication.
AppTypeDB = "db"
// AppTypeCache specifies the Cache span type and can be used as a tag value
// for a span's SpanType tag. If possible, consider using SpanTypeRedis or
// SpanTypeMemcached.
AppTypeCache = "cache"
// AppTypeRPC specifies the RPC span type and can be used as a tag value
// for a span's SpanType tag.
AppTypeRPC = "rpc"
)
// Span types have similar behaviour to "app types" and help categorize
// traces in the Datadog application. They can also help fine grain agent
// level bahviours such as obfuscation and quantization, when these are
// enabled in the agent's configuration.
const (
// SpanTypeWeb marks a span as an HTTP server request.
SpanTypeWeb = "web"
// SpanTypeHTTP marks a span as an HTTP client request.
SpanTypeHTTP = "http"
// SpanTypeSQL marks a span as an SQL operation. These spans may
// have an "sql.command" tag.
SpanTypeSQL = "sql"
// SpanTypeCassandra marks a span as a Cassandra operation. These
// spans may have an "sql.command" tag.
SpanTypeCassandra = "cassandra"
// SpanTypeRedis marks a span as a Redis operation. These spans may
// also have a "redis.raw_command" tag.
SpanTypeRedis = "redis"
// SpanTypeMemcached marks a span as a memcached operation.
SpanTypeMemcached = "memcached"
// SpanTypeMongoDB marks a span as a MongoDB operation.
SpanTypeMongoDB = "mongodb"
// SpanTypeElasticSearch marks a span as an ElasticSearch operation.
// These spans may also have an "elasticsearch.body" tag.
SpanTypeElasticSearch = "elasticsearch"
// SpanTypeLevelDB marks a span as a leveldb operation
SpanTypeLevelDB = "leveldb"
// SpanTypeDNS marks a span as a DNS operation.
SpanTypeDNS = "dns"
// SpanTypeMessageConsumer marks a span as a queue operation
SpanTypeMessageConsumer = "queue"
// SpanTypeMessageProducer marks a span as a queue operation.
SpanTypeMessageProducer = "queue"
)
package ext
const (
// CassandraQuery is the tag name used for cassandra queries.
CassandraQuery = "cassandra.query"
// CassandraConsistencyLevel is the tag name to set for consitency level.
CassandraConsistencyLevel = "cassandra.consistency_level"
// CassandraCluster specifies the tag name that is used to set the cluster.
CassandraCluster = "cassandra.cluster"
// CassandraRowCount specifies the tag name to use when settings the row count.
CassandraRowCount = "cassandra.row_count"
// CassandraKeyspace is used as tag name for setting the key space.
CassandraKeyspace = "cassandra.keyspace"
// CassandraPaginated specifies the tag name for paginated queries.
CassandraPaginated = "cassandra.paginated"
)
package ext
const (
// DBApplication indicates the application using the database.
DBApplication = "db.application"
// DBName indicates the database name.
DBName = "db.name"
// DBType indicates the type of Database.
DBType = "db.type"
// DBInstance indicates the instance name of Database.
DBInstance = "db.instance"
// DBUser indicates the user name of Database, e.g. "readonly_user" or "reporting_user".
DBUser = "db.user"
// DBStatement records a database statement for the given database type.
DBStatement = "db.statement"
)
package ext
const (
// PeerHostIPV4 records IPv4 host address of the peer.
PeerHostIPV4 = "peer.ipv4"
// PeerHostIPV6 records the IPv6 host address of the peer.
PeerHostIPV6 = "peer.ipv6"
// PeerService records the service name of the peer service.
PeerService = "peer.service"
// PeerHostname records the host name of the peer.
PeerHostname = "peer.hostname"
// PeerPort records the port number of the peer.
PeerPort = "peer.port"
)
package ext
// Priority is a hint given to the backend so that it knows which traces to reject or kept.
// In a distributed context, it should be set before any context propagation (fork, RPC calls) to be effective.
const (
// PriorityUserReject informs the backend that a trace should be rejected and not stored.
// This should be used by user code overriding default priority.
PriorityUserReject = -1
// PriorityAutoReject informs the backend that a trace should be rejected and not stored.
// This is used by the builtin sampler.
PriorityAutoReject = 0
// PriorityAutoKeep informs the backend that a trace should be kept and not stored.
// This is used by the builtin sampler.
PriorityAutoKeep = 1
// PriorityUserKeep informs the backend that a trace should be kept and not stored.
// This should be used by user code overriding default priority.
PriorityUserKeep = 2
)
package ext
// Standard system metadata names
const (
// The pid of the traced process
Pid = "system.pid"
)
// Package ext contains a set of Datadog-specific constants. Most of them are used
// for setting span metadata.
package ext
const (
// TargetHost sets the target host address.
TargetHost = "out.host"
// TargetPort sets the target host port.
TargetPort = "out.port"
// SamplingPriority is the tag that marks the sampling priority of a span.
SamplingPriority = "sampling.priority"
// SQLType sets the sql type tag.
SQLType = "sql"
// SQLQuery sets the sql query tag on a span.
SQLQuery = "sql.query"
// HTTPMethod specifies the HTTP method used in a span.
HTTPMethod = "http.method"
// HTTPCode sets the HTTP status code as a tag.
HTTPCode = "http.status_code"
// HTTPURL sets the HTTP URL for a span.
HTTPURL = "http.url"
// TODO: In the next major version, prefix these constants (SpanType, etc)
// with "Key*" (KeySpanType, etc) to more easily differentiate between
// constants representing tag values and constants representing keys.
// SpanType defines the Span type (web, db, cache).
SpanType = "span.type"
// ServiceName defines the Service name for this Span.
ServiceName = "service.name"
// ResourceName defines the Resource name for the Span.
ResourceName = "resource.name"
// Error specifies the error tag. It's value is usually of type "error".
Error = "error"
// ErrorMsg specifies the error message.
ErrorMsg = "error.msg"
// ErrorType specifies the error type.
ErrorType = "error.type"
// ErrorStack specifies the stack dump.
ErrorStack = "error.stack"
// Environment specifies the environment to use with a trace.
Environment = "env"
// EventSampleRate specifies the rate at which this span will be sampled
// as an APM event.
EventSampleRate = "_dd1.sr.eausr"
)
package internal // import "gopkg.in/DataDog/dd-trace-go.v1/ddtrace/internal"
import (
"sync"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace"
)
var (
mu sync.RWMutex // guards globalTracer
globalTracer ddtrace.Tracer = &NoopTracer{}
)
// SetGlobalTracer sets the global tracer to t.
func SetGlobalTracer(t ddtrace.Tracer) {
mu.Lock()
defer mu.Unlock()
if !Testing {
// avoid infinite loop when calling (*mocktracer.Tracer).Stop
globalTracer.Stop()
}
globalTracer = t
}
// GetGlobalTracer returns the currently active tracer.
func GetGlobalTracer() ddtrace.Tracer {
mu.RLock()
defer mu.RUnlock()
return globalTracer
}
// Testing is set to true when the mock tracer is active. It usually signifies that we are in a test
// environment. This value is used by tracer.Start to prevent overriding the GlobalTracer in tests.
var Testing = false
var _ ddtrace.Tracer = (*NoopTracer)(nil)
// NoopTracer is an implementation of ddtrace.Tracer that is a no-op.
type NoopTracer struct{}
// StartSpan implements ddtrace.Tracer.
func (NoopTracer) StartSpan(operationName string, opts ...ddtrace.StartSpanOption) ddtrace.Span {
return NoopSpan{}
}
// SetServiceInfo implements ddtrace.Tracer.
func (NoopTracer) SetServiceInfo(name, app, appType string) {}
// Extract implements ddtrace.Tracer.
func (NoopTracer) Extract(carrier interface{}) (ddtrace.SpanContext, error) {
return NoopSpanContext{}, nil
}
// Inject implements ddtrace.Tracer.
func (NoopTracer) Inject(context ddtrace.SpanContext, carrier interface{}) error { return nil }
// Stop implements ddtrace.Tracer.
func (NoopTracer) Stop() {}
var _ ddtrace.Span = (*NoopSpan)(nil)
// NoopSpan is an implementation of ddtrace.Span that is a no-op.
type NoopSpan struct{}
// SetTag implements ddtrace.Span.
func (NoopSpan) SetTag(key string, value interface{}) {}
// SetOperationName implements ddtrace.Span.
func (NoopSpan) SetOperationName(operationName string) {}
// BaggageItem implements ddtrace.Span.
func (NoopSpan) BaggageItem(key string) string { return "" }
// SetBaggageItem implements ddtrace.Span.
func (NoopSpan) SetBaggageItem(key, val string) {}
// Finish implements ddtrace.Span.
func (NoopSpan) Finish(opts ...ddtrace.FinishOption) {}
// Tracer implements ddtrace.Span.
func (NoopSpan) Tracer() ddtrace.Tracer { return NoopTracer{} }
// Context implements ddtrace.Span.
func (NoopSpan) Context() ddtrace.SpanContext { return NoopSpanContext{} }
var _ ddtrace.SpanContext = (*NoopSpanContext)(nil)
// NoopSpanContext is an implementation of ddtrace.SpanContext that is a no-op.
type NoopSpanContext struct{}
// SpanID implements ddtrace.SpanContext.
func (NoopSpanContext) SpanID() uint64 { return 0 }
// TraceID implements ddtrace.SpanContext.
func (NoopSpanContext) TraceID() uint64 { return 0 }
// ForeachBaggageItem implements ddtrace.SpanContext.
func (NoopSpanContext) ForeachBaggageItem(handler func(k, v string) bool) {}
package opentracer // import "gopkg.in/DataDog/dd-trace-go.v1/ddtrace/opentracer"
import (
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace/ext"
opentracing "github.com/opentracing/opentracing-go"
)
// ServiceName can be used with opentracing.StartSpan to set the
// service name of a span.
func ServiceName(name string) opentracing.StartSpanOption {
return opentracing.Tag{Key: ext.ServiceName, Value: name}
}
// ResourceName can be used with opentracing.StartSpan to set the
// resource name of a span.
func ResourceName(name string) opentracing.StartSpanOption {
return opentracing.Tag{Key: ext.ResourceName, Value: name}
}
// SpanType can be used with opentracing.StartSpan to set the type of a span.
func SpanType(name string) opentracing.StartSpanOption {
return opentracing.Tag{Key: ext.SpanType, Value: name}
}
package opentracer
import (
"fmt"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace/ext"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer"
opentracing "github.com/opentracing/opentracing-go"
"github.com/opentracing/opentracing-go/log"
)
var _ opentracing.Span = (*span)(nil)
// span implements opentracing.Span on top of ddtrace.Span.
type span struct {
ddtrace.Span
*opentracer
}
func (s *span) Context() opentracing.SpanContext { return s.Span.Context() }
func (s *span) Finish() { s.Span.Finish() }
func (s *span) Tracer() opentracing.Tracer { return s.opentracer }
func (s *span) LogEvent(event string) { /* deprecated */ }
func (s *span) LogEventWithPayload(event string, payload interface{}) { /* deprecated */ }
func (s *span) Log(data opentracing.LogData) { /* deprecated */ }
func (s *span) FinishWithOptions(opts opentracing.FinishOptions) {
for _, lr := range opts.LogRecords {
if len(lr.Fields) > 0 {
s.LogFields(lr.Fields...)
}
}
s.Span.Finish(tracer.FinishTime(opts.FinishTime))
}
func (s *span) LogFields(fields ...log.Field) {
// catch standard opentracing keys and adjust to internal ones as per spec:
// https://github.com/opentracing/specification/blob/master/semantic_conventions.md#log-fields-table
for _, f := range fields {
switch f.Key() {
case "event":
if v, ok := f.Value().(string); ok && v == "error" {
s.SetTag("error", true)
}
case "error", "error.object":
if err, ok := f.Value().(error); ok {
s.SetTag("error", err)
}
case "message":
s.SetTag(ext.ErrorMsg, fmt.Sprint(f.Value()))
case "stack":
s.SetTag(ext.ErrorStack, fmt.Sprint(f.Value()))
default:
// not implemented
}
}
}
func (s *span) LogKV(keyVals ...interface{}) {
fields, err := log.InterleavedKVToFields(keyVals...)
if err != nil {
// TODO(gbbr): create a log package
return
}
s.LogFields(fields...)
}
func (s *span) SetBaggageItem(key, val string) opentracing.Span {
s.Span.SetBaggageItem(key, val)
return s
}
func (s *span) SetOperationName(operationName string) opentracing.Span {
s.Span.SetOperationName(operationName)
return s
}
func (s *span) SetTag(key string, value interface{}) opentracing.Span {
s.Span.SetTag(key, value)
return s
}
// Package opentracer provides a wrapper on top of the Datadog tracer that can be used with Opentracing.
// It also provides a set of opentracing.StartSpanOption that are specific to Datadog's APM product.
// To use it, simply call "New".
//
// Note that there are currently some small incompatibilities between the Opentracing spec and the Datadog
// APM product, which we are in the process of addressing on the long term. When using Datadog, the
// Opentracing operation name is what is called resource in Datadog's terms and the Opentracing "component"
// tag is Datadog's operation name. Meaning that in order to define (in Opentracing terms) a span that
// has the operation name "/user/profile" and the component "http.request", one would do:
// opentracing.StartSpan("http.request", opentracer.ResourceName("/user/profile"))
//
// Some libraries and frameworks are supported out-of-the-box by using our integrations. You can see a list
// of supported integrations here: https://godoc.org/gopkg.in/DataDog/dd-trace-go.v1/contrib. They are fully
// compatible with the Opentracing implementation.
package opentracer
import (
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace/internal"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer"
opentracing "github.com/opentracing/opentracing-go"
)
// New creates, instantiates and returns an Opentracing compatible version of the
// Datadog tracer using the provided set of options.
func New(opts ...tracer.StartOption) opentracing.Tracer {
tracer.Start(opts...)
return &opentracer{internal.GetGlobalTracer()}
}
var _ opentracing.Tracer = (*opentracer)(nil)
// opentracer implements opentracing.Tracer on top of ddtrace.Tracer.
type opentracer struct{ ddtrace.Tracer }
// StartSpan implements opentracing.Tracer.
func (t *opentracer) StartSpan(operationName string, options ...opentracing.StartSpanOption) opentracing.Span {
var sso opentracing.StartSpanOptions
for _, o := range options {
o.Apply(&sso)
}
opts := []ddtrace.StartSpanOption{tracer.StartTime(sso.StartTime)}
for _, ref := range sso.References {
if v, ok := ref.ReferencedContext.(ddtrace.SpanContext); ok && ref.Type == opentracing.ChildOfRef {
opts = append(opts, tracer.ChildOf(v))
break // can only have one parent
}
}
for k, v := range sso.Tags {
opts = append(opts, tracer.Tag(k, v))
}
return &span{
Span: t.Tracer.StartSpan(operationName, opts...),
opentracer: t,
}
}
// Inject implements opentracing.Tracer.
func (t *opentracer) Inject(ctx opentracing.SpanContext, format interface{}, carrier interface{}) error {
sctx, ok := ctx.(ddtrace.SpanContext)
if !ok {
return opentracing.ErrUnsupportedFormat
}
switch format {
case opentracing.TextMap, opentracing.HTTPHeaders:
return t.Tracer.Inject(sctx, carrier)
default:
return opentracing.ErrUnsupportedFormat
}
}
// Extract implements opentracing.Tracer.
func (t *opentracer) Extract(format interface{}, carrier interface{}) (opentracing.SpanContext, error) {
switch format {
case opentracing.TextMap, opentracing.HTTPHeaders:
return t.Tracer.Extract(carrier)
default:
return nil, opentracing.ErrUnsupportedFormat
}
}
package tracer
import (
"context"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace/internal"
)
type contextKey struct{}
var activeSpanKey = contextKey{}
// ContextWithSpan returns a copy of the given context which includes the span s.
func ContextWithSpan(ctx context.Context, s Span) context.Context {
return context.WithValue(ctx, activeSpanKey, s)
}
// SpanFromContext returns the span contained in the given context. A second return
// value indicates if a span was found in the context. If no span is found, a no-op
// span is returned.
func SpanFromContext(ctx context.Context) (Span, bool) {
if ctx == nil {
return &internal.NoopSpan{}, false
}
v := ctx.Value(activeSpanKey)
if s, ok := v.(ddtrace.Span); ok {
return s, true
}
return &internal.NoopSpan{}, false
}
// StartSpanFromContext returns a new span with the given operation name and options. If a span
// is found in the context, it will be used as the parent of the resulting span. If the ChildOf
// option is passed, the span from context will take precedence over it as the parent span.
func StartSpanFromContext(ctx context.Context, operationName string, opts ...StartSpanOption) (Span, context.Context) {
if s, ok := SpanFromContext(ctx); ok {
opts = append(opts, ChildOf(s.Context()))
}
s := StartSpan(operationName, opts...)
return s, ContextWithSpan(ctx, s)
}
// Package tracer contains Datadog's core tracing client. It is used to trace
// requests as they flow across web servers, databases and microservices, giving
// developers visibility into bottlenecks and troublesome requests. To start the
// tracer, simply call the start method along with an optional set of options.
// By default, the trace agent is considered to be found at "localhost:8126". In a
// setup where this would be different (let's say 127.0.0.1:1234), we could do:
// tracer.Start(tracer.WithAgentAddr("127.0.0.1:1234"))
// defer tracer.Stop()
//
// The tracing client can perform trace sampling. While the trace agent
// already samples traces to reduce bandwidth usage, client sampling reduces
// performance overhead. To make use of it, the package comes with a ready-to-use
// rate sampler that can be passed to the tracer. To use it and keep only 30% of the
// requests, one would do:
// s := tracer.NewRateSampler(0.3)
// tracer.Start(tracer.WithSampler(s))
//
// All spans created by the tracer contain a context hereby referred to as the span
// context. Note that this is different from Go's context. The span context is used
// to package essential information from a span, which is needed when creating child
// spans that inherit from it. Thus, a child span is created from a span's span context.
// The span context can originate from within the same process, but also a
// different process or even a different machine in the case of distributed tracing.
//
// To make use of distributed tracing, a span's context may be injected via a carrier
// into a transport (HTTP, RPC, etc.) to be extracted on the other end and used to
// create spans that are direct descendants of it. A couple of carrier interfaces
// which should cover most of the use-case scenarios are readily provided, such as
// HTTPCarrier and TextMapCarrier. Users are free to create their own, which will work
// with our propagation algorithm as long as they implement the TextMapReader and TextMapWriter
// interfaces. An example alternate implementation is the MDCarrier in our gRPC integration.
//
// As an example, injecting a span's context into an HTTP request would look like this:
// req, err := http.NewRequest("GET", "http://example.com", nil)
// // ...
// err := tracer.Inject(span.Context(), tracer.HTTPHeadersCarrier(req.Header))
// // ...
// http.DefaultClient.Do(req)
// Then, on the server side, to continue the trace one would do:
// sctx, err := tracer.Extract(tracer.HTTPHeadersCarrier(req.Header))
// // ...
// span := tracer.StartSpan("child.span", tracer.ChildOf(sctx))
// In the same manner, any means can be used as a carrier to inject a context into a transport. Go's
// context can also be used as a means to transport spans within the same process. The methods
// StartSpanFromContext, ContextWithSpan and SpanFromContext exist for this reason.
//
// Some libraries and frameworks are supported out-of-the-box by using one
// of our integrations. You can see a list of supported integrations here:
// https://godoc.org/gopkg.in/DataDog/dd-trace-go.v1/contrib
package tracer // import "gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer"
package tracer
import (
"fmt"
"log"
"strconv"
)
var errorPrefix = fmt.Sprintf("Datadog Tracer Error (%s): ", tracerVersion)
type traceEncodingError struct{ context error }
func (e *traceEncodingError) Error() string {
return fmt.Sprintf("error encoding trace: %s", e.context)
}
type spanBufferFullError struct{}
func (e *spanBufferFullError) Error() string {
return fmt.Sprintf("trace span cap (%d) reached, dropping trace", traceMaxSize)
}
type dataLossError struct {
count int // number of items lost
context error // any context error, if available
}
func (e *dataLossError) Error() string {
return fmt.Sprintf("lost traces (count: %d), error: %v", e.count, e.context)
}
type errorSummary struct {
Count int
Example string
}
func aggregateErrors(errChan <-chan error) map[string]errorSummary {
errs := make(map[string]errorSummary, len(errChan))
for {
select {
case err := <-errChan:
if err == nil {
break
}
key := fmt.Sprintf("%T", err)
summary := errs[key]
summary.Count++
summary.Example = err.Error()
errs[key] = summary
default: // stop when there's no more data
return errs
}
}
}
// logErrors logs the errors, preventing log file flooding, when there
// are many messages, it caps them and shows a quick summary.
// As of today it only logs using standard golang log package, but
// later we could send those stats to agent // TODO(ufoot).
func logErrors(errChan <-chan error) {
errs := aggregateErrors(errChan)
for _, v := range errs {
var repeat string
if v.Count > 1 {
repeat = " (repeated " + strconv.Itoa(v.Count) + " times)"
}
log.Println(errorPrefix + v.Example + repeat)
}
}
package tracer
import (
"net/http"
"os"
"path/filepath"
"time"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace/ext"
)
// config holds the tracer configuration.
type config struct {
// debug, when true, writes details to logs.
debug bool
// serviceName specifies the name of this application.
serviceName string
// sampler specifies the sampler that will be used for sampling traces.
sampler Sampler
// agentAddr specifies the hostname and of the agent where the traces
// are sent to.
agentAddr string
// globalTags holds a set of tags that will be automatically applied to
// all spans.
globalTags map[string]interface{}
// transport specifies the Transport interface which will be used to send data to the agent.
transport transport
// propagator propagates span context cross-process
propagator Propagator
// httpRoundTripper defines the http.RoundTripper used by the agent transport.
httpRoundTripper http.RoundTripper
}
// StartOption represents a function that can be provided as a parameter to Start.
type StartOption func(*config)
// defaults sets the default values for a config.
func defaults(c *config) {
c.serviceName = filepath.Base(os.Args[0])
c.sampler = NewAllSampler()
c.agentAddr = defaultAddress
}
// WithPrioritySampling is deprecated, and priority sampling is enabled by default.
// When using distributed tracing, the priority sampling value is propagated in order to
// get all the parts of a distributed trace sampled.
// To learn more about priority sampling, please visit:
// https://docs.datadoghq.com/tracing/getting_further/trace_sampling_and_storage/#priority-sampling-for-distributed-tracing
func WithPrioritySampling() StartOption {
return func(c *config) {
// This is now enabled by default.
}
}
// WithDebugMode enables debug mode on the tracer, resulting in more verbose logging.
func WithDebugMode(enabled bool) StartOption {
return func(c *config) {
c.debug = enabled
}
}
// WithPropagator sets an alternative propagator to be used by the tracer.
func WithPropagator(p Propagator) StartOption {
return func(c *config) {
c.propagator = p
}
}
// WithServiceName sets the default service name to be used with the tracer.
func WithServiceName(name string) StartOption {
return func(c *config) {
c.serviceName = name
}
}
// WithAgentAddr sets the address where the agent is located. The default is
// localhost:8126. It should contain both host and port.
func WithAgentAddr(addr string) StartOption {
return func(c *config) {
c.agentAddr = addr
}
}
// WithGlobalTag sets a key/value pair which will be set as a tag on all spans
// created by tracer. This option may be used multiple times.
func WithGlobalTag(k string, v interface{}) StartOption {
return func(c *config) {
if c.globalTags == nil {
c.globalTags = make(map[string]interface{})
}
c.globalTags[k] = v
}
}
// WithSampler sets the given sampler to be used with the tracer. By default
// an all-permissive sampler is used.
func WithSampler(s Sampler) StartOption {
return func(c *config) {
c.sampler = s
}
}
// WithHTTPRoundTripper allows customizing the underlying HTTP transport for
// emitting spans. This is useful for advanced customization such as emitting
// spans to a unix domain socket. The default should be used in most cases.
func WithHTTPRoundTripper(r http.RoundTripper) StartOption {
return func(c *config) {
c.httpRoundTripper = r
}
}
// StartSpanOption is a configuration option for StartSpan. It is aliased in order
// to help godoc group all the functions returning it together. It is considered
// more correct to refer to it as the type as the origin, ddtrace.StartSpanOption.
type StartSpanOption = ddtrace.StartSpanOption
// Tag sets the given key/value pair as a tag on the started Span.
func Tag(k string, v interface{}) StartSpanOption {
return func(cfg *ddtrace.StartSpanConfig) {
if cfg.Tags == nil {
cfg.Tags = map[string]interface{}{}
}
cfg.Tags[k] = v
}
}
// ServiceName sets the given service name on the started span. For example "http.server".
func ServiceName(name string) StartSpanOption {
return Tag(ext.ServiceName, name)
}
// ResourceName sets the given resource name on the started span. A resource could
// be an SQL query, a URL, an RPC method or something else.
func ResourceName(name string) StartSpanOption {
return Tag(ext.ResourceName, name)
}
// SpanType sets the given span type on the started span. Some examples in the case of
// the Datadog APM product could be "web", "db" or "cache".
func SpanType(name string) StartSpanOption {
return Tag(ext.SpanType, name)
}
// ChildOf tells StartSpan to use the given span context as a parent for the
// created span.
func ChildOf(ctx ddtrace.SpanContext) StartSpanOption {
return func(cfg *ddtrace.StartSpanConfig) {
cfg.Parent = ctx
}
}
// StartTime sets a custom time as the start time for the created span. By
// default a span is started using the creation time.
func StartTime(t time.Time) StartSpanOption {
return func(cfg *ddtrace.StartSpanConfig) {
cfg.StartTime = t
}
}
// FinishOption is a configuration option for FinishSpan. It is aliased in order
// to help godoc group all the functions returning it together. It is considered
// more correct to refer to it as the type as the origin, ddtrace.FinishOption.
type FinishOption = ddtrace.FinishOption
// FinishTime sets the given time as the finishing time for the span. By default,
// the current time is used.
func FinishTime(t time.Time) FinishOption {
return func(cfg *ddtrace.FinishConfig) {
cfg.FinishTime = t
}
}
// WithError marks the span as having had an error. It uses the information from
// err to set tags such as the error message, error type and stack trace.
func WithError(err error) FinishOption {
return func(cfg *ddtrace.FinishConfig) {
cfg.Error = err
}
}
// NoDebugStack prevents any error presented using the WithError finishing option
// from generating a stack trace. This is useful in situations where errors are frequent
// and performance is critical.
func NoDebugStack() FinishOption {
return func(cfg *ddtrace.FinishConfig) {
cfg.NoDebugStack = true
}
}
package tracer
import (
"bytes"
"encoding/binary"
"io"
"github.com/tinylib/msgp/msgp"
)
// payload is a wrapper on top of the msgpack encoder which allows constructing an
// encoded array by pushing its entries sequentially, one at a time. It basically
// allows us to encode as we would with a stream, except that the contents of the stream
// can be read as a slice by the msgpack decoder at any time. It follows the guidelines
// from the msgpack array spec:
// https://github.com/msgpack/msgpack/blob/master/spec.md#array-format-family
//
// payload implements io.Reader and can be used with the decoder directly. To create
// a new payload use the newPayload method.
//
// payload is not safe for concurrent use.
//
// This structure basically allows us to push traces into the payload one at a time
// in order to always have knowledge of the payload size, but also making it possible
// for the agent to decode it as an array.
type payload struct {
// header specifies the first few bytes in the msgpack stream
// indicating the type of array (fixarray, array16 or array32)
// and the number of items contained in the stream.
header []byte
// off specifies the current read position on the header.
off int
// count specifies the number of items in the stream.
count uint64
// buf holds the sequence of msgpack-encoded items.
buf bytes.Buffer
}
var _ io.Reader = (*payload)(nil)
// newPayload returns a ready to use payload.
func newPayload() *payload {
p := &payload{
header: make([]byte, 8),
off: 8,
}
return p
}
// push pushes a new item into the stream.
func (p *payload) push(t spanList) error {
if err := msgp.Encode(&p.buf, t); err != nil {
return err
}
p.count++
p.updateHeader()
return nil
}
// itemCount returns the number of items available in the srteam.
func (p *payload) itemCount() int {
return int(p.count)
}
// size returns the payload size in bytes. After the first read the value becomes
// inaccurate by up to 8 bytes.
func (p *payload) size() int {
return p.buf.Len() + len(p.header) - p.off
}
// reset resets the internal buffer, counter and read offset.
func (p *payload) reset() {
p.off = 8
p.count = 0
p.buf.Reset()
}
// https://github.com/msgpack/msgpack/blob/master/spec.md#array-format-family
const (
msgpackArrayFix byte = 144 // up to 15 items
msgpackArray16 = 0xdc // up to 2^16-1 items, followed by size in 2 bytes
msgpackArray32 = 0xdd // up to 2^32-1 items, followed by size in 4 bytes
)
// updateHeader updates the payload header based on the number of items currently
// present in the stream.
func (p *payload) updateHeader() {
n := p.count
switch {
case n <= 15:
p.header[7] = msgpackArrayFix + byte(n)
p.off = 7
case n <= 1<<16-1:
binary.BigEndian.PutUint64(p.header, n) // writes 2 bytes
p.header[5] = msgpackArray16
p.off = 5
default: // n <= 1<<32-1
binary.BigEndian.PutUint64(p.header, n) // writes 4 bytes
p.header[3] = msgpackArray32
p.off = 3
}
}
// Read implements io.Reader. It reads from the msgpack-encoded stream.
func (p *payload) Read(b []byte) (n int, err error) {
if p.off < len(p.header) {
// reading header
n = copy(b, p.header[p.off:])
p.off += n
return n, nil
}
return p.buf.Read(b)
}
package tracer
import (
"errors"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace"
)
// Propagator implementations should be able to inject and extract
// SpanContexts into an implementation specific carrier.
type Propagator interface {
// Inject takes the SpanContext and injects it into the carrier.
Inject(context ddtrace.SpanContext, carrier interface{}) error
// Extract returns the SpanContext from the given carrier.
Extract(carrier interface{}) (ddtrace.SpanContext, error)
}
// TextMapWriter allows setting key/value pairs of strings on the underlying
// data structure. Carriers implementing TextMapWriter are compatible to be
// used with Datadog's TextMapPropagator.
type TextMapWriter interface {
// Set sets the given key/value pair.
Set(key, val string)
}
// TextMapReader allows iterating over sets of key/value pairs. Carriers implementing
// TextMapReader are compatible to be used with Datadog's TextMapPropagator.
type TextMapReader interface {
// ForeachKey iterates over all keys that exist in the underlying
// carrier. It takes a callback function which will be called
// using all key/value pairs as arguments. ForeachKey will return
// the first error returned by the handler.
ForeachKey(handler func(key, val string) error) error
}
var (
// ErrInvalidCarrier is returned when the carrier provided to the propagator
// does not implemented the correct interfaces.
ErrInvalidCarrier = errors.New("invalid carrier")
// ErrInvalidSpanContext is returned when the span context found in the
// carrier is not of the expected type.
ErrInvalidSpanContext = errors.New("invalid span context")
// ErrSpanContextCorrupted is returned when there was a problem parsing
// the information found in the carrier.
ErrSpanContextCorrupted = errors.New("span context corrupted")
// ErrSpanContextNotFound represents missing information in the given carrier.
ErrSpanContextNotFound = errors.New("span context not found")
)
package tracer
import (
cryptorand "crypto/rand"
"log"
"math"
"math/big"
"math/rand"
"sync"
"time"
)
// random holds a thread-safe source of random numbers.
var random *rand.Rand
func init() {
var seed int64
n, err := cryptorand.Int(cryptorand.Reader, big.NewInt(math.MaxInt64))
if err == nil {
seed = n.Int64()
} else {
log.Printf("%scannot generate random seed: %v; using current time\n", errorPrefix, err)
seed = time.Now().UnixNano()
}
random = rand.New(&safeSource{
source: rand.NewSource(seed),
})
}
// safeSource holds a thread-safe implementation of rand.Source64.
type safeSource struct {
source rand.Source
sync.Mutex
}
func (rs *safeSource) Int63() int64 {
rs.Lock()
n := rs.source.Int63()
rs.Unlock()
return n
}
func (rs *safeSource) Uint64() uint64 { return uint64(rs.Int63()) }
func (rs *safeSource) Seed(seed int64) {
rs.Lock()
rs.Seed(seed)
rs.Unlock()
}
package tracer
import (
"encoding/json"
"io"
"math"
"sync"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace/ext"
)
// Sampler is the generic interface of any sampler. It must be safe for concurrent use.
type Sampler interface {
// Sample returns true if the given span should be sampled.
Sample(span Span) bool
}
// RateSampler is a sampler implementation which randomly selects spans using a
// provided rate. For example, a rate of 0.75 will permit 75% of the spans.
// RateSampler implementations should be safe for concurrent use.
type RateSampler interface {
Sampler
// Rate returns the current sample rate.
Rate() float64
// SetRate sets a new sample rate.
SetRate(rate float64)
}
// rateSampler samples from a sample rate.
type rateSampler struct {
sync.RWMutex
rate float64
}
// NewAllSampler is a short-hand for NewRateSampler(1). It is all-permissive.
func NewAllSampler() RateSampler { return NewRateSampler(1) }
// NewRateSampler returns an initialized RateSampler with a given sample rate.
func NewRateSampler(rate float64) RateSampler {
return &rateSampler{rate: rate}
}
// Rate returns the current rate of the sampler.
func (r *rateSampler) Rate() float64 {
r.RLock()
defer r.RUnlock()
return r.rate
}
// SetRate sets a new sampling rate.
func (r *rateSampler) SetRate(rate float64) {
r.Lock()
r.rate = rate
r.Unlock()
}
// constants used for the Knuth hashing, same as agent.
const knuthFactor = uint64(1111111111111111111)
// Sample returns true if the given span should be sampled.
func (r *rateSampler) Sample(spn ddtrace.Span) bool {
if r.rate == 1 {
// fast path
return true
}
s, ok := spn.(*span)
if !ok {
return false
}
r.RLock()
defer r.RUnlock()
return sampledByRate(s.TraceID, r.rate)
}
// sampledByRate verifies if the number n should be sampled at the specified
// rate.
func sampledByRate(n uint64, rate float64) bool {
if rate < 1 {
return n*knuthFactor < uint64(rate*math.MaxUint64)
}
return true
}
// prioritySampler holds a set of per-service sampling rates and applies
// them to spans.
type prioritySampler struct {
mu sync.RWMutex
rates map[string]float64
defaultRate float64
}
func newPrioritySampler() *prioritySampler {
return &prioritySampler{
rates: make(map[string]float64),
defaultRate: 1.,
}
}
// readRatesJSON will try to read the rates as JSON from the given io.ReadCloser.
func (ps *prioritySampler) readRatesJSON(rc io.ReadCloser) error {
var payload struct {
Rates map[string]float64 `json:"rate_by_service"`
}
if err := json.NewDecoder(rc).Decode(&payload); err != nil {
return err
}
rc.Close()
const defaultRateKey = "service:,env:"
ps.mu.Lock()
defer ps.mu.Unlock()
ps.rates = payload.Rates
if v, ok := ps.rates[defaultRateKey]; ok {
ps.defaultRate = v
delete(ps.rates, defaultRateKey)
}
return nil
}
// getRate returns the sampling rate to be used for the given span. Callers must
// guard the span.
func (ps *prioritySampler) getRate(spn *span) float64 {
key := "service:" + spn.Service + ",env:" + spn.Meta[ext.Environment]
ps.mu.RLock()
defer ps.mu.RUnlock()
if rate, ok := ps.rates[key]; ok {
return rate
}
return ps.defaultRate
}
// apply applies sampling priority to the given span. Caller must ensure it is safe
// to modify the span.
func (ps *prioritySampler) apply(spn *span) {
rate := ps.getRate(spn)
if sampledByRate(spn.TraceID, rate) {
spn.SetTag(ext.SamplingPriority, ext.PriorityAutoKeep)
} else {
spn.SetTag(ext.SamplingPriority, ext.PriorityAutoReject)
}
spn.SetTag(samplingPriorityRateKey, rate)
}
//go:generate msgp -unexported -marshal=false -o=span_msgp.go -tests=false
package tracer
import (
"fmt"
"reflect"
"runtime/debug"
"strings"
"sync"
"time"
"github.com/tinylib/msgp/msgp"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace/ext"
)
type (
// spanList implements msgp.Encodable on top of a slice of spans.
spanList []*span
// spanLists implements msgp.Decodable on top of a slice of spanList.
// This type is only used in tests.
spanLists []spanList
)
var (
_ ddtrace.Span = (*span)(nil)
_ msgp.Encodable = (*spanList)(nil)
_ msgp.Decodable = (*spanLists)(nil)
)
// span represents a computation. Callers must call Finish when a span is
// complete to ensure it's submitted.
type span struct {
sync.RWMutex `msg:"-"`
Name string `msg:"name"` // operation name
Service string `msg:"service"` // service name (i.e. "grpc.server", "http.request")
Resource string `msg:"resource"` // resource name (i.e. "/user?id=123", "SELECT * FROM users")
Type string `msg:"type"` // protocol associated with the span (i.e. "web", "db", "cache")
Start int64 `msg:"start"` // span start time expressed in nanoseconds since epoch
Duration int64 `msg:"duration"` // duration of the span expressed in nanoseconds
Meta map[string]string `msg:"meta,omitempty"` // arbitrary map of metadata
Metrics map[string]float64 `msg:"metrics,omitempty"` // arbitrary map of numeric metrics
SpanID uint64 `msg:"span_id"` // identifier of this span
TraceID uint64 `msg:"trace_id"` // identifier of the root span
ParentID uint64 `msg:"parent_id"` // identifier of the span's direct parent
Error int32 `msg:"error"` // error status of the span; 0 means no errors
finished bool `msg:"-"` // true if the span has been submitted to a tracer.
context *spanContext `msg:"-"` // span propagation context
}
// Context yields the SpanContext for this Span. Note that the return
// value of Context() is still valid after a call to Finish(). This is
// called the span context and it is different from Go's context.
func (s *span) Context() ddtrace.SpanContext { return s.context }
// SetBaggageItem sets a key/value pair as baggage on the span. Baggage items
// are propagated down to descendant spans and injected cross-process. Use with
// care as it adds extra load onto your tracing layer.
func (s *span) SetBaggageItem(key, val string) {
s.context.setBaggageItem(key, val)
}
// BaggageItem gets the value for a baggage item given its key. Returns the
// empty string if the value isn't found in this Span.
func (s *span) BaggageItem(key string) string {
return s.context.baggageItem(key)
}
// SetTag adds a set of key/value metadata to the span.
func (s *span) SetTag(key string, value interface{}) {
s.Lock()
defer s.Unlock()
// We don't lock spans when flushing, so we could have a data race when
// modifying a span as it's being flushed. This protects us against that
// race, since spans are marked `finished` before we flush them.
if s.finished {
return
}
if key == ext.Error {
s.setTagError(value, true)
return
}
if v, ok := value.(string); ok {
s.setTagString(key, v)
return
}
if v, ok := toFloat64(value); ok {
s.setTagNumeric(key, v)
return
}
// not numeric, not a string and not an error, the likelihood of this
// happening is close to zero, but we should nevertheless account for it.
s.Meta[key] = fmt.Sprint(value)
}
// setTagError sets the error tag. It accounts for various valid scenarios.
// This method is not safe for concurrent use.
func (s *span) setTagError(value interface{}, debugStack bool) {
if s.finished {
return
}
switch v := value.(type) {
case bool:
// bool value as per Opentracing spec.
if !v {
s.Error = 0
} else {
s.Error = 1
}
case error:
// if anyone sets an error value as the tag, be nice here
// and provide all the benefits.
s.Error = 1
s.Meta[ext.ErrorMsg] = v.Error()
s.Meta[ext.ErrorType] = reflect.TypeOf(v).String()
if debugStack {
s.Meta[ext.ErrorStack] = string(debug.Stack())
}
case nil:
// no error
s.Error = 0
default:
// in all other cases, let's assume that setting this tag
// is the result of an error.
s.Error = 1
}
}
// setTagString sets a string tag. This method is not safe for concurrent use.
func (s *span) setTagString(key, v string) {
switch key {
case ext.ServiceName:
s.Service = v
case ext.ResourceName:
s.Resource = v
case ext.SpanType:
s.Type = v
default:
s.Meta[key] = v
}
}
// setTagNumeric sets a numeric tag, in our case called a metric. This method
// is not safe for concurrent use.
func (s *span) setTagNumeric(key string, v float64) {
switch key {
case ext.SamplingPriority:
// setting sampling priority per spec
s.Metrics[samplingPriorityKey] = v
s.context.setSamplingPriority(int(v))
default:
s.Metrics[key] = v
}
}
// Finish closes this Span (but not its children) providing the duration
// of its part of the tracing session.
func (s *span) Finish(opts ...ddtrace.FinishOption) {
var cfg ddtrace.FinishConfig
for _, fn := range opts {
fn(&cfg)
}
var t int64
if cfg.FinishTime.IsZero() {
t = now()
} else {
t = cfg.FinishTime.UnixNano()
}
if cfg.Error != nil {
s.Lock()
s.setTagError(cfg.Error, !cfg.NoDebugStack)
s.Unlock()
}
s.finish(t)
}
// SetOperationName sets or changes the operation name.
func (s *span) SetOperationName(operationName string) {
s.Lock()
defer s.Unlock()
s.Name = operationName
}
func (s *span) finish(finishTime int64) {
s.Lock()
defer s.Unlock()
// We don't lock spans when flushing, so we could have a data race when
// modifying a span as it's being flushed. This protects us against that
// race, since spans are marked `finished` before we flush them.
if s.finished {
// already finished
return
}
if s.Duration == 0 {
s.Duration = finishTime - s.Start
}
s.finished = true
if s.context.drop {
// not sampled by local sampler
return
}
s.context.finish()
}
// String returns a human readable representation of the span. Not for
// production, just debugging.
func (s *span) String() string {
lines := []string{
fmt.Sprintf("Name: %s", s.Name),
fmt.Sprintf("Service: %s", s.Service),
fmt.Sprintf("Resource: %s", s.Resource),
fmt.Sprintf("TraceID: %d", s.TraceID),
fmt.Sprintf("SpanID: %d", s.SpanID),
fmt.Sprintf("ParentID: %d", s.ParentID),
fmt.Sprintf("Start: %s", time.Unix(0, s.Start)),
fmt.Sprintf("Duration: %s", time.Duration(s.Duration)),
fmt.Sprintf("Error: %d", s.Error),
fmt.Sprintf("Type: %s", s.Type),
"Tags:",
}
s.RLock()
for key, val := range s.Meta {
lines = append(lines, fmt.Sprintf("\t%s:%s", key, val))
}
for key, val := range s.Metrics {
lines = append(lines, fmt.Sprintf("\t%s:%f", key, val))
}
s.RUnlock()
return strings.Join(lines, "\n")
}
const (
samplingPriorityKey = "_sampling_priority_v1"
samplingPriorityRateKey = "_sampling_priority_rate_v1"
)
package tracer
// NOTE: THIS FILE WAS PRODUCED BY THE
// MSGP CODE GENERATION TOOL (github.com/tinylib/msgp)
// DO NOT EDIT
import (
"github.com/tinylib/msgp/msgp"
)
// DecodeMsg implements msgp.Decodable
func (z *span) DecodeMsg(dc *msgp.Reader) (err error) {
var field []byte
_ = field
var zb0001 uint32
zb0001, err = dc.ReadMapHeader()
if err != nil {
return
}
for zb0001 > 0 {
zb0001--
field, err = dc.ReadMapKeyPtr()
if err != nil {
return
}
switch msgp.UnsafeString(field) {
case "name":
z.Name, err = dc.ReadString()
if err != nil {
return
}
case "service":
z.Service, err = dc.ReadString()
if err != nil {
return
}
case "resource":
z.Resource, err = dc.ReadString()
if err != nil {
return
}
case "type":
z.Type, err = dc.ReadString()
if err != nil {
return
}
case "start":
z.Start, err = dc.ReadInt64()
if err != nil {
return
}
case "duration":
z.Duration, err = dc.ReadInt64()
if err != nil {
return
}
case "meta":
var zb0002 uint32
zb0002, err = dc.ReadMapHeader()
if err != nil {
return
}
if z.Meta == nil && zb0002 > 0 {
z.Meta = make(map[string]string, zb0002)
} else if len(z.Meta) > 0 {
for key := range z.Meta {
delete(z.Meta, key)
}
}
for zb0002 > 0 {
zb0002--
var za0001 string
var za0002 string
za0001, err = dc.ReadString()
if err != nil {
return
}
za0002, err = dc.ReadString()
if err != nil {
return
}
z.Meta[za0001] = za0002
}
case "metrics":
var zb0003 uint32
zb0003, err = dc.ReadMapHeader()
if err != nil {
return
}
if z.Metrics == nil && zb0003 > 0 {
z.Metrics = make(map[string]float64, zb0003)
} else if len(z.Metrics) > 0 {
for key := range z.Metrics {
delete(z.Metrics, key)
}
}
for zb0003 > 0 {
zb0003--
var za0003 string
var za0004 float64
za0003, err = dc.ReadString()
if err != nil {
return
}
za0004, err = dc.ReadFloat64()
if err != nil {
return
}
z.Metrics[za0003] = za0004
}
case "span_id":
z.SpanID, err = dc.ReadUint64()
if err != nil {
return
}
case "trace_id":
z.TraceID, err = dc.ReadUint64()
if err != nil {
return
}
case "parent_id":
z.ParentID, err = dc.ReadUint64()
if err != nil {
return
}
case "error":
z.Error, err = dc.ReadInt32()
if err != nil {
return
}
default:
err = dc.Skip()
if err != nil {
return
}
}
}
return
}
// EncodeMsg implements msgp.Encodable
func (z *span) EncodeMsg(en *msgp.Writer) (err error) {
// map header, size 12
// write "name"
err = en.Append(0x8c, 0xa4, 0x6e, 0x61, 0x6d, 0x65)
if err != nil {
return
}
err = en.WriteString(z.Name)
if err != nil {
return
}
// write "service"
err = en.Append(0xa7, 0x73, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65)
if err != nil {
return
}
err = en.WriteString(z.Service)
if err != nil {
return
}
// write "resource"
err = en.Append(0xa8, 0x72, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65)
if err != nil {
return
}
err = en.WriteString(z.Resource)
if err != nil {
return
}
// write "type"
err = en.Append(0xa4, 0x74, 0x79, 0x70, 0x65)
if err != nil {
return
}
err = en.WriteString(z.Type)
if err != nil {
return
}
// write "start"
err = en.Append(0xa5, 0x73, 0x74, 0x61, 0x72, 0x74)
if err != nil {
return
}
err = en.WriteInt64(z.Start)
if err != nil {
return
}
// write "duration"
err = en.Append(0xa8, 0x64, 0x75, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e)
if err != nil {
return
}
err = en.WriteInt64(z.Duration)
if err != nil {
return
}
// write "meta"
err = en.Append(0xa4, 0x6d, 0x65, 0x74, 0x61)
if err != nil {
return
}
err = en.WriteMapHeader(uint32(len(z.Meta)))
if err != nil {
return
}
for za0001, za0002 := range z.Meta {
err = en.WriteString(za0001)
if err != nil {
return
}
err = en.WriteString(za0002)
if err != nil {
return
}
}
// write "metrics"
err = en.Append(0xa7, 0x6d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73)
if err != nil {
return
}
err = en.WriteMapHeader(uint32(len(z.Metrics)))
if err != nil {
return
}
for za0003, za0004 := range z.Metrics {
err = en.WriteString(za0003)
if err != nil {
return
}
err = en.WriteFloat64(za0004)
if err != nil {
return
}
}
// write "span_id"
err = en.Append(0xa7, 0x73, 0x70, 0x61, 0x6e, 0x5f, 0x69, 0x64)
if err != nil {
return
}
err = en.WriteUint64(z.SpanID)
if err != nil {
return
}
// write "trace_id"
err = en.Append(0xa8, 0x74, 0x72, 0x61, 0x63, 0x65, 0x5f, 0x69, 0x64)
if err != nil {
return
}
err = en.WriteUint64(z.TraceID)
if err != nil {
return
}
// write "parent_id"
err = en.Append(0xa9, 0x70, 0x61, 0x72, 0x65, 0x6e, 0x74, 0x5f, 0x69, 0x64)
if err != nil {
return
}
err = en.WriteUint64(z.ParentID)
if err != nil {
return
}
// write "error"
err = en.Append(0xa5, 0x65, 0x72, 0x72, 0x6f, 0x72)
if err != nil {
return
}
err = en.WriteInt32(z.Error)
if err != nil {
return
}
return
}
// Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message
func (z *span) Msgsize() (s int) {
s = 1 + 5 + msgp.StringPrefixSize + len(z.Name) + 8 + msgp.StringPrefixSize + len(z.Service) + 9 + msgp.StringPrefixSize + len(z.Resource) + 5 + msgp.StringPrefixSize + len(z.Type) + 6 + msgp.Int64Size + 9 + msgp.Int64Size + 5 + msgp.MapHeaderSize
if z.Meta != nil {
for za0001, za0002 := range z.Meta {
_ = za0002
s += msgp.StringPrefixSize + len(za0001) + msgp.StringPrefixSize + len(za0002)
}
}
s += 8 + msgp.MapHeaderSize
if z.Metrics != nil {
for za0003, za0004 := range z.Metrics {
_ = za0004
s += msgp.StringPrefixSize + len(za0003) + msgp.Float64Size
}
}
s += 8 + msgp.Uint64Size + 9 + msgp.Uint64Size + 10 + msgp.Uint64Size + 6 + msgp.Int32Size
return
}
// DecodeMsg implements msgp.Decodable
func (z *spanList) DecodeMsg(dc *msgp.Reader) (err error) {
var zb0002 uint32
zb0002, err = dc.ReadArrayHeader()
if err != nil {
return
}
if cap((*z)) >= int(zb0002) {
(*z) = (*z)[:zb0002]
} else {
(*z) = make(spanList, zb0002)
}
for zb0001 := range *z {
if dc.IsNil() {
err = dc.ReadNil()
if err != nil {
return
}
(*z)[zb0001] = nil
} else {
if (*z)[zb0001] == nil {
(*z)[zb0001] = new(span)
}
err = (*z)[zb0001].DecodeMsg(dc)
if err != nil {
return
}
}
}
return
}
// EncodeMsg implements msgp.Encodable
func (z spanList) EncodeMsg(en *msgp.Writer) (err error) {
err = en.WriteArrayHeader(uint32(len(z)))
if err != nil {
return
}
for zb0003 := range z {
if z[zb0003] == nil {
err = en.WriteNil()
if err != nil {
return
}
} else {
err = z[zb0003].EncodeMsg(en)
if err != nil {
return
}
}
}
return
}
// Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message
func (z spanList) Msgsize() (s int) {
s = msgp.ArrayHeaderSize
for zb0003 := range z {
if z[zb0003] == nil {
s += msgp.NilSize
} else {
s += z[zb0003].Msgsize()
}
}
return
}
// DecodeMsg implements msgp.Decodable
func (z *spanLists) DecodeMsg(dc *msgp.Reader) (err error) {
var zb0003 uint32
zb0003, err = dc.ReadArrayHeader()
if err != nil {
return
}
if cap((*z)) >= int(zb0003) {
(*z) = (*z)[:zb0003]
} else {
(*z) = make(spanLists, zb0003)
}
for zb0001 := range *z {
var zb0004 uint32
zb0004, err = dc.ReadArrayHeader()
if err != nil {
return
}
if cap((*z)[zb0001]) >= int(zb0004) {
(*z)[zb0001] = ((*z)[zb0001])[:zb0004]
} else {
(*z)[zb0001] = make(spanList, zb0004)
}
for zb0002 := range (*z)[zb0001] {
if dc.IsNil() {
err = dc.ReadNil()
if err != nil {
return
}
(*z)[zb0001][zb0002] = nil
} else {
if (*z)[zb0001][zb0002] == nil {
(*z)[zb0001][zb0002] = new(span)
}
err = (*z)[zb0001][zb0002].DecodeMsg(dc)
if err != nil {
return
}
}
}
}
return
}
// EncodeMsg implements msgp.Encodable
func (z spanLists) EncodeMsg(en *msgp.Writer) (err error) {
err = en.WriteArrayHeader(uint32(len(z)))
if err != nil {
return
}
for zb0005 := range z {
err = en.WriteArrayHeader(uint32(len(z[zb0005])))
if err != nil {
return
}
for zb0006 := range z[zb0005] {
if z[zb0005][zb0006] == nil {
err = en.WriteNil()
if err != nil {
return
}
} else {
err = z[zb0005][zb0006].EncodeMsg(en)
if err != nil {
return
}
}
}
}
return
}
// Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message
func (z spanLists) Msgsize() (s int) {
s = msgp.ArrayHeaderSize
for zb0005 := range z {
s += msgp.ArrayHeaderSize
for zb0006 := range z[zb0005] {
if z[zb0005][zb0006] == nil {
s += msgp.NilSize
} else {
s += z[zb0005][zb0006].Msgsize()
}
}
}
return
}
package tracer
import (
"sync"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace/internal"
)
var _ ddtrace.SpanContext = (*spanContext)(nil)
// SpanContext represents a span state that can propagate to descendant spans
// and across process boundaries. It contains all the information needed to
// spawn a direct descendant of the span that it belongs to. It can be used
// to create distributed tracing by propagating it using the provided interfaces.
type spanContext struct {
// the below group should propagate only locally
trace *trace // reference to the trace that this span belongs too
span *span // reference to the span that hosts this context
drop bool // when true, the span will not be sent to the agent
// the below group should propagate cross-process
traceID uint64
spanID uint64
mu sync.RWMutex // guards below fields
baggage map[string]string
priority int
hasPriority bool
}
// newSpanContext creates a new SpanContext to serve as context for the given
// span. If the provided parent is not nil, the context will inherit the trace,
// baggage and other values from it. This method also pushes the span into the
// new context's trace and as a result, it should not be called multiple times
// for the same span.
func newSpanContext(span *span, parent *spanContext) *spanContext {
context := &spanContext{
traceID: span.TraceID,
spanID: span.SpanID,
span: span,
}
if v, ok := span.Metrics[samplingPriorityKey]; ok {
context.hasPriority = true
context.priority = int(v)
}
if parent != nil {
context.trace = parent.trace
context.drop = parent.drop
context.hasPriority = parent.hasSamplingPriority()
context.priority = parent.samplingPriority()
parent.ForeachBaggageItem(func(k, v string) bool {
context.setBaggageItem(k, v)
return true
})
}
if context.trace == nil {
context.trace = newTrace()
}
// put span in context's trace
context.trace.push(span)
return context
}
// SpanID implements ddtrace.SpanContext.
func (c *spanContext) SpanID() uint64 { return c.spanID }
// TraceID implements ddtrace.SpanContext.
func (c *spanContext) TraceID() uint64 { return c.traceID }
// ForeachBaggageItem implements ddtrace.SpanContext.
func (c *spanContext) ForeachBaggageItem(handler func(k, v string) bool) {
c.mu.RLock()
defer c.mu.RUnlock()
for k, v := range c.baggage {
if !handler(k, v) {
break
}
}
}
func (c *spanContext) setSamplingPriority(p int) {
c.mu.Lock()
defer c.mu.Unlock()
c.priority = p
c.hasPriority = true
}
func (c *spanContext) samplingPriority() int {
c.mu.RLock()
defer c.mu.RUnlock()
return c.priority
}
func (c *spanContext) hasSamplingPriority() bool {
c.mu.RLock()
defer c.mu.RUnlock()
return c.hasPriority
}
func (c *spanContext) setBaggageItem(key, val string) {
c.mu.Lock()
defer c.mu.Unlock()
if c.baggage == nil {
c.baggage = make(map[string]string, 1)
}
c.baggage[key] = val
}
func (c *spanContext) baggageItem(key string) string {
c.mu.RLock()
defer c.mu.RUnlock()
return c.baggage[key]
}
// finish marks this span as finished in the trace.
func (c *spanContext) finish() { c.trace.ackFinish() }
// trace holds information about a specific trace. This structure is shared
// between all spans in a trace.
type trace struct {
mu sync.RWMutex // guards below fields
spans []*span // all the spans that are part of this trace
finished int // the number of finished spans
full bool // signifies that the span buffer is full
}
var (
// traceStartSize is the initial size of our trace buffer,
// by default we allocate for a handful of spans within the trace,
// reasonable as span is actually way bigger, and avoids re-allocating
// over and over. Could be fine-tuned at runtime.
traceStartSize = 10
// traceMaxSize is the maximum number of spans we keep in memory.
// This is to avoid memory leaks, if above that value, spans are randomly
// dropped and ignore, resulting in corrupted tracing data, but ensuring
// original program continues to work as expected.
traceMaxSize = int(1e5)
)
// newTrace creates a new trace using the given callback which will be called
// upon completion of the trace.
func newTrace() *trace {
return &trace{spans: make([]*span, 0, traceStartSize)}
}
// push pushes a new span into the trace. If the buffer is full, it returns
// a errBufferFull error.
func (t *trace) push(sp *span) {
t.mu.Lock()
defer t.mu.Unlock()
if t.full {
return
}
if len(t.spans) >= traceMaxSize {
// capacity is reached, we will not be able to complete this trace.
t.full = true
t.spans = nil // GC
if tr, ok := internal.GetGlobalTracer().(*tracer); ok {
// we have a tracer we can submit errors too.
tr.pushError(&spanBufferFullError{})
}
return
}
t.spans = append(t.spans, sp)
}
// ackFinish aknowledges that another span in the trace has finished, and checks
// if the trace is complete, in which case it calls the onFinish function.
func (t *trace) ackFinish() {
t.mu.Lock()
defer t.mu.Unlock()
if t.full {
// capacity has been reached, the buffer is no longer tracking
// all the spans in the trace, so the below conditions will not
// be accurate and would trigger a pre-mature flush, exposing us
// to a race condition where spans can be modified while flushing.
return
}
t.finished++
if len(t.spans) != t.finished {
return
}
if tr, ok := internal.GetGlobalTracer().(*tracer); ok {
// we have a tracer that can receive completed traces.
tr.pushTrace(t.spans)
}
t.spans = nil
t.finished = 0 // important, because a buffer can be used for several flushes
}
package tracer
import (
"net/http"
"strconv"
"strings"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace"
)
// HTTPHeadersCarrier wraps an http.Header as a TextMapWriter and TextMapReader, allowing
// it to be used using the provided Propagator implementation.
type HTTPHeadersCarrier http.Header
var _ TextMapWriter = (*HTTPHeadersCarrier)(nil)
var _ TextMapReader = (*HTTPHeadersCarrier)(nil)
// Set implements TextMapWriter.
func (c HTTPHeadersCarrier) Set(key, val string) {
http.Header(c).Set(key, val)
}
// ForeachKey implements TextMapReader.
func (c HTTPHeadersCarrier) ForeachKey(handler func(key, val string) error) error {
for k, vals := range c {
for _, v := range vals {
if err := handler(k, v); err != nil {
return err
}
}
}
return nil
}
// TextMapCarrier allows the use of a regular map[string]string as both TextMapWriter
// and TextMapReader, making it compatible with the provided Propagator.
type TextMapCarrier map[string]string
var _ TextMapWriter = (*TextMapCarrier)(nil)
var _ TextMapReader = (*TextMapCarrier)(nil)
// Set implements TextMapWriter.
func (c TextMapCarrier) Set(key, val string) {
c[key] = val
}
// ForeachKey conforms to the TextMapReader interface.
func (c TextMapCarrier) ForeachKey(handler func(key, val string) error) error {
for k, v := range c {
if err := handler(k, v); err != nil {
return err
}
}
return nil
}
const (
// DefaultBaggageHeaderPrefix specifies the prefix that will be used in
// HTTP headers or text maps to prefix baggage keys.
DefaultBaggageHeaderPrefix = "ot-baggage-"
// DefaultTraceIDHeader specifies the key that will be used in HTTP headers
// or text maps to store the trace ID.
DefaultTraceIDHeader = "x-datadog-trace-id"
// DefaultParentIDHeader specifies the key that will be used in HTTP headers
// or text maps to store the parent ID.
DefaultParentIDHeader = "x-datadog-parent-id"
// DefaultPriorityHeader specifies the key that will be used in HTTP headers
// or text maps to store the sampling priority value.
DefaultPriorityHeader = "x-datadog-sampling-priority"
)
// PropagatorConfig defines the configuration for initializing a propagator.
type PropagatorConfig struct {
// BaggagePrefix specifies the prefix that will be used to store baggage
// items in a map. It defaults to DefaultBaggageHeaderPrefix.
BaggagePrefix string
// TraceHeader specifies the map key that will be used to store the trace ID.
// It defaults to DefaultTraceIDHeader.
TraceHeader string
// ParentHeader specifies the map key that will be used to store the parent ID.
// It defaults to DefaultParentIDHeader.
ParentHeader string
// PriorityHeader specifies the map key that will be used to store the sampling priority.
// It deafults to DefaultPriorityHeader.
PriorityHeader string
}
// NewPropagator returns a new propagator which uses TextMap to inject
// and extract values. It propagates trace and span IDs and baggage.
// To use the defaults, nil may be provided in place of the config.
func NewPropagator(cfg *PropagatorConfig) Propagator {
if cfg == nil {
cfg = new(PropagatorConfig)
}
if cfg.BaggagePrefix == "" {
cfg.BaggagePrefix = DefaultBaggageHeaderPrefix
}
if cfg.TraceHeader == "" {
cfg.TraceHeader = DefaultTraceIDHeader
}
if cfg.ParentHeader == "" {
cfg.ParentHeader = DefaultParentIDHeader
}
if cfg.PriorityHeader == "" {
cfg.PriorityHeader = DefaultPriorityHeader
}
return &propagator{cfg}
}
// propagator implements a propagator which uses TextMap internally.
// It propagates the trace and span IDs, as well as the baggage from the
// context.
type propagator struct{ cfg *PropagatorConfig }
// Inject defines the Propagator to propagate SpanContext data
// out of the current process. The implementation propagates the
// TraceID and the current active SpanID, as well as the Span baggage.
func (p *propagator) Inject(spanCtx ddtrace.SpanContext, carrier interface{}) error {
switch v := carrier.(type) {
case TextMapWriter:
return p.injectTextMap(spanCtx, v)
default:
return ErrInvalidCarrier
}
}
func (p *propagator) injectTextMap(spanCtx ddtrace.SpanContext, writer TextMapWriter) error {
ctx, ok := spanCtx.(*spanContext)
if !ok || ctx.traceID == 0 || ctx.spanID == 0 {
return ErrInvalidSpanContext
}
// propagate the TraceID and the current active SpanID
writer.Set(p.cfg.TraceHeader, strconv.FormatUint(ctx.traceID, 10))
writer.Set(p.cfg.ParentHeader, strconv.FormatUint(ctx.spanID, 10))
if ctx.hasSamplingPriority() {
writer.Set(p.cfg.PriorityHeader, strconv.Itoa(ctx.samplingPriority()))
}
// propagate OpenTracing baggage
for k, v := range ctx.baggage {
writer.Set(p.cfg.BaggagePrefix+k, v)
}
return nil
}
// Extract implements Propagator.
func (p *propagator) Extract(carrier interface{}) (ddtrace.SpanContext, error) {
switch v := carrier.(type) {
case TextMapReader:
return p.extractTextMap(v)
default:
return nil, ErrInvalidCarrier
}
}
func (p *propagator) extractTextMap(reader TextMapReader) (ddtrace.SpanContext, error) {
var ctx spanContext
err := reader.ForeachKey(func(k, v string) error {
var err error
key := strings.ToLower(k)
switch key {
case p.cfg.TraceHeader:
ctx.traceID, err = parseUint64(v)
if err != nil {
return ErrSpanContextCorrupted
}
case p.cfg.ParentHeader:
ctx.spanID, err = parseUint64(v)
if err != nil {
return ErrSpanContextCorrupted
}
case p.cfg.PriorityHeader:
priority, err := strconv.Atoi(v)
if err != nil {
return ErrSpanContextCorrupted
}
ctx.setSamplingPriority(priority)
default:
if strings.HasPrefix(key, p.cfg.BaggagePrefix) {
ctx.setBaggageItem(strings.TrimPrefix(key, p.cfg.BaggagePrefix), v)
}
}
return nil
})
if err != nil {
return nil, err
}
if ctx.traceID == 0 || ctx.spanID == 0 {
return nil, ErrSpanContextNotFound
}
return &ctx, nil
}
// +build !windows
package tracer
import "time"
// now returns current UTC time in nanos.
func now() int64 {
return time.Now().UTC().UnixNano()
}
package tracer
import (
"log"
"time"
"golang.org/x/sys/windows"
)
// This method is more precise than the go1.8 time.Now on Windows
// See https://msdn.microsoft.com/en-us/library/windows/desktop/hh706895(v=vs.85).aspx
// It is however ~10x slower and requires Windows 8+.
func highPrecisionNow() int64 {
var ft windows.Filetime
windows.GetSystemTimePreciseAsFileTime(&ft)
return ft.Nanoseconds()
}
func lowPrecisionNow() int64 {
return time.Now().UTC().UnixNano()
}
var now func() int64
// If GetSystemTimePreciseAsFileTime is not available we default to the less
// precise implementation based on time.Now()
func init() {
if err := windows.LoadGetSystemTimePreciseAsFileTime(); err != nil {
log.Printf("Unable to load high precison timer, defaulting to time.Now()")
now = lowPrecisionNow
} else {
log.Printf("Using high precision timer")
now = highPrecisionNow
}
}
package tracer
import (
"errors"
"log"
"os"
"strconv"
"time"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace/ext"
"gopkg.in/DataDog/dd-trace-go.v1/ddtrace/internal"
)
var _ ddtrace.Tracer = (*tracer)(nil)
// tracer creates, buffers and submits Spans which are used to time blocks of
// computation. They are accumulated and streamed into an internal payload,
// which is flushed to the agent whenever its size exceeds a specific threshold
// or when a certain interval of time has passed, whichever happens first.
//
// tracer operates based on a worker loop which responds to various request
// channels. It additionally holds two buffers which accumulates error and trace
// queues to be processed by the payload encoder.
type tracer struct {
*config
*payload
flushAllReq chan chan<- struct{}
flushTracesReq chan struct{}
flushErrorsReq chan struct{}
exitReq chan struct{}
payloadQueue chan []*span
errorBuffer chan error
// stopped is a channel that will be closed when the worker has exited.
stopped chan struct{}
// syncPush is used for testing. When non-nil, it causes pushTrace to become
// a synchronous (blocking) operation, meaning that it will only return after
// the trace has been fully processed and added onto the payload.
syncPush chan struct{}
// prioritySampling holds an instance of the priority sampler.
prioritySampling *prioritySampler
}
const (
// flushInterval is the interval at which the payload contents will be flushed
// to the transport.
flushInterval = 2 * time.Second
// payloadMaxLimit is the maximum payload size allowed and should indicate the
// maximum size of the package that the agent can receive.
payloadMaxLimit = 9.5 * 1024 * 1024 // 9.5 MB
// payloadSizeLimit specifies the maximum allowed size of the payload before
// it will trigger a flush to the transport.
payloadSizeLimit = payloadMaxLimit / 2
)
// Start starts the tracer with the given set of options. It will stop and replace
// any running tracer, meaning that calling it several times will result in a restart
// of the tracer by replacing the current instance with a new one.
func Start(opts ...StartOption) {
if internal.Testing {
return // mock tracer active
}
internal.SetGlobalTracer(newTracer(opts...))
}
// Stop stops the started tracer. Subsequent calls are valid but become no-op.
func Stop() {
internal.SetGlobalTracer(&internal.NoopTracer{})
}
// Span is an alias for ddtrace.Span. It is here to allow godoc to group methods returning
// ddtrace.Span. It is recommended and is considered more correct to refer to this type as
// ddtrace.Span instead.
type Span = ddtrace.Span
// StartSpan starts a new span with the given operation name and set of options.
// If the tracer is not started, calling this function is a no-op.
func StartSpan(operationName string, opts ...StartSpanOption) Span {
return internal.GetGlobalTracer().StartSpan(operationName, opts...)
}
// Extract extracts a SpanContext from the carrier. The carrier is expected
// to implement TextMapReader, otherwise an error is returned.
// If the tracer is not started, calling this function is a no-op.
func Extract(carrier interface{}) (ddtrace.SpanContext, error) {
return internal.GetGlobalTracer().Extract(carrier)
}
// Inject injects the given SpanContext into the carrier. The carrier is
// expected to implement TextMapWriter, otherwise an error is returned.
// If the tracer is not started, calling this function is a no-op.
func Inject(ctx ddtrace.SpanContext, carrier interface{}) error {
return internal.GetGlobalTracer().Inject(ctx, carrier)
}
const (
// payloadQueueSize is the buffer size of the trace channel.
payloadQueueSize = 1000
// errorBufferSize is the buffer size of the error channel.
errorBufferSize = 200
)
func newTracer(opts ...StartOption) *tracer {
c := new(config)
defaults(c)
for _, fn := range opts {
fn(c)
}
if c.transport == nil {
c.transport = newTransport(c.agentAddr, c.httpRoundTripper)
}
if c.propagator == nil {
c.propagator = NewPropagator(nil)
}
t := &tracer{
config: c,
payload: newPayload(),
flushAllReq: make(chan chan<- struct{}),
flushTracesReq: make(chan struct{}, 1),
flushErrorsReq: make(chan struct{}, 1),
exitReq: make(chan struct{}),
payloadQueue: make(chan []*span, payloadQueueSize),
errorBuffer: make(chan error, errorBufferSize),
stopped: make(chan struct{}),
prioritySampling: newPrioritySampler(),
}
go t.worker()
return t
}
// worker receives finished traces to be added into the payload, as well
// as periodically flushes traces to the transport.
func (t *tracer) worker() {
defer close(t.stopped)
ticker := time.NewTicker(flushInterval)
defer ticker.Stop()
for {
select {
case trace := <-t.payloadQueue:
t.pushPayload(trace)
case <-ticker.C:
t.flush()
case done := <-t.flushAllReq:
t.flush()
done <- struct{}{}
case <-t.flushTracesReq:
t.flushTraces()
case <-t.flushErrorsReq:
t.flushErrors()
case <-t.exitReq:
t.flush()
return
}
}
}
func (t *tracer) pushTrace(trace []*span) {
select {
case <-t.stopped:
return
default:
}
select {
case t.payloadQueue <- trace:
default:
t.pushError(&dataLossError{
context: errors.New("payload queue full, dropping trace"),
count: len(trace),
})
}
if t.syncPush != nil {
// only in tests
<-t.syncPush
}
}
func (t *tracer) pushError(err error) {
select {
case <-t.stopped:
return
default:
}
if len(t.errorBuffer) >= cap(t.errorBuffer)/2 { // starts being full, anticipate, try and flush soon
select {
case t.flushErrorsReq <- struct{}{}:
default: // a flush was already requested, skip
}
}
select {
case t.errorBuffer <- err:
default:
// OK, if we get this, our error error buffer is full,
// we can assume it is filled with meaningful messages which
// are going to be logged and hopefully read, nothing better
// we can do, blocking would make things worse.
}
}
// StartSpan creates, starts, and returns a new Span with the given `operationName`.
func (t *tracer) StartSpan(operationName string, options ...ddtrace.StartSpanOption) ddtrace.Span {
var opts ddtrace.StartSpanConfig
for _, fn := range options {
fn(&opts)
}
var startTime int64
if opts.StartTime.IsZero() {
startTime = now()
} else {
startTime = opts.StartTime.UnixNano()
}
var context *spanContext
if opts.Parent != nil {
if ctx, ok := opts.Parent.(*spanContext); ok {
context = ctx
}
}
id := random.Uint64()
// span defaults
span := &span{
Name: operationName,
Service: t.config.serviceName,
Resource: operationName,
Meta: map[string]string{},
Metrics: map[string]float64{},
SpanID: id,
TraceID: id,
ParentID: 0,
Start: startTime,
}
if context != nil {
// this is a child span
span.TraceID = context.traceID
span.ParentID = context.spanID
if context.hasSamplingPriority() {
span.Metrics[samplingPriorityKey] = float64(context.samplingPriority())
}
if context.span != nil {
// it has a local parent, inherit the service
context.span.RLock()
span.Service = context.span.Service
context.span.RUnlock()
}
}
span.context = newSpanContext(span, context)
if context == nil || context.span == nil {
// this is either a root span or it has a remote parent, we should add the PID.
span.SetTag(ext.Pid, strconv.Itoa(os.Getpid()))
}
// add tags from options
for k, v := range opts.Tags {
span.SetTag(k, v)
}
// add global tags
for k, v := range t.config.globalTags {
span.SetTag(k, v)
}
if context == nil {
// this is a brand new trace, sample it
t.sample(span)
}
return span
}
// Stop stops the tracer.
func (t *tracer) Stop() {
select {
case <-t.stopped:
return
default:
t.exitReq <- struct{}{}
<-t.stopped
}
}
// Inject uses the configured or default TextMap Propagator.
func (t *tracer) Inject(ctx ddtrace.SpanContext, carrier interface{}) error {
return t.config.propagator.Inject(ctx, carrier)
}
// Extract uses the configured or default TextMap Propagator.
func (t *tracer) Extract(carrier interface{}) (ddtrace.SpanContext, error) {
return t.config.propagator.Extract(carrier)
}
// flushTraces will push any currently buffered traces to the server.
func (t *tracer) flushTraces() {
if t.payload.itemCount() == 0 {
return
}
size, count := t.payload.size(), t.payload.itemCount()
if t.config.debug {
log.Printf("Sending payload: size: %d traces: %d\n", size, count)
}
rc, err := t.config.transport.send(t.payload)
if err != nil {
t.pushError(&dataLossError{context: err, count: count})
}
if err == nil {
t.prioritySampling.readRatesJSON(rc) // TODO: handle error?
}
t.payload.reset()
}
// flushErrors will process log messages that were queued
func (t *tracer) flushErrors() {
logErrors(t.errorBuffer)
}
func (t *tracer) flush() {
t.flushTraces()
t.flushErrors()
}
// forceFlush forces a flush of data (traces and services) to the agent.
// Flushes are done by a background task on a regular basis, so you never
// need to call this manually, mostly useful for testing and debugging.
func (t *tracer) forceFlush() {
done := make(chan struct{})
t.flushAllReq <- done
<-done
}
// pushPayload pushes the trace onto the payload. If the payload becomes
// larger than the threshold as a result, it sends a flush request.
func (t *tracer) pushPayload(trace []*span) {
if err := t.payload.push(trace); err != nil {
t.pushError(&traceEncodingError{context: err})
}
if t.payload.size() > payloadSizeLimit {
// getting large
select {
case t.flushTracesReq <- struct{}{}:
default:
// flush already queued
}
}
if t.syncPush != nil {
// only in tests
t.syncPush <- struct{}{}
}
}
// sampleRateMetricKey is the metric key holding the applied sample rate. Has to be the same as the Agent.
const sampleRateMetricKey = "_sample_rate"
// Sample samples a span with the internal sampler.
func (t *tracer) sample(span *span) {
if span.context.hasPriority {
// sampling decision was already made
return
}
sampler := t.config.sampler
if !sampler.Sample(span) {
span.context.drop = true
return
}
if rs, ok := sampler.(RateSampler); ok && rs.Rate() < 1 {
span.Metrics[sampleRateMetricKey] = rs.Rate()
}
t.prioritySampling.apply(span)
}
package tracer
import (
"fmt"
"io"
"net"
"net/http"
"os"
"runtime"
"strconv"
"strings"
"time"
)
var (
// TODO(gbbr): find a more effective way to keep this up to date,
// e.g. via `go generate`
tracerVersion = "v1.7.0"
// We copy the transport to avoid using the default one, as it might be
// augmented with tracing and we don't want these calls to be recorded.
// See https://golang.org/pkg/net/http/#DefaultTransport .
defaultRoundTripper = &http.Transport{
Proxy: http.ProxyFromEnvironment,
DialContext: (&net.Dialer{
Timeout: 30 * time.Second,
KeepAlive: 30 * time.Second,
DualStack: true,
}).DialContext,
MaxIdleConns: 100,
IdleConnTimeout: 90 * time.Second,
TLSHandshakeTimeout: 10 * time.Second,
ExpectContinueTimeout: 1 * time.Second,
}
)
const (
defaultHostname = "localhost"
defaultPort = "8126"
defaultAddress = defaultHostname + ":" + defaultPort
defaultHTTPTimeout = time.Second // defines the current timeout before giving up with the send process
traceCountHeader = "X-Datadog-Trace-Count" // header containing the number of traces in the payload
)
// transport is an interface for span submission to the agent.
type transport interface {
// send sends the payload p to the agent using the transport set up.
// It returns a non-nil response body when no error occurred.
send(p *payload) (body io.ReadCloser, err error)
}
// newTransport returns a new Transport implementation that sends traces to a
// trace agent running on the given hostname and port, using a given
// http.RoundTripper. If the zero values for hostname and port are provided,
// the default values will be used ("localhost" for hostname, and "8126" for
// port). If roundTripper is nil, a default is used.
//
// In general, using this method is only necessary if you have a trace agent
// running on a non-default port, if it's located on another machine, or when
// otherwise needing to customize the transport layer, for instance when using
// a unix domain socket.
func newTransport(addr string, roundTripper http.RoundTripper) transport {
if roundTripper == nil {
roundTripper = defaultRoundTripper
}
return newHTTPTransport(addr, roundTripper)
}
// newDefaultTransport return a default transport for this tracing client
func newDefaultTransport() transport {
return newHTTPTransport(defaultAddress, defaultRoundTripper)
}
type httpTransport struct {
traceURL string // the delivery URL for traces
client *http.Client // the HTTP client used in the POST
headers map[string]string // the Transport headers
}
// newHTTPTransport returns an httpTransport for the given endpoint
func newHTTPTransport(addr string, roundTripper http.RoundTripper) *httpTransport {
// initialize the default EncoderPool with Encoder headers
defaultHeaders := map[string]string{
"Datadog-Meta-Lang": "go",
"Datadog-Meta-Lang-Version": strings.TrimPrefix(runtime.Version(), "go"),
"Datadog-Meta-Lang-Interpreter": runtime.Compiler + "-" + runtime.GOARCH + "-" + runtime.GOOS,
"Datadog-Meta-Tracer-Version": tracerVersion,
"Content-Type": "application/msgpack",
}
return &httpTransport{
traceURL: fmt.Sprintf("http://%s/v0.4/traces", resolveAddr(addr)),
client: &http.Client{
Transport: roundTripper,
Timeout: defaultHTTPTimeout,
},
headers: defaultHeaders,
}
}
func (t *httpTransport) send(p *payload) (body io.ReadCloser, err error) {
// prepare the client and send the payload
req, err := http.NewRequest("POST", t.traceURL, p)
if err != nil {
return nil, fmt.Errorf("cannot create http request: %v", err)
}
for header, value := range t.headers {
req.Header.Set(header, value)
}
req.Header.Set(traceCountHeader, strconv.Itoa(p.itemCount()))
req.Header.Set("Content-Length", strconv.Itoa(p.size()))
response, err := t.client.Do(req)
if err != nil {
return nil, err
}
if code := response.StatusCode; code >= 400 {
// error, check the body for context information and
// return a nice error.
msg := make([]byte, 1000)
n, _ := response.Body.Read(msg)
response.Body.Close()
txt := http.StatusText(code)
if n > 0 {
return nil, fmt.Errorf("%s (Status: %s)", msg[:n], txt)
}
return nil, fmt.Errorf("%s", txt)
}
return response.Body, nil
}
// resolveAddr resolves the given agent address and fills in any missing host
// and port using the defaults. Some environment variable settings will
// take precedence over configuration.
func resolveAddr(addr string) string {
host, port, err := net.SplitHostPort(addr)
if err != nil {
// no port in addr
host = addr
}
if host == "" {
host = defaultHostname
}
if port == "" {
port = defaultPort
}
if v := os.Getenv("DD_AGENT_HOST"); v != "" {
host = v
}
if v := os.Getenv("DD_TRACE_AGENT_PORT"); v != "" {
port = v
}
return fmt.Sprintf("%s:%s", host, port)
}
package tracer
import (
"strconv"
"strings"
)
// toFloat64 attempts to convert value into a float64. If it succeeds it returns
// the value and true, otherwise 0 and false.
func toFloat64(value interface{}) (f float64, ok bool) {
switch i := value.(type) {
case byte:
return float64(i), true
case float32:
return float64(i), true
case float64:
return i, true
case int:
return float64(i), true
case int16:
return float64(i), true
case int32:
return float64(i), true
case int64:
return float64(i), true
case uint:
return float64(i), true
case uint16:
return float64(i), true
case uint32:
return float64(i), true
case uint64:
return float64(i), true
default:
return 0, false
}
}
// parseUint64 parses a uint64 from either an unsigned 64 bit base-10 string
// or a signed 64 bit base-10 string representing an unsigned integer
func parseUint64(str string) (uint64, error) {
if strings.HasPrefix(str, "-") {
id, err := strconv.ParseInt(str, 10, 64)
if err != nil {
return 0, err
}
return uint64(id), nil
}
return strconv.ParseUint(str, 10, 64)
}
...@@ -66,6 +66,51 @@ ...@@ -66,6 +66,51 @@
"revision": "3304cc8863525cd0b328fbfd5bf745bbd38e7106", "revision": "3304cc8863525cd0b328fbfd5bf745bbd38e7106",
"revisionTime": "2018-11-12T10:25:10Z" "revisionTime": "2018-11-12T10:25:10Z"
}, },
{
"path": "github.com/lightstep/lightstep-tracer-go",
"revision": "c0184d44cb322c9d9abcaebb2139dc754b3c0145",
"revisionTime": "2019-02-19T15:09:25Z"
},
{
"path": "github.com/lightstep/lightstep-tracer-go/collectorpb",
"revision": "c0184d44cb322c9d9abcaebb2139dc754b3c0145"
},
{
"path": "github.com/lightstep/lightstep-tracer-go/lightstep/rand",
"revision": "c0184d44cb322c9d9abcaebb2139dc754b3c0145"
},
{
"path": "github.com/lightstep/lightstep-tracer-go/lightsteppb",
"revision": "c0184d44cb322c9d9abcaebb2139dc754b3c0145"
},
{
"checksumSHA1": "a/DHmc9bdsYlZZcwp6i3xhvV7Pk=",
"path": "github.com/opentracing/opentracing-go",
"revision": "25a84ff92183e2f8ac018ba1db54f8a07b3c0e04",
"revisionTime": "2019-02-18T02:30:34Z"
},
{
"checksumSHA1": "hZnCURJIhg56jbKk0UFRfZkcQ+c=",
"path": "github.com/opentracing/opentracing-go/ext",
"revision": "25a84ff92183e2f8ac018ba1db54f8a07b3c0e04",
"revisionTime": "2019-02-18T02:30:34Z"
},
{
"checksumSHA1": "tnkdNJbJxNKuPZMWapP1xhKIIGw=",
"path": "github.com/opentracing/opentracing-go/log",
"revision": "25a84ff92183e2f8ac018ba1db54f8a07b3c0e04",
"revisionTime": "2019-02-18T02:30:34Z"
},
{
"checksumSHA1": "8U5pEHFpXd1/Klgp+C/a6TqWoh8=",
"path": "github.com/philhofer/fwd",
"revision": "bb6d471dc95d4fe11e432687f8b70ff496cf3136",
"revisionTime": "2017-09-05T21:21:22Z"
},
{
"path": "github.com/pkg/errors",
"revision": ""
},
{ {
"checksumSHA1": "LuFv4/jlrmFNnDb/5SCSEPAM9vU=", "checksumSHA1": "LuFv4/jlrmFNnDb/5SCSEPAM9vU=",
"path": "github.com/pmezard/go-difflib/difflib", "path": "github.com/pmezard/go-difflib/difflib",
...@@ -110,6 +155,48 @@ ...@@ -110,6 +155,48 @@
"revision": "865fb2c8f5af9ccbaaadfd20d18236358dccaaff", "revision": "865fb2c8f5af9ccbaaadfd20d18236358dccaaff",
"revisionTime": "2018-09-01T16:25:39Z" "revisionTime": "2018-09-01T16:25:39Z"
}, },
{
"checksumSHA1": "XqGf8TvdzfKDhkE+HDZAT7RlERU=",
"path": "github.com/tinylib/msgp/msgp",
"revision": "ade0ca4ace05af96235d107023ab018ee921309c",
"revisionTime": "2019-01-03T19:08:39Z"
},
{
"path": "github.com/uber/jaeger-client-go",
"revision": ""
},
{
"checksumSHA1": "VGtI2lhUBiBj3XcEqL9oQfQMsz0=",
"path": "github.com/uber/jaeger-client-go/config",
"revision": "64f57863bf63d3842dbe79cdc793d57baaff9ab5",
"revisionTime": "2019-02-14T18:28:10Z"
},
{
"path": "github.com/uber/jaeger-client-go/internal/baggage/remote",
"revision": ""
},
{
"path": "github.com/uber/jaeger-client-go/internal/throttler/remote",
"revision": ""
},
{
"checksumSHA1": "tMP/vxbHwNAbOEaUhic5/meKfac=",
"path": "github.com/uber/jaeger-client-go/log",
"revision": "64f57863bf63d3842dbe79cdc793d57baaff9ab5",
"revisionTime": "2019-02-14T18:28:10Z"
},
{
"path": "github.com/uber/jaeger-client-go/rpcmetrics",
"revision": ""
},
{
"path": "github.com/uber/jaeger-client-go/transport",
"revision": ""
},
{
"path": "github.com/uber/jaeger-lib/metrics",
"revision": ""
},
{ {
"checksumSHA1": "AEuc/0oF2vF1nYiibaT/UIxMgbg=", "checksumSHA1": "AEuc/0oF2vF1nYiibaT/UIxMgbg=",
"path": "gitlab.com/gitlab-org/gitaly-proto/go/gitalypb", "path": "gitlab.com/gitlab-org/gitaly-proto/go/gitalypb",
...@@ -142,6 +229,38 @@ ...@@ -142,6 +229,38 @@
"version": "v1.7.0", "version": "v1.7.0",
"versionExact": "v1.7.0" "versionExact": "v1.7.0"
}, },
{
"checksumSHA1": "WMOuBgCyclqy+Mqunb0NbykaC4Y=",
"path": "gitlab.com/gitlab-org/labkit/correlation",
"revision": "0c3fc7cdd57c57da5ab474aa72b6640d2bdc9ebb",
"revisionTime": "2019-02-21T12:25:36Z",
"version": "master",
"versionExact": "master"
},
{
"checksumSHA1": "6SAh0LdyizW+RICpQglU6WLMhus=",
"path": "gitlab.com/gitlab-org/labkit/tracing",
"revision": "0c3fc7cdd57c57da5ab474aa72b6640d2bdc9ebb",
"revisionTime": "2019-02-21T12:25:36Z",
"version": "master",
"versionExact": "master"
},
{
"checksumSHA1": "uIvjqXAsMQK/Y5FgWRaGydYGbYs=",
"path": "gitlab.com/gitlab-org/labkit/tracing/connstr",
"revision": "0c3fc7cdd57c57da5ab474aa72b6640d2bdc9ebb",
"revisionTime": "2019-02-21T12:25:36Z",
"version": "master",
"versionExact": "master"
},
{
"checksumSHA1": "hB59Es/WTWfBPLSAheQaRyHGSXA=",
"path": "gitlab.com/gitlab-org/labkit/tracing/impl",
"revision": "0c3fc7cdd57c57da5ab474aa72b6640d2bdc9ebb",
"revisionTime": "2019-02-21T12:25:36Z",
"version": "master",
"versionExact": "master"
},
{ {
"checksumSHA1": "BGm8lKZmvJbf/YOJLeL1rw2WVjA=", "checksumSHA1": "BGm8lKZmvJbf/YOJLeL1rw2WVjA=",
"path": "golang.org/x/crypto/ssh/terminal", "path": "golang.org/x/crypto/ssh/terminal",
...@@ -472,6 +591,36 @@ ...@@ -472,6 +591,36 @@
"version": "v1.16.0", "version": "v1.16.0",
"versionExact": "v1.16.0" "versionExact": "v1.16.0"
}, },
{
"checksumSHA1": "GE96+o57Lj5bo7eUP/rPHAJvhOY=",
"path": "gopkg.in/DataDog/dd-trace-go.v1/ddtrace",
"revision": "823d51722f66a748804943f6fa6be18776073b82",
"revisionTime": "2019-01-24T09:21:27Z"
},
{
"checksumSHA1": "fHmbkWZvCViDUFo1S4vVETnZ8V0=",
"path": "gopkg.in/DataDog/dd-trace-go.v1/ddtrace/ext",
"revision": "823d51722f66a748804943f6fa6be18776073b82",
"revisionTime": "2019-01-24T09:21:27Z"
},
{
"checksumSHA1": "Y+OLwoHr6B/kbFMbI8IRui26wZ0=",
"path": "gopkg.in/DataDog/dd-trace-go.v1/ddtrace/internal",
"revision": "823d51722f66a748804943f6fa6be18776073b82",
"revisionTime": "2019-01-24T09:21:27Z"
},
{
"checksumSHA1": "5eLUeoF0F5+BYLgztcwUCg/T3dc=",
"path": "gopkg.in/DataDog/dd-trace-go.v1/ddtrace/opentracer",
"revision": "823d51722f66a748804943f6fa6be18776073b82",
"revisionTime": "2019-01-24T09:21:27Z"
},
{
"checksumSHA1": "6Q+wVAvbgJzvcE+k8ofJlPECDko=",
"path": "gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer",
"revision": "823d51722f66a748804943f6fa6be18776073b82",
"revisionTime": "2019-01-24T09:21:27Z"
},
{ {
"checksumSHA1": "fALlQNY1fM99NesfLJ50KguWsio=", "checksumSHA1": "fALlQNY1fM99NesfLJ50KguWsio=",
"path": "gopkg.in/yaml.v2", "path": "gopkg.in/yaml.v2",
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment