Commit 72a73198 authored by Andrew Gerrand's avatar Andrew Gerrand

goinstall: documentation for new remote repository behavior and tweaks

R=rsc, julian
CC=golang-dev
https://golang.org/cl/4642049
parent 6a2e2432
...@@ -5,7 +5,8 @@ ...@@ -5,7 +5,8 @@
/* /*
Goinstall is an experiment in automatic package installation. Goinstall is an experiment in automatic package installation.
It installs packages, possibly downloading them from the internet. It installs packages, possibly downloading them from the internet.
It maintains a list of public Go packages at http://godashboard.appspot.com/package. It maintains a list of public Go packages at
http://godashboard.appspot.com/package.
Usage: Usage:
goinstall [flags] importpath... goinstall [flags] importpath...
...@@ -41,9 +42,22 @@ Another common idiom is to use ...@@ -41,9 +42,22 @@ Another common idiom is to use
to update, recompile, and reinstall all goinstalled packages. to update, recompile, and reinstall all goinstalled packages.
The source code for a package with import path foo/bar is expected The source code for a package with import path foo/bar is expected
to be in the directory $GOROOT/src/pkg/foo/bar/. If the import to be in the directory $GOROOT/src/pkg/foo/bar/ or $GOPATH/src/foo/bar/.
path refers to a code hosting site, goinstall will download the code See "The GOPATH Environment Variable" for more about GOPATH.
if necessary. The recognized code hosting sites are:
By default, goinstall prints output only when it encounters an error.
The -v flag causes goinstall to print information about packages
being considered and installed.
Goinstall ignores Makefiles.
Remote Repositories
If a package import path refers to a remote repository, goinstall will
download the code if necessary.
Goinstall recognizes packages from a few common code hosting sites:
BitBucket (Mercurial) BitBucket (Mercurial)
...@@ -72,7 +86,6 @@ if necessary. The recognized code hosting sites are: ...@@ -72,7 +86,6 @@ if necessary. The recognized code hosting sites are:
import "launchpad.net/~user/project/branch" import "launchpad.net/~user/project/branch"
import "launchpad.net/~user/project/branch/sub/directory" import "launchpad.net/~user/project/branch/sub/directory"
If the destination directory (e.g., $GOROOT/src/pkg/bitbucket.org/user/project) If the destination directory (e.g., $GOROOT/src/pkg/bitbucket.org/user/project)
already exists and contains an appropriate checkout, goinstall will not already exists and contains an appropriate checkout, goinstall will not
attempt to fetch updates. The -u flag changes this behavior, attempt to fetch updates. The -u flag changes this behavior,
...@@ -84,19 +97,42 @@ named "release". If there is one, it uses that version of the code. ...@@ -84,19 +97,42 @@ named "release". If there is one, it uses that version of the code.
Otherwise it uses the default version selected by the version control Otherwise it uses the default version selected by the version control
system, typically HEAD for git, tip for Mercurial. system, typically HEAD for git, tip for Mercurial.
After a successful download and installation of a publicly accessible After a successful download and installation of one of these import paths,
remote package, goinstall reports the installation to godashboard.appspot.com, goinstall reports the installation to godashboard.appspot.com, which
which increments a count associated with the package and the time increments a count associated with the package and the time of its most
of its most recent installation. This mechanism powers the package list recent installation. This mechanism powers the package list at
at http://godashboard.appspot.com/package, allowing Go programmers http://godashboard.appspot.com/package, allowing Go programmers to learn about
to learn about popular packages that might be worth looking at. popular packages that might be worth looking at.
The -dashboard=false flag disables this reporting. The -dashboard=false flag disables this reporting.
By default, goinstall prints output only when it encounters an error. For code hosted on other servers, goinstall recognizes the general form
The -v flag causes goinstall to print information about packages
being considered and installed. repository.vcs/path
as denoting the given repository, with or without the .vcs suffix, using
the named version control system, and then the path inside that repository.
The supported version control systems are:
Bazaar .bzr
Git .git
Mercurial .hg
Subversion .svn
For example,
import "example.org/user/foo.hg"
denotes the root directory of the Mercurial repository at example.org/user/foo
or foo.hg, and
import "example.org/repo.git/foo/bar"
denotes the foo/bar directory of the Git repository at example.com/repo or
repo.git.
Goinstall does not use make. Makefiles are ignored by goinstall. When a version control system supports multiple protocols, goinstall tries each
in turn.
For example, for Git it tries git://, then https://, then http://.
The GOPATH Environment Variable The GOPATH Environment Variable
......
...@@ -8,6 +8,7 @@ package main ...@@ -8,6 +8,7 @@ package main
import ( import (
"exec" "exec"
"fmt"
"http" "http"
"os" "os"
"path/filepath" "path/filepath"
...@@ -32,12 +33,6 @@ func maybeReportToDashboard(path string) { ...@@ -32,12 +33,6 @@ func maybeReportToDashboard(path string) {
} }
} }
type host struct {
pattern *regexp.Regexp
protocol string
suffix string
}
// a vcs represents a version control system // a vcs represents a version control system
// like Mercurial, Git, or Subversion. // like Mercurial, Git, or Subversion.
type vcs struct { type vcs struct {
...@@ -59,9 +54,10 @@ type vcs struct { ...@@ -59,9 +54,10 @@ type vcs struct {
defaultHosts []host defaultHosts []host
} }
type vcsMatch struct { type host struct {
*vcs pattern *regexp.Regexp
prefix, repo string protocol string
suffix string
} }
var hg = vcs{ var hg = vcs{
...@@ -97,7 +93,7 @@ var git = vcs{ ...@@ -97,7 +93,7 @@ var git = vcs{
log: "show-ref", log: "show-ref",
logLimitFlag: "", logLimitFlag: "",
logReleaseFlag: "release", logReleaseFlag: "release",
check: "peek-remote", check: "ls-remote",
protocols: []string{"git", "https", "http"}, protocols: []string{"git", "https", "http"},
suffix: ".git", suffix: ".git",
defaultHosts: []host{ defaultHosts: []host{
...@@ -147,27 +143,56 @@ var bzr = vcs{ ...@@ -147,27 +143,56 @@ var bzr = vcs{
var vcsList = []*vcs{&git, &hg, &bzr, &svn} var vcsList = []*vcs{&git, &hg, &bzr, &svn}
func (v *vcs) findRepo(prefix string) *vcsMatch { type vcsMatch struct {
for _, proto := range v.protocols { *vcs
for _, suffix := range []string{v.suffix, ""} { prefix, repo string
repo := proto + "://" + prefix + suffix }
out, err := exec.Command(v.cmd, v.check, repo).CombinedOutput()
if err == nil { // findHostedRepo checks whether pkg is located at one of
return &vcsMatch{v, prefix + v.suffix, repo} // the supported code hosting sites and, if so, returns a match.
func findHostedRepo(pkg string) (*vcsMatch, os.Error) {
for _, v := range vcsList {
for _, host := range v.defaultHosts {
if hm := host.pattern.FindStringSubmatch(pkg); hm != nil {
if host.suffix != "" && strings.HasSuffix(hm[1], host.suffix) {
return nil, os.NewError("repository " + pkg + " should not have " + v.suffix + " suffix")
} }
printf("find %s: %s %s %s: %v\n%s\n", prefix, v.cmd, v.check, repo, err, out) repo := host.protocol + "://" + hm[1] + host.suffix
return &vcsMatch{v, hm[1], repo}, nil
} }
} }
}
errorf("find %s: couldn't find %s repository\n", prefix, v.name) return nil, nil
return nil
} }
func findRepo(pkg string) *vcsMatch { // findAnyRepo looks for a vcs suffix in pkg (.git, etc) and returns a match.
func findAnyRepo(pkg string) (*vcsMatch, os.Error) {
for _, v := range vcsList { for _, v := range vcsList {
i := strings.Index(pkg+"/", v.suffix+"/") i := strings.Index(pkg+"/", v.suffix+"/")
if i >= 0 { if i < 0 {
return v.findRepo(pkg[:i]) continue
}
if !strings.Contains(pkg[:i], "/") {
continue // don't match vcs suffix in the host name
}
if m := v.find(pkg[:i]); m != nil {
return m, nil
}
return nil, fmt.Errorf("couldn't find %s repository", v.name)
}
return nil, nil
}
func (v *vcs) find(pkg string) *vcsMatch {
for _, proto := range v.protocols {
for _, suffix := range []string{"", v.suffix} {
repo := proto + "://" + pkg + suffix
out, err := exec.Command(v.cmd, v.check, repo).CombinedOutput()
if err == nil {
printf("find %s: found %s\n", pkg, repo)
return &vcsMatch{v, pkg + v.suffix, repo}
}
printf("find %s: %s %s %s: %v\n%s\n", pkg, v.cmd, v.check, repo, err, out)
} }
} }
return nil return nil
...@@ -193,27 +218,29 @@ func download(pkg, srcDir string) os.Error { ...@@ -193,27 +218,29 @@ func download(pkg, srcDir string) os.Error {
if strings.Contains(pkg, "..") { if strings.Contains(pkg, "..") {
return os.NewError("invalid path (contains ..)") return os.NewError("invalid path (contains ..)")
} }
dashpath := pkg dashReport := true
var m *vcsMatch m, err := findHostedRepo(pkg)
for _, v := range vcsList { if err != nil {
for _, host := range v.defaultHosts { return err
if hm := host.pattern.FindStringSubmatch(pkg); hm != nil {
if v.suffix != "" && strings.HasSuffix(hm[1], v.suffix) {
return os.NewError("repository " + pkg + " should not have " + v.suffix + " suffix")
}
repo := host.protocol + "://" + hm[1] + host.suffix
m = &vcsMatch{v, hm[1], repo}
}
}
} }
if m == nil { if m == nil {
m = findRepo(pkg) m, err = findAnyRepo(pkg)
dashpath = "" // don't report to dashboard if err != nil {
return err
}
dashReport = false // only report public code hosting sites
} }
if m == nil { if m == nil {
return os.NewError("cannot download: " + pkg) return os.NewError("cannot download: " + pkg)
} }
return vcsCheckout(m.vcs, srcDir, m.prefix, m.repo, dashpath) installed, err := m.checkoutRepo(srcDir, m.prefix, m.repo)
if err != nil {
return err
}
if dashReport && installed {
maybeReportToDashboard(pkg)
}
return nil
} }
// Try to detect if a "release" tag exists. If it does, update // Try to detect if a "release" tag exists. If it does, update
...@@ -232,49 +259,46 @@ func (v *vcs) updateRepo(dst string) os.Error { ...@@ -232,49 +259,46 @@ func (v *vcs) updateRepo(dst string) os.Error {
return nil return nil
} }
// vcsCheckout checks out repo into dst using vcs. // checkoutRepo checks out repo into dst using vcs.
// It tries to check out (or update, if the dst already // It tries to check out (or update, if the dst already
// exists and -u was specified on the command line) // exists and -u was specified on the command line)
// the repository at tag/branch "release". If there is no // the repository at tag/branch "release". If there is no
// such tag or branch, it falls back to the repository tip. // such tag or branch, it falls back to the repository tip.
func vcsCheckout(vcs *vcs, srcDir, pkgprefix, repo, dashpath string) os.Error { func (vcs *vcs) checkoutRepo(srcDir, pkgprefix, repo string) (installed bool, err os.Error) {
dst := filepath.Join(srcDir, filepath.FromSlash(pkgprefix)) dst := filepath.Join(srcDir, filepath.FromSlash(pkgprefix))
dir, err := os.Stat(filepath.Join(dst, vcs.metadir)) dir, err := os.Stat(filepath.Join(dst, vcs.metadir))
if err == nil && !dir.IsDirectory() { if err == nil && !dir.IsDirectory() {
return os.NewError("not a directory: " + dst) err = os.NewError("not a directory: " + dst)
return
} }
if err != nil { if err != nil {
parent, _ := filepath.Split(dst) parent, _ := filepath.Split(dst)
if err := os.MkdirAll(parent, 0777); err != nil { if err = os.MkdirAll(parent, 0777); err != nil {
return err return
}
if err := run(string(filepath.Separator), nil, vcs.cmd, vcs.clone, repo, dst); err != nil {
return err
} }
if err := vcs.updateRepo(dst); err != nil { if err = run(string(filepath.Separator), nil, vcs.cmd, vcs.clone, repo, dst); err != nil {
return err return
} }
// success on first installation - report if err = vcs.updateRepo(dst); err != nil {
if dashpath != "" { return
maybeReportToDashboard(dashpath)
} }
installed = true
} else if *update { } else if *update {
// Retrieve new revisions from the remote branch, if the VCS // Retrieve new revisions from the remote branch, if the VCS
// supports this operation independently (e.g. svn doesn't) // supports this operation independently (e.g. svn doesn't)
if vcs.pull != "" { if vcs.pull != "" {
if vcs.pullForceFlag != "" { if vcs.pullForceFlag != "" {
if err := run(dst, nil, vcs.cmd, vcs.pull, vcs.pullForceFlag); err != nil { if err = run(dst, nil, vcs.cmd, vcs.pull, vcs.pullForceFlag); err != nil {
return err return
} }
} else if err := run(dst, nil, vcs.cmd, vcs.pull); err != nil { } else if err = run(dst, nil, vcs.cmd, vcs.pull); err != nil {
return err return
} }
} }
// Update to release or latest revision // Update to release or latest revision
if err := vcs.updateRepo(dst); err != nil { if err = vcs.updateRepo(dst); err != nil {
return err return
} }
} }
return nil return
} }
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment