Followup to https://golang.org/cl/197997

If you know the number of elements, you don't need append at all.
Either use append to grow, or allocate and index. Here we choose
number 2.

Change-Id: Ic58637231789640ff7b293ece04a95a8de7ccf8f
Reviewed-on: https://go-review.googlesource.com/c/go/+/198097Reviewed-by: Ian Lance Taylor <iant@golang.org>