Similar to ArgPassSpec.  Previously we kept them on the CLFunction,
but there are places where we want to specify a certain parameter
signature without having a clfunction available.