There were a couple of issues here:

1. Because an empty `labels` key was in the params, it was blowing up as
   ActiveRecord's expecting a board's labels to be an association.
2. The service wouldn't allow removing labels, assignee, etc.