According to this post, there is no way to delete swatches yet.
it would be nice to delete colors from the color pallet.
You’re talking about swatches right? Totally! This was our rough v1
release. In our next release we want to make the whole swatches
management more robust.