How do I delete image from custom attribute?

How do I delete the image selected for a custom attribute? I want to switch to use variation image but I notice that the image is still there.

For example, if I switch from variation image back to image for the custom attribute, the originally selected image returns. I even tried to delete the custom attribute and then add it back but the image still re-appears.

Is there a way to permanently delete the image?

Thanks,

Paul