So I started searching around, and quickly found the removeStylesheet() method in the sfWebResponse class. This didn't work however, when I called this in my controller. Googling around a bit more I found this mailinglist discussion where Alexander Deruwe describes exactly what I need to do! Instead of removing it programmatically in the controller, I can just do it in my module view.yml file. So my view.yml of the module now looks like this:
stylesheets: [-main, formdefault.css]
By negating a stylesheet that was added on a higher level of configuration, it is removed again from the set of stylesheets, and not added to the actual response. Thanks Alexander!