If the answer is not understandable, then obviously points are subtracted, but even then you don't subtract for poor english, you subtract for the fact that the student did not succeed in demonstrating knowledge of the subject area.
So yes, obviously. If your English is such that it prevents you from communicating clearly and precisely about the subject-area, then you will indeed get a poorer grade, and that's perfectly fair. In that case you're not being punished for poor English as such. You're being given a lower grade because you did not succeed in demonstrating that you deserve the higher grade. (you use English for this demonstration)
I also picked examples specifically to not fall under this. If I write an answer about optics, and write about refraction of ligth instead of refraction of light, then it's perfectly clear what I mean, and I should get full credits. (not "minus a few percent")
Most minor mistakes in spelling and/or grammar fall in this category, for example in this very thread, we have one commenter who says he suffered because of differences between New Zealand and US english. Giving someone a lower grade in physics, math or history on account of "uses New Zealand spelling" is nonsense.