It's not against your consent if you know in advance the option is there when you contribute.
If you don't want them to have the option then don't contribute. This isn't much different
than any project which requires copyright assignment as that's essentially what you are doing.
It's not as if the GPL code can be "taken back" as once the license is granted the code is out
there to be relicensed.