You err because you want PowerShell to act exactly the way POSIX shells do. PowerShell is actually a bit more structured and it may require you to do things in a little different way. The advantage is that many of the error-prone constructs in bash are avoided.
For your example try this:
And then try
PowerShell actually integrates quite nicely with external programs; it does so even retaining (not re-interpreting) arguments.
In your example you told PS to pass a string containing "arg1 arg2" to an external program - which it did. You assume that the entire commandline is turned into text and re-interpreted (the way of POSIX shells).
As you can see above, PowerShell understands arrays and will readily pass the array items as discrete arguments. It's not harder - just a little different and a lot more robust and avoids the risk of injection vulnerabilities.