The VLC plugin for Mozilla (and ActiveX) has been exposing video controls to Javascript for as long as it existed. Furthermore (correct me if I'm wrong) I believe Totem mimics VLC. So not quite outside the reach of Javascript really. In fact, early versions of then "Google Video", way before the YouTube buyout, used the VLC plugin.
That most commercial sites use Flash is a different problem. Almost only free/libre-leaning web sites use anything else. It all boils down to Flash having a larger market coverage of desktop computers.