Arguably they were the second. The LG Prada, a touch-screen phone, was
released three months before the iPhone was announced.
True, the iPhone uses some interesting touchscreen-based UI tricks (e.g.
pinch), but these two were invented by others, there was a video on youtube
showing an academic research demoing these techniques over two years before
the iPhone debuted.
Apple were the first to successfully create a product based on existing
techniques, not the first to invent those techniques. It's to Apple's credit
that they can exploit known techniques and the talent of their developers to
create great products; Microsoft and Google, despite employing similarly
skilled developers, have failed to create next-gen products (MS never got
beyond Windows+Office, Google has yet to get beyond Search+Ads).
The thing is though, ability to execute is not enough for a patent. It must
be new, innovative, and not at all obvious. None of these are true of that