Tableau Desktop client has to talk to Tableau server

Tableau architecture shows that Desktop connects to tableau server (using gateway) and then to data server which are on tableau server. My question is do tableau client has to talk to tableau server? Can't I just install tableau desktop and connect to required database (even hosted on cloud)?

Regards, G.

#tableau