PyCharm remote interpreter and local plot?

I'm using PyCharm with a remote interpreter. My code is showing a scatter figure using matplotlib. I can see the figure normally and interact with (zoom and rotate), but the interaction is too slow. I think this is because it's done through the SSH X-server.

My question is how to manually make the figure interactions graphically (using mouse, like zooming, panning, rotation, ...) execute faster? I think this should be via allowing the figure interactions to happen on local machine (machine running PyCharm), not the remote server.

One workaround is to save a .fig file, then copy it to my local machine, then load it. But I don't prefer this workaround.

Thank you.

#python