scp -r user@your.server.example.com:/path/to/foo /home/user/Desktop/

By not including the trailing '/' at the end of foo, you will move the directory itself (including contents), rather than only the contents of the directory.

From man scp (See online manual)

-r Recursively copy entire directories