git clone -b <branch> <remote_repo>

Example:

git clone -b my-branch git@github.com:user/myproject.git

With Git 1.7.10 and later, add --single-branch to prevent fetching of all branches. Example, with OpenCV 2.4 branch:

git clone -b opencv-2.4 --single-branch https://github.com/Itseez/opencv.git