Once you have established that they don't match, you still have a problem -- what to do about it. Often, the certificate may merely be assembled incorrectly. When a CA signs your certificate, they send you a block that looks something like
-----BEGIN CERTIFICATE----- MIIAA-and-a-buncha-nonsense-that-is-your-certificate -and-a-buncha-nonsense-that-is-your-certificate-and- a-buncha-nonsense-that-is-your-certificate-and-a-bun cha-nonsense-that-is-your-certificate-and-a-buncha-n onsense-that-is-your-certificate-AA+ -----END CERTIFICATE-----
they'll also send you a bundle (often two certificates) that represent their authority to grant you a certificate. this will look something like
-----BEGIN CERTIFICATE----- MIICC-this-is-the-certificate-that-signed-your-request -this-is-the-certificate-that-signed-your-request-this -is-the-certificate-that-signed-your-request-this-is-t he-certificate-that-signed-your-request-this-is-the-ce rtificate-that-signed-your-request-A -----END CERTIFICATE----- -----BEGIN CERTIFICATE----- MIICC-this-is-the-certificate-that-signed-for-that-one -this-is-the-certificate-that-signed-for-that-one-this -is-the-certificate-that-signed-for-that-one-this-is-t he-certificate-that-signed-for-that-one-this-is-the-ce rtificate-that-signed-for-that-one-this-is-the-certifi cate-that-signed-for-that-one-AA -----END CERTIFICATE-----
except that unfortunately, they won't be so clearly labeled.
a common practice, then, is to bundle these all up into one file -- your certificate, then the signing certificates. But since they aren't easily distinguished, it sometimes happens that someone accidentally puts them in the other order -- signing certs, then the final cert -- without noticing. In that case, your cert will not match your key.
You can test to see what the cert thinks it represents by running
openssl x509 -noout -text -in yourcert.cert
Near the top, you should see "Subject:" and then stuff that looks like your data. If instead it lookslike your CA, your bundle is probably in the wrong order; you might try making a backup, and then moving the last cert to the beginning, hoping that is the one that is your cert.
If this doesn't work, you might just have to get the cert re-issued. When I make a CSR, I like to clearly label what server it's for (instead of just ssl.key or server.key) and make a copy of it with the date in the name, like mydomain.20150306.key etc. that way they private and public key pairs are unlikely to get mixed up with another set.