The Google Checkout module has been updated in the new v4.2.1 version of WHMCS and the release notes have a slight omission; the Google Checkout module no longer works out of the box with the default settings in your Google Checkout account.
In order to fix this, go to Settings and then Integration and then under “Shopping cart post security” untick the box next to “My company will only post digitally signed carts”.
According to WHMCS, this is due to this feature causing problems. It is probably worth reading the Google Checkout knowledge base article about the potential security holes that disabling this feature will introduce.
WHMCS will check the amount on the XML Google send back that it uses to mark the appropriate invoice(s) as paid, so you won’t need to manually verify the details as the knowledge base article suggests; if someone fiddles with the data sent you send to Google from your invoice screen then the invoice will only be marked as partially paid.