Proving Reserves Exist on the Blockchain
This is extremely simple. The exchange will simply publish the public keys for each wallet where balances are stored. This address forms a link to a third party which can be used to validate the holdings, however more advanced users can also check independently.
Privacy Concerns
Some exchanges report concerns over user privacy, since it's possible to trace transactions to and from those addresses. User privacy is certainly an important consideration. All information is already publicly available on the blockchain, and for most large exchanges, advanced blockchain analysis can already determine which wallets most likely belong to each exchange. While most people don't have the knowledge and ability to perform this analysis on their own, cybercriminals, governments, and other organizations typically have access to such talent. The privacy afforded by an exchange not confirming ownership of public keys is more of a facade, and might actually give a false sense of privacy. Truly privacy conscious users should use privacy coins and/or set up new wallets for each transaction.
Security Concerns
Some have suggested that this could impact the security of the exchange. It's true that this provides greater certainty for an attacker as to the exact balance they could obtain through a successful hack operation. Rather than approximate by the size of the exchange and public volume reports, or perform detailed blockchain analysis, a hacker would know the exact amount definitively with much less effort. If the amount is high, that might make a hacker more likely to attempt a break-in, so an exchange with weak security would fall victim to a hack sooner.
The hacker could also see the backend wallet architecture. That might get some clues as to when funds are moved around. It's worth noting that this information is all on the blockchain, so it's already public, however the hacker may not know which wallets belong to the exchange with certainty (as opposed to an educated guess). Either way, one should hope there aren't any exploitable secrets hiding here depending on such a flimsy layer of obscurity.
It's important to understand the difference between a public and private key. While the private key would allow spending of money and must remain secret, the public key is designed to be public. The public key is the same key that you give to someone if you'd like them to pay you. It's a fundamental part of asymmetric cryptography that revealing the public key doesn't assist with finding the private key. If this wasn't true, all balances in the blockchain would be hackable, since all public keys are already part of each block.