Since the emergence of affiliate marketing, there has been little control over affiliate activity. Unscrupulous affiliates have used spam, false advertising, forced clicks (to get tracking cookies set on users' computers), adware, and other methods to drive traffic to their sponsors. Although many affiliate programs have terms of service that contain rules against spam, this marketing method has historically proven to attract abuse from spammers.
Always disclose your affiliate relationship. Most visitors will probably understand that graphic ads will lead to your getting paid, but if you write a review or use an in-text link as a recommendation, you want your readers to know that may lead to compensation as well. This ensures you retain transparency and trust with your readers, but also, it's required by the FTC's endorsement rules.