At some stage, your site is going to have need of some code that existing modules cannot fulfill.

However, do search the repository and ask on the forums if it's been done before. Often the modules are named in ways that are non-obvious to what you want them to do.

Next, scan the PHP snippets (both page and block) to see if there are any ideas there. But be aware that most 'snippet' code is of 'proof of concept' quality only and is often inefficient or deficient in several ways.

Most importantly, using inline PHP code in a block or node body is a painful way to program and test. The sooner you bite the bullet and start your own custom tweaks module, the better.

Make your own module

Your own module can implement as many or as few of the available hooks as you like. Start small.

A big advantage of putting your code in a module is that it's easier to version, diff, backup and deploy to other sites, eg staging to live. This is much trickier when you've created a PHP node or block, and you then have to copy & paste the code from one place to anther using the Drupal UI. Add to this the risk that a mistake in a PHP block can crash the entire site.
Moving the code into a module - even if it's only a single-use thing - starts you thinking about configurations and re-use. Hard-coding a taxonomy ID for example looks obvious on a PHP page, (see the snippets) but IDs may be different between different sites (dev|staging|live) and writing the same hard-coded number into a .module file would be a mistake.

A .module file should avoid ever producing any actual HTML

As there are core theme functions, such as l(), theme_item_list(), and even theme_table() to use.
The exception is actual theme_functions defined by the module itself, which are designed to be over-ridden.This is a general guideline for module development. Write-once custom code can take a few short-cuts