Once you have installed your Joomla site onto your server, the first thing that you would usually do is start customising it and if this is your first every Joomla site customisation, you'll be either looking for a cheap template to start with, one of the default core templates that come with Joomla or you will be looking for a few free templates that are available for Joomla 3.
Good free templates for Joomla 3.1.x and onwards are hard to come by but we've put together a nice collection and resource here that will make your life a little easier when starting out.
A lot of people revert to stealing or illegally downloading templates from torrent websites. I highly discourage this practise, not because you're bypassing the paying mechanism of the template providers but rather these templates from torrent websites are known to have viruses, malware and other nasties in them. So best to avoid these unless you want to support the hackers.
SiteGround's Free Template Library
First off, our good friends at SiteGround have a huge list of free templates for your Joomla 2.5 AND Joomla 3 website. Lots to choose from, all free and all ready for you to use.
http://www.siteground.com/joomla-templates.htm#joomla3
A great collection there and the perfect place for a beginner to start.
YouJoomla
These guys we've reviewed in the past in other template review blog posts and they have a few really nice free templates.
http://www.youjoomla.com/free-joomla-templates.html
At the time of writing this there are only two free templates from YouJoomla but again, if this is your very first Joomla website, these two templates are very flexible in what you can do and customise in regards to the content and layout.
Bow Themes - Free Mobile App Template
If you have a mobile app that you are promoting and wanting to use Joomla to do so, the team over at Bow Themes have put together a free mobile app template just for that reason. Looks great and of course it is free.
Well worth a look at if you're using Joomla just for that reason. These templates are also built on the T3 framework as well and if you have been following the podcast or my other blog posts you'll know that I'm quite a fan of that framework.
ZooTemplates Free Joomla Template Gallery
Lastly, ZooTemplates have a little gallery of free templates to choose from as well. Another great place to start when trying out some free templates.
As you can see there are plenty of free templates that are out there for you Joomla website. It is just a matter of knowing where to look at figuring out which template providers to go with. Each may also use a different framework which sits on top of Joomla and you will also have to learn and adapt to as well. Something else to keep in mind.
Check out Joomlart for some awesome responsive Joomla templates.