Click for your Flying Doctor
Deleting a file doesn't delete it. Imagine a town full of buildings - all the buildings are files on your computer. When you 'delete' a file you are just putting a sign up saying the building is empty and the site is available for redevelopment. At some random point in the future, the computer might knock the house down and build a new one. Until then, the building is still there.
In the meantime you can go into the recycle bin and restore the file, you are just occupying the house again.
When you empty the recycle bin, you knock the house down but the foundations and rubble are still there. So someone could come along and find out what was there, a bit like Time Team on the TV.
Only when your computer saves a file to the same place does it clear the site and build a new house, deleting all traces of the old one.
You can get file wiping software, which is a bit like a bulldozer and clears all the sites marked as ready for redevelopment.