I just read AlephOne's paper on smashing the stack, and implemented it on my machine (Ubuntu 12.04), it was a bit old so had to take a lot of help from the internet but I enjoyed it.
Now, I want to learn how is that prevented in real life.
It would be great if you can suggest me some papers or online resources that demonstrate how it is prevented.
There are two primary techniques for mitigating stack based buffer overflows.
The first technique is attacking the problem at the roots. Buffer overflows occur when a program tries to write outside the bounds of a data structure. The most effective way to stop a buffer overflow is to stop this from occurring. Use a programming language, be it Java, Python, Ruby, that will manage the memory for you. If you must use C, ensure that you perform proper checks before writing to a buffer.
The second technique involves mitigating the damage a buffer overflow can cause. Techniques like stack canaries, DEP and ASLR fall under this category. They work to limit the damage a buffer overflow can cause by making it more difficult for an attacker to execute arbitrary code after an overflow. Such techniques can be defeated. For more information about DEP and ASLR, see this question.
Just a small note:
Since, you were using Ubuntu 12.04 for the testing, you must have given your system a command (assuming that you took a lot of help from the internet and this was the first time your were trying to smash the stack):
echo 0 > /proc/sys/kernel/randomize_va_space
Or some related command. This command basically disables the ASLR (mentioned by Terry) on the system. By default, ASLR is an enabled feature on the kernel versions 2.6 and above.
(Actually wanted to post this as a comment, but was not able to do so because of Reputation issues :))
