A VM is nominally supposed to prevent code in the guest from escaping to the host. So when such an escape is possible, then it is due to a security hole in the VM implementation. Some such holes have been found in various VM engines.
See for instance Cloudburst, an exploit described in 2009. In this case, the video emulation is instrumental to the escape, because the engine's goal of providing fast video emulation implies using some internal sharing of resources between host and guest, and this it at odds with the security goal of the isolation layer. Modern computers are complex systems, so full virtualization is also complex, and it is rather unavoidable such holes of that kind happen.
To be fair, holes which allow true guest escape are rather rare. Good maintenance (promptly applying security fixes from the vendor) for a VM engine is a reasonably low-risk strategy. Note, though, that the same cannot be said for information leaks: a VM can learn things on what happens in other VM, or in the host, through timing attacks. It has been demonstrated (in lab conditions) for encryption algorithms: one VM could learn the secret key used by another VM, because that other VM was using a table-based AES implementation, and was running on the sister core of the attacker's VM (thus with a shared L1 cache).
Guest escape can be facilitated by the configuration, though. For instance, when the host shares part of its filesystem with the guest, then, by definition, the guest has access to part of the host filesystem. Similarly, host and guest often share a more or less virtual network, thus remotely exploitable holes on the host are in range from the guest.
