I think that feature came out with one of the earlier Android updates, so it might not be on some of the first Android phones, but has been on them for quite some time. I'm not 100% on that though, but the feature has been on any Android phone I've ever used.

My non-technical understanding is that in Android apps can run in the background, but the OS chooses when to close them, not the user, so you're never 100% sure that the app you want to go back to has continued running the entire time. For me personally, the bigger issue is that sometimes I want to minimize an app, so that i can switch back to it in the same place I was in, but other times I want to close the app so that when I restart it starts fresh (say at the browsers home page instead of the page you were last on). Android hasn't made it easy to do the second of these, there is basically just the minimize for most programs, and my understanding is that manually quitting a program can screw things up a bit. Not sure if the swipe to close thing in ICS really close the application, or just removes it from the recently used list.