There are a number of suggested reasons for this obsession. This article suggests being shamed by the public gaze out of loitering and into working has an effect, and also that the short bursts of time writers stay (typically a couple of hours) has an effect similar to the Pomodoro technique.
Research has shown that coffee shops can also be stimulating to creativity. It's not the caffeine, but the noise levels. Too much noise can be really distracting - like my downstairs neighbour who's currently blasting the TV loud enough I can hear it through my floor. Too little noise and suddenly the ticking clock in the next room is the herald of your writing time slipping away. Quiet can be intimidating. Just enough - like coffee shop chatter - can function as white noise, happily placing the writer in the Goldilocks zone between distraction and fear.
Unfortunately there are no decent coffee shops in my town. Not one. Because I need a certain level of background noise to get anything done, I've started using a website called Coffitivity. As the name suggests, it plays a coffee shop soundtrack (there's a choice of three). The chatter is just quiet enough that you can't hear what anyone is saying, which is a pitfall of real coffee shops (I got no writing done the day I ended up a table over from three women discussing their friends who work in the internet sex industry).
To be fair, any white noise would probably have the same effect. I quite like the idea of swapping my imitation coffee shop for an imitation beach, or rainstorm, so I'll probably seek some out. In the meantime I'll keep listening to Coffitivity's "Morning Murmer" track and hoping that someone finally opens a proper coffee shop in this town.