Completely agree. I think we could see from that flashback to the hotel room that Philip at least found Elizabeth physically attractive from very early on. Well, I guess we know that from when he told her his impressions of their first meeting too, but in any case, I think Philip wouldn't have seen the situation as that bad, initially. The US wasn't such a bad place to get "stationed" and he was married off to a really cute girl! Given the adaptability of his personality, there was little problem. I think something closer to "love" probably took longer to form, but there seemed to be chemistry and attraction there for Philip from the start.