[This is a methodological appendix to my post on how the Times reports on second homes in the Hudson Valley. For the original post, click here. To browse the 233 articles with references to second homes, click here. For general analysis and links to the entire NYT/HV archive, click here.]
Studying references to second homes involves the use of content analysis, a method of systematically discerning and measuring characteristics of textual content. Using this method, my research assistant Neil Kotru Gode and I divvied up all the articles in our n=694 archive of New York Times articles about the Hudson Valley, and read them for particular phrases and written statements that conveyed what we decided counted as “references to second homes.” This process corresponds to what media scholars call latent coding. Whereas competent readers can determine, say, the year a news article was published or the newspaper section it was published in with a fairly simple, relatively non-interpretive set of steps (corresponding to manifest coding, which we used in Part 1 of this study), latent coding involves looking for specific meanings conveyed by countless combinations of words. This requires an intense exercise of interpretation, filtered through the idiosyncratic sensibility of a given reader who must overcome their normal tendency toward “I know it when I see it.” Consequently, content analysis of latent coding requires two or more people to confirm the interpretations and assign measurements a quantitative degree of reliability.
What counts as a reference to a second home?
“I know it when I see it” is only the start of the process. Through reading the NYT/HV archive and becoming accustomed to the many ways that the Times wrote about second homes, Neil and I developed a set of rules for observing their references in the text.
As we conceptualize it, second homes include formally owned properties, vacation rentals, or other places for personal residence of at least “seasonal” stay (i.e., not a week’s stay at an Airbnb or a summer month at a writer’s retreat resort). Second homes may be acquired for recreational purposes, as economic investments, for sentimental-familial reasons, or some combination. They aren’t necessarily experienced as travel-based recreation; for some profiled subjects, second homes are in fact an alternative to travel — at least if that involves choosing new destinations or unfamiliar lodging to visit.
We looked for variations on phrases like “weekenders,” “weekend house,” “vacation home,” “country house,” and statements about people “dividing their time between” different residences; once these phrases were spotted, we read further to verify the reference to second homes. We don’t presume NY Times writers and editors are so artless as to use only such stock phrases, so we read each article for relevant implications. For instance, an article might report on a period of home constructions or major renovations that continued well after initial purchase or commencement, respectively; we would count those as references to a second home. To be clear: we coded for references to second homes in an archive containing articles about the Hudson Valley. Two or three times, we came across articles that included the Hudson Valley in reporting on multiple regions where the second home reference was made to a non-HV region. We counted these too.
These rules constitute a conservative standard for textual observation that means we excluded a lot of articles from our archive about Hudson Valley places that are well known as second-home locations. For instance, as a resident of Rhinebeck, I know the town has a significant second-home population, but neither Neil or I could find any explicit references to second homes in this article (which frames Rhinebeck more as “a popular destination for day-trippers up from New York City, as well as a community where retirees and younger urbanites choose to relocate”), so we didn’t count it. As well, we didn’t count articles of clear interest to Hudson Valley second-home owners. For example, this article about a boutique in Hudson is likely to excite a lot of the city’s second-home population, but because we didn’t see references to second homes, we didn’t count it.
Intercoder reliability
After Neil coded his subset of NYT/HV articles (n=256) and I coded mine (n=438), we then selected a 10% sample of each other’s subset and coded them independently to confirm or identify discrepancies in how the two of us coded references to second homes. We met to discuss the discrepancies, correct them in the archive, and, where needed, to elaborate new clarifying rules for textual observations. After two stages of non-overlapping samples, we arrived at a final discrepancy rate of 5.7% in our coding of second home references. This statistic isn’t the same as a margin of error, nor does it necessarily suggest the entire n=694 archive includes 5.7% of erroneously coded articles. (It’s likely less than that, because Neil and I corrected two stages of samples, or 20% of the entire archive from which we draw our figures.)