Undercounting the Homeless 2010


Undercounting the Homeless 2010

Why the City's Annual Street Homeless "Estimate"
Fails to Measure Homelessness Accurately and Misleads the Public

By Patrick Markee, Senior Policy Analyst, Coalition for the Homeless

January 24, 2010

Download the complete briefing paper here

This year, the New York City Department of Homeless Services is conducting its eighth annual HOPE street homelessness survey. And once again, there is mounting evidence that the City's survey and its resulting "estimate" fail to measure homelessness accurately and mislead the general public about the scale of the problem.

In 2004, after the City initial trial HOPE surveys (conducted only in Manhattan, Brooklyn, and Staten Island), Coalition for the Homeless released the report Undercounting the Homeless, which detailed the various ways that the City's survey has an inherently flawed methodology and how it fails to capture large numbers of unsheltered homeless New Yorkers.

The Coalition also warned then and in subsequent years that, by marketing the annual HOPE survey to the news media and using an aggressive public relations strategy both on the night of the survey and when the results are released, the Bloomberg administration is misleading the general public about the scale of the homelessness problem and creating an "undercount" that would only become a source of contention and dispute.

Sadly, the problems identified in our 2004 report have only multiplied. Following is a summary of the major flaws in the HOPE survey, and an analysis of how those flaws serve to mislead the general public.

1. The HOPE survey is an estimate, not a count - a fact that the City's public relations strategy obscures.

The City's HOPE survey is not an actual enumeration or count of New York City's street homeless population. It is, in fact, a statistical estimate based on a single night's survey conducted largely by volunteers who have received only a brief training. Thus, from the manner in which the survey is conducted (see below) to the methods of statistical estimate, the results of the HOPE survey are bound to be plagued by error and mistakes.

For example, the Department of Homeless Services acknowledges that it surveys only a portion of the city's surface area (around 20 percent) and only a portion of subway stations, and that it does not survey subway trains themselves. To conduct the survey, the agency divides areas of the city into "high density" and "low density" zones, based on where it believes homeless people reside, and then sends the majority of survey teams to the "high" density" areas. However, if the designation of an area as "high" or "low density" is incorrect, the agency will end up under-sampling or over-sampling certain areas, thus distorting the final estimate. However, because City officials refuse to release data on the homeless people actually counted during the surveys (see below), it is impossible to determine how accurate the estimate is.

Unfortunately, the City's aggressive public relations strategy obscures the genuine nature of the HOPE survey. For instance, as we saw during last year's mayoral race and during the 2008 presidential campaign, the results of public opinion polls are always reported with a confidence interval, typically presented as "an error rate of plus or minus X percent." In contrast, City officials and press spokespeople have never revealed the "error rate" of the HOPE survey, choosing instead to treat the number as an actual "count" of unsheltered homeless people.

2. The City has refused to reveal how many homeless people are actually counted.

For the last four years, the Department of Homeless Services has refused to reveal how many homeless people were actually counted during the HOPE surveys. Indeed, in 2006 the number of homeless people actually counted by survey teams was higher than the previous year, even though the City produced an estimate saying that the street homeless population had decreased.

3. The HOPE survey fails to count homeless people in non-visible locations.

Even City officials acknowledge that the HOPE survey only attempts to count homeless people is visible locations and in public spaces. This, however, fails to account for the large percentage of homeless people who sleep in out-of-the-way, hidden places - for example, lots, abandoned buildings, alleyways, and virtually any private property. In addition, HOPE volunteers report that survey teams are prohibited from "counting" even in some public locations, like under bridges or along much of the city's waterfront.

Academic researchers from Columbia University and New York University analyzed this phenomenon in a September 2007 study, in which they interviewed hundreds of unsheltered homeless individuals about the places that they actually slept on the night of one of the HOPE surveys. The researchers found that:

• Nearly one-third (32 percent) of unsheltered homeless people in Manhattan slept in non-visible places;
• Nearly half (49 percent) of unsheltered homeless people in the other boroughs slept in non-visible places; and
• Nearly one in seventeen (6 percent) of homeless people in the subway system slept on non-visible places.

The researchers summarized their findings in this way:

"[T]he most striking finding is that a substantial proportion of homeless individuals reported being in places during the HOPE street count where they could not have been seen by the volunteer enumerators. This was especially true of surface sites in the outer boroughs, which provide more hidden places to stay (e.g., abandoned buildings, vehicles, porches, backs of buildings) than are available in Manhattan, where most buildings are flush to sidewalks and fences often prevent access to backs of buildings."

4. The HOPE survey has failed to adjust for survey error.

From the outset, advocates and researchers urged the City to include to adjust its findings when there is evidence of mistakes made by volunteer enumerators - in particular, when the survey teams fail to canvass all of their assigned area, or when teams simply missed counting homeless individuals. Two academic researchers noted:

"...the apparent ease with which some of the counting teams deviated from their instructions. In particular, reports of counters walking entirely past plants typify the practice of discounting, that is, ignoring certain street occupants because ‘they don't look homeless' and instead, according to some tacit guide, approaching only those who do."

Thus, experts urged the City to adopt a "plant-capture" technique in which decoys (people posing as homeless individuals) would be placed in various locations. If some of the decoys are not included in the count, the estimate would then be adjusted to reflect the survey error.

After initially refusing to incorporate the "plant-capture" technique, City officials finally agreed to do so in 2005 and in subsequent years. However, there have been significant disputes between the academic researchers hired to conduct the "plant-capture" adjustment and Department of Homeless Services officials, who in many instances refused to acknowledge survey error and hence to adjust the HOPE estimate upwards.

In May 2007, the New York Times reported:

"A Columbia University professor hired by the city to help conduct an annual estimate of homeless people has disavowed the project, saying its methods cause the number to be understated.

"The professor, Julien O. Teitler, said yesterday that he was withdrawing from the project because he did not think the city's approach would arrive at an accurate figure."

Using the adjustment that Dr. Teitler prepared, the HOPE estimate that year would have resulted in an increase in the estimate of street homelessness, not the decrease produced by Department of Homeless Services officials. Since 2007, the City has moved to secure even more control over the "plant-capture" adjustment, raising even more questions about the accuracy of the HOPE survey.

5. Changing weather and other conditions make it impossible to compare one HOPE estimate with another.

One of the fundamental flaws of the HOPE survey is that, because it is conducted under different conditions each year, it is impossible to compare one year's estimate with another. As research and experience have conclusively shown, weather conditions has a significant impact on the number of unsheltered homeless people and on where they sleep.

This is most especially true of weather. For instance, on the night the HOPE survey was conducted in 2009 (January 26th), the low temperature was 19 degrees F and the high temperature was only 29 degrees F, both well below freezing. However, as the 2010 HOPE survey was being prepared, the low temperature was forecast to be 36 degrees F and high temperature 55 degrees F, both well above freezing. Given the dramatically different weather conditions, it is hard to see how homelessness estimates (much less actual counts) produced on these days can be compared with one another.

Another important factor is what part of the month the survey is conducted. Many homeless people, like other poor people, rely on monthly disability benefits like Supplemental Security Income (SSI) or Social Security Disability Insurance (SSDI). As long experience has shown, many homeless recipients of these benefits use them to obtain temporary accommodations for the early part of each month, but, when the meager benefits are exhausted, are forced to turn again to the streets until the next payment arrives. Thus, it is predictable that there will be more homeless individuals on the streets (and in shelters) in the latter part of the month than at the beginning of the month.

Nevertheless, DHS conducted its 2005 HOPE survey on the night of March 7th (in part due to a blizzard on the initially scheduled date), while most other HOPE surveys have been conducted late in the month, again raising questions about the validity of comparing one-years estimate with another's.

6. The City's claim that homelessness has decreased during the recession is not credible.

Last year, in the midst of the worst economic downturn since the Great Depression, City officials claimed that the unsheltered homeless population had declined for the fifth consecutive year. Even more startling, City officials claimed that the number of unsheltered homeless New Yorkers was almost half of what it was in 2005.

The City's remarkable claim is simply not credible. Moreover, it flies in the face of every other verifiable data source about the dramatic rise in homelessness and poverty since the recession began. Indeed, as the Coalition has documented extensively, indisputable City data about homeless shelter populations show dramatic increases in homelessness among both families and single adults. In addition, organizations like the New York City Coalition Against Hunger documented increased use of emergency food programs by homeless people. And across the United States, cities and states have reported large increases in homelessness, both on the streets and in shelters.