Notions of rhetoric vs. reality, intent vs. implementation, design vs. local variations are integral to the notion of “evaluability assessment.”  In a long research career, if I learned one way of seeing that enhanced peripheral vision, it was that the first step toward evaluating anything whatsoever—initiatives to combat drug addiction, AA meetings, marriage counseling, or Santa Claus as social change agent—involves comparing the grand design vs. actual implementations. 

Coming from many disciplines, the inventors of  “evaluability assessment” applied the concept to everything from overhauling the plumbing system of an old house to implementing new government regulations.  “An evaluability assessment is a systematic process that helps to identify whether a program is in a condition to be evaluated, and whether an evaluation is justified, feasible and likely to provide useful information.” I learned about “EA,” as we called it, by sitting at the knees of its inventors as I struggled with how to evaluate an innovation that then existed in 10 state and 4 Federal adult prisons: co-corrections. That is, prisons housing both men and women and allowing for some level of interaction between them. 

At best, coed prisons were driven by the conviction that separating men and women during confinement had severe corrosive effects manifested in sexual and other violence.  Keeping men and women together was expected to minimize such effects. What we learned by visiting 6 of state and 4 Federal coed prisons in 1977 was that only a couple were begun as true therapeutic interventions.  The rest came about to utilize space more fully, especially at the state level, where women’s prisons had surplus bed space. Men were brought in to fill empty beds.  In theory, this opened up additional programmatic options for women. The biggest downside for women was that most had ended up in prison due to problematic relationships with men.  Most had earned their livings as sex workers to support drug habits.  So, the argument goes, for most women being housed with men was a horrible idea, totally incompatible with rehabilitation. 

As we visited the prisons and talked with everyone we could, including prisoners, five different designs for what a coed prison was intended to accomplish emerged.  In nearly every prison, two or three different models were being implemented simultaneously.  At numerous points, the different designs contradicted each other in how they were to be implemented.  For example, although reducing sexual violence among male prisoners and opening up a wider range of programmatic options were considered positive aspects of coed prisons, it was difficult to achieve either because restrictions were typically placed on movement that wouldn’t have been needed in a single-sex prison.  Therefore, in practice, women often couldn’t gain access to programs available to men at an institution.  In addition, any attempt to create a more “natural” environment was thwarted by punishing male-female couples who engaged in sex, even though same-sex sexual encounters were routinely tolerated. 

Coed prisons might have been a good idea, at least for certain prisoners, but overall as implemented they failed because, in every prison, efforts to implement a given design were defeated by competing, incompatible designs.  Before there was any chance to act on this realization, women came to the rescue by committing more crimes in greater variety and demanding prisons of their own.   (Anyone interested in learning more about coeducational prisons is invited to contact the author.)  

Another example of how implementation can diverge from and fall short of design can be seen in how the U.S. Census operated in Harlem in 1970. Before attending grad school at Howard, I also had the privilege of being a census enumerator in Harlem. In response to charges of systematic undercounts of high-minority communities, the Census was being pressured to reach out and perform a more thorough count in Harlem, still reeling from race riots and the assassinations of Malcolm X in 1965 and MLK in 1968.  Harlem remained the cultural and political capital of black New Yorkers and, in some respects, black America. Based on the 1960 census, only 51% of housing in Harlem was “sound” compared to 85% elsewhere in the city, and Harlemites were being overcharged to live in cramped quarters. After exodus by those who could afford to leave, those who remained were the poorest and least skilled. Harlem’s towering subsidized housing projects and converted single family homes were filled with the poorest.

To encourage us to be more vigilant in outreach., they paid us piecework in Harlem. In the rest of the city, enumerators were paid $2.50 per hour.  We were offered $1.30 for completion of a short form, $5.00 for a long form, plus thirty cents additional per person enumerated.  So, if every hour I completed a short form in a household with 4 people and a second short form in a household with two people, I made $4.40, much more than the $2.50 hourly rate.  And, as I quickly learned, there were ways to earn vastly more.   

A number of factors built into the implementation of the 1970 Harlem census defeated the noble design for outreach. The supervisors, nearly all of whom were black, trained us enumerators, who likewise were nearly all black, to anticipate anger, fear, and even mortal danger when we knocked on the doors of Harlem’s residents. They warned: the people of Harlem don’t trust government.  Each young black man shipped as fodder to Vietnam confirmed the pervasive belief that draft boards conspired to exterminate black men. As a result of this conflicting message — reach out aggressively, but fear for your life — many enumerators never returned after accepting their first assignments or they “curbstoned” (literally, sat on the curb and faked the answers) rather than placing themselves unduly in harm’s way. 

For me, when I knocked on door after Harlem door, brownstones and tenements alike, a voice routinely demanded, “What you want?” When I replied, “I come to take your Census,” the occupants almost uniformly opened the door—often triple and quadruple locked—and civilly invited me in.  There were a few moments when I felt my life was at risk. For example, on one occasion, a machete was placed firmly against my throat by an apartment’s occupant, but we got past that, had a meeting of the minds, and I made my $1.30 + $.30 =$1.60.  I’m confident I would have felt even greater risk knocking on doors in wealthy, white, gated communities. 

Slated for demolition, a once-glitzy theater on West 125^th Street had been transformed into the Harlem outpost for the 1970 census.  Where the orchestra section was once located, Census enumerators now met with supervisors at makeshift tables to review completed assignments and receive new ones.   Above, in the former balcony, bosses kept watchful eye and walked the aisles.   

Most enumerators weren’t fired immediately for curbstoning, either because they started out more-or-less honest or because they forged with a deftness that required investigation and confirmation.   We called the orchestra “the pit” because when enumerators and supervisors shouted at, charged at, and faked striking each other, they looked like stock brokers on the trading floor—notoriously known as “the pit.”   A typical conversation went like this:

Supervisor: There’s a problem with your work.

Enumerator: Why?  What’sa problem?

S: You forged the data forms.  

E: Whadju mean?

S: I mean, I can’t pay you for this work and you’re fired.   

E: I worked hard filling out those forms.

S: Who helped you?  Your mother?  Your girlfriend?

E: What’re you talking about?  

S:  We’re pros.  We look at patterns. We check against things we already know about each household.    And we call some back to check.  

E: You liars never said you were going to do those things.  

S: We told you we check.  

E: Hey, I need this job.

S:  You’re done.  Please hand me your bag and your badge (reaching to take both).

E: No, man, give me another chance (pulling back bag).

S: I’m letting you off easy. I could have you charged with perpetrating fraud against the United States Government.  

E: No man, please. What about all the time I put in?   

S: In front of your TV?

E: No, man. I really did went to some of the households.   

S: Yeah, which ones?

E: You pay me for ‘em?

S:  You had a great deal. The operative word is had. I don’t ever want to see your face again. Get out.   

Occasionally, during shouting matches, one of the bosses came down from the balcony to rescue an enumerator who was on the ropes.   Rarely did the bosses appear at our level.   

“You don’t ever want to go up there,” my supervisor, Roxanne, told me.

“Why?” I asked. “It doesn’t look like anybody’s ever doing anything up there.”

“It’s a different culture,” she said. “If you work for any of the bosses, and one of them wants sex, you have to put out because your job’s at stake.  Going up there isn’t a reward for work performance.  Oh, you could go up there if you wanted to, but don’t think being a white male would protect you. They would gobble you up.”

A heavy-set, soft-spoken Puerto Rican and one of three woman supervisors, Roxanne wasn’t a yeller, but meant business, and had a knack for detecting fraud early. By the 4^th of July, she only had a handful of enumerators.  I typically reported around 12:30, got her lunch order and picked up her lunch at a Puerto Rican place around the corner.  Her typical order consisted of chicken or chorizo, a starch (yucca , yams, or plantains), and three vegetables (asparagus, cabbage, cauliflower, chayote, eggplant, or West Indian pumpkin).  She always over-ordered and when I delivered her food she surveyed it with dismay and said, “There’s far too much here for me.  Please help yourself.”  Wearing her oversized pink shades, Roxanne checked in my work in excruciating detail. 

Roxanne piled new work on me daily and expected results within 24 hours.  She got excited after I started discovering stately, single-family homes that had been subdivided into multiple housing units 40 years ago, undetected for decades by the all-seeing-eye of the Census Bureau.  Documenting such oversights justified allegations the Census under-counted minorities, but also helped correct the undercount.      

One Wednesday in mid-July, I showed up for work, looked around “the pit,” but couldn’t find Roxanne.  A man in his 50s with curly white hair waved me over to his table. “I’m Marcellos, your new supervisor.  Roxanne said you’re good people. We’re gonna get along.”  For six weeks, I got all work I could handle because they couldn’t train and keep enumerators willing to knock on Harlem doors. I made far more money than I’d ever seen. Now and then, enumerators were “promoted” to the balcony, but nobody bothered me.      

Over the next decade, Harlem continued a rapid decline characterized by wholesale abandonment of privately-owned housing by landlords, shuttering of stores, mass exodus of the black population, and a heavy reliance on illegal activity for income. In hindsight, my summer in Harlem looks like the neighborhood’s high-water mark.

I think we all know what I learned working as a Census enumerator in Harlem and evaluating coed prisons: the integrity of any design is only as good as the care accorded to its implementations. The gap between design rhetoric and implementation realities is often wider than the Grand Canyon. We should never make the mistake of assuming design equals reality.