In Harmonium

Posted By Marc on September 29, 2009

In the US House Report 111-166 – NATIONAL DEFENSE AUTHORIZATION ACT FOR FISCAL YEAR 2010, there is a very interesting set of requirements regarding a direction to implement an independent assessment of the Human Terrain System (major hat tip to John Stanton for letting me know – his own article on it is available here).

Why an assessment?

According to the minutes of the House Armed Services Committee (HASC) minutes, the requirement is necessary because:

The committee is aware of anecdotal evidence indicating the benefits of the program supporting operations in the Republic of Iraq and the Islamic Republic of Afghanistan. The committee also notes that a number of press accounts provide anecdotal evidence indicating problems with management and resourcing. The committee finds it difficult to evaluate either set of information in the absence of reliable, empirical data.

This lack of demonstrable, empirical evidence was a problem I highlighted last December in several posts (here and here) when I called for a program review, and even earlier in a discussion of the ethics debates (here and here).  I can certainly understand why the “committee finds it difficult to evaluate either set of information”!

What is HASC asking for?

Specifically, the minutes note that

Therefore, the committee directs the Secretary of Defense to conduct an independent assessment of the Human Terrain System, and submit to the congressional defense committees a report detailing that assessment by March 1, 2010. The independent assessment should consider the following elements:

1. An overview of all of the components of HTS, including related technology development efforts;
2. The adequacy of the management structure for HTS;
3. The metrics used to evaluate each of the components of HTS;
4. The adequacy of human resourcing and recruiting efforts, including the implications of converting some contractor positions to government positions;
5. An identification of skills that are not resident in government or military positions, and how the Army can leverage academic networks or contracting opportunities to fill those gaps;
6. An identification of policy or regulatory issues hindering program execution; and
7. The potential to integrate HTS capabilities into existing exercises.

Having done some consulting work in my time, I can tell you that this is not a simple request, and a March 1st, 2010 deadline is extremely tight.  My rough and ready take, and I have been thinking about a program review for a while now, is that it would require a pretty large team to complete, and that team would have to be pretty good at being “neutral”.

Some problems…

The one key problem I see with what HASC is looking for is revealed in point 5.  They are looking for ways to recruit those skills without asking for an examination of the structures and other factors that constrain the recruitment of people with the specific skills they need.  Possibly even more damning is that those skills are not really identified, and they use proxies for them, such as someone having a PhD.

A second problem I see resides in the use of the term “metrics”.  Metrics are useful ways of measuring something only if you know both what you are measuring and why you are measuring it.  As with many applications of quantification, metrics work very nicely for physical “things”, but pretty poorly for cultural and perceptual “things”.  Now, that last statement may seem a little odd in light of the extensive use of survey research, but I stand by it for a number of reasons;

1. all to often, survey questions are extremely simplistic;
2. attitudinal surveys only measure a snapshot in time, so you really need longitudinal surveys; and
3. survey questions (and responses) are often “interpreted” (or manipulated) to meet an audience’s perceptions.

Why the “mini-rant” against metrics?  Well, the types of simplistic metrics that will probably end up being used by any assessment will, in all probability, be a poor reflection of the reality.  Ideally, such an assessment should involve extensive interviewing and participant observation, both in the field with deployed HTTs and in the classrooms, and at least some of this interviewing should be done well before any surveys are conducted.

A final comment…

As you can imagine, I am going to be watching this extremely closely.

Category: Human Terrain System (HTS)
Tags: Human Terrain System (HTS)