The use of crowd intelligence seems to be
continuing to grow. Perhaps decision democratization or committee consensus
would be better descriptions, but neither has quite the cache of “intelligence”.
Even if our crowds are not expert, (and they are supposed to be cast wider than
just experts,) we should hope that they are wise.
The latest piece of crowd intelligence to catch my
eye is Bernstein et al’s study in
medical diagnosis (1). The diagnostic problem was the assessment of hip
fractures. That is the problem of a break at the neck of the femur when, if
there is displacement of the ball from the socket, the surgical solution is a
hip replacement. If it is not displaced, pinning is the treatment. Diagnosis is
done from X-ray pictures; nobody waits until they get to the table to decide.
The baseline for correct diagnosis was 69% for the
twelve orthopedic surgeons chosen for the test. Note: no unlucky old folks were
affected by these diagnoses. Next, small committees were formed of various
sizes and the diagnosis was decided by majority vote.
Committees did better with 3-member committees
hitting 77% and 5-member committees getting to 80% correct.
The big experiment was done next with a 40-member
internet-based group. The critical issue here is response time. The input from
a group of nine was taken as a sufficient “crowd” to get things right and this
could be done in about an hour, with the longest time in a ten-case study being
less than two and a half hours.
The authors conclude that their “wise crowd” should
have people with diverse opinions and, of course, the democracy of the internet
means that no one dominates the committee. So this may be a low-cost error-reduction procedure, which sounds good.
However, the “crowd” is still a crowd of “experts,”
in other word, a “wise crowd” and this takes us away from the real idea of
crowd intelligence where the non-expert is included thus enabling an unexpected
solution to emerge.
Personally, this leaves me in a bit of a dilemma.
The wise crowd decision over a broken leg seems to work OK, but it seemed to be
a failure in Congress where six “wise experts” couldn’t achieve a consensus.
The latter is clearly a case where true crowd intelligence is required.
