“Collective intelligence” refers to the concept that a diverse collection of independently deciding individuals is likely to make certain types of decisions and predictions better than individuals or even experts. This is at the heart of consensus diagnosis in medicine.

Consensus diagnosis has its roots in a phenomenon described by Francis Galton in 1907. He observed that in a country fair contest, the weight of an ox was estimated poorly by individuals, yet the mean of these guesses was within 99.2% of the true value—a collective estimate that was more accurate than the estimates given by cattle experts.

From that observation, it was recognized that the aggregation of information from groups might yield better decisions and solutions than could have been offered by individual experts. The expertise possessed by the group in toto has been termed “the wisdom of the crowd” or “crowd intelligence.”

Case Example: Orthopaedic Oncology Collective Intelligence Project

In collaboration with the Musculoskeletal Tumor Society, OrthopaedicsOne is currently hosting a pilot study on collective intelligence, called the Orthopaedic Oncology Collective Intelligence Project. The purpose of this project is to determine whether groups of orthopaedic specialists can collaborate effectively online to determine consensus on treatment options for complex patient cases, and whether that consensus impacts final treatment.

Participants in the Orthopaedic Oncology Collective Intelligence Project are fellowship-trained, American Board of Orthopaedic Surgery-certified musculoskeletal oncologists. The process for submitting and commenting on cases is straightforward:

  • The group member submitting the case completes a template on OrthopaedicsOne that includes the clinical history, physical examination, and diagnostic imaging for the case. He or she may ask the group for consensus on up to three questions regarding the case.
  • One of the moderators reviews the case, asks any additional questions needed for clarification, and then converts the three questions to a survey that is distributed to the group for review. The moderator sets a deadline for responding.
  • The group members review the case, consider the questions, and then submit their answers for tabulation. Questions are answered in a blinded fashion.
  • Once the deadline has passed, the moderator closes the survey and the results are released.
  • The project will evaluate whether the group consensus changes the initial decisions provided by the group member submitting the case.

The model seeks to aggregate anonymously produced data, seeing the wisdom emerging when a large number of people independently enter their decisions without influencing each other’s findings. If this proof-of-concept pilot bears out, it will pave the way for orthopaedic surgeons in other specialties to use the functionality of OrthopaedicsOne to develop their own collective intelligence projects.

Case Example: OrthopaedicsOne Collective Intelligence Platform

In addition to the customized collective intelligence spaces that OrthopaedicsOne can support, we have created a Collective Intelligence Platform that allows a Principal Investigator (PI) to set up and manage a Collective Intelligence Study.

A PI and co-PIs can invite collaborators to submit cases for “reading” by either a group of invited readers or the entire OrthopaedicsOne community. The platform handles important research aspects, such as moderation, mirror image creation, and email notification. With the system, researchers can evaluate diagnostic errors, classification systems, or decision-making algorithms.

Contact Information

For more information about collective intelligence projects on OrthopaedicsOne, pleasecontact our Editorial Team

Reference

  1. Bernstein J, Long JS, Veillette C, Ahn J. Source: Crowd Intelligence for the Classification of Fractures and Beyond. PLoS ONE 6(11): e27620. doi:10.1371/journal.pone.0027620, available at http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0027620