Think of something like trying to figure out if you are seeing a mirage. You see it, but when you try to touch it, you can’t. The modes (vision and touch) diverge in results. We draw the conclusion that the mirage is not a real object. But are multiple modes of analysis always a good method of checking something? Additionally, under what circumstances are multiple modes not a good method?
These questions can be made more substantial by thinking about the following tension: Folk wisdom tells us that two heads are better than one But we also know that when many people get together and think, there’s a risk of cognitive herding behavior. The former means to highlight the idea that there is added cognitive machinery when individual agents partner up to think. The latter means to highlight the synchronization of machines when agents partner up to think. In this post, I’d like to draw attention to, as well as illustrate a simple methodological tool for understanding how multiple modes can be used to check each other’s results. This applies to both the scientific context as well as our daily lives.
Let’s start with the scientific context. Replication seems like a scientific virtue. The more instances of corroboration we have the stronger our evidential set seems to be. But replication is not always virtuous. Here’s a simple example.
For a few, very exciting, months it seemed as though physicists had detected particles that travelled faster than the speed of light. A group of physicists generated muon neutrinos at the Super Proton Synchrotron (SPS) particle accelerator at the CERN LHC complex in Geneva (Amelino-Camelia 2012). They accelerated these neutrinos down a 1 km beam line toward the Gran Sasso National Laboratory in Italy. Incredibly, neutrinos travelling from CERN to Gran Sasso were measured to make the trip 60.7 nanoseconds faster than light speed. The measurement procedure was very carefully planned. For example, time was measured with GPS timing signals and a cesium atomic clock. The GPS also allowed the physicists to track any small movements in the Earth itself. Even factors such as day vs. night and seasonal trends were taken into account. One of the most important and interesting facts about the methodology is that the experiment was repeated 15,000 times! Additionally, later experiments, checking for systematic error by fiddling with different proton pulse profiles, seemed to replicate the results.
It seems as though each trial (and each replication) serves as a “check” on the other trials. However, this fails to be case of good methodological checking. The reason is the repeated/replicated measurement procedures are not independent. It was discovered (Cartlidge 2012) that the 60 nanoseconds discrepancy came from a poor connection between a fiber optic cable that connects the GPS receiver (used to correct the timing of the neutrinos' flight) with an electronic card in a computer. The connection was tightened and the faster-than-light neutrinos were no more. Each of the 15,000 trials could not serve as a “check” for other trials because they all suffer from the same systematic mistake. To properly check results, we need independence of modes.
The independence of modes is something that we can experiment on in a very simple way. Have a large group of people guess your weight without debating with each other about any of the factors. The likely result is that when you add up and average everyone’s guess, you will get a number very close to your actual weight. The reason why this works is because some people guess low, others guess high, and the error cancels out. But each person’s error is independent of another person’s error. As long as this is the case, then multiple heads are better than one.
But what if you’re not dealing with multiple individuals or multiple sources of information? How does independence of modes apply to you? The title of the post is, after all, about you checking yourself. The next example will give us a simple view of modes and independence.
MacArthur Wheeler robs a bank in broad daylight (Kruger and Dunning 2009). He does not wear a mask and is captured within an hour. When captured, he is in complete shock, “But I wore the juice.” Wheeler believed that putting lemon juice on his face would make him invisible to cameras. He arrived at this conclusion by, what seemed to be, good checking methods. He put lemon juice on his face and took a series of selfie-polaroids. In this case, each photo seems to be an independent mode of measurement. In the results of the polaroids, Wheeler was nowhere to be found. How could all of the photos converge on this result? Simple: What compromises the independence of each measurement process is Wheeler’s systematic stupidity. Wheeler is the engineer behind each measurement process and the interpretation of each result. As such, he is part of the design, implementation, and interpretation of each measurement process. So whatever mistakes he makes, he continues to make, unless an independent engineer steps in and corrects them. This is what is referred to as the Dunning-Kruger Effect: general incompetence produces lack of awareness of incompetence. Think for a moment, though: consistent with the Dunning-Kruger effect, checking yourself is something that everyone can reflect on, and nothing that anyone can truly do for themselves.
We've come to the conclusion that checking requires the independence of modes. We have illustrated what kinds of independence are good for checking. The next step is defining independence of modes. Not only is it important to specify conditions for independence, but it is also important to characterize what kind of independence is sufficient (e.g. partial vs. full independence).
Work Cited:
Amelino-Camelia, G., (2011). OPERA data. Phenomenology of Philosophy of Science. arXiv: 1206.3554v1[physics.hist-ph]
Cartlidge, Edwin, (2012). BREAKING NEWS: Error Undoes Faster-Than-Light NeutrinoResults.
J. Kruger, D. Dunning (2009). Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments. Psychology, 1 (2009), pp. 30–46
Vadim Keyser
Department of Philosophy
Sacramento State
Let’s start with the scientific context. Replication seems like a scientific virtue. The more instances of corroboration we have the stronger our evidential set seems to be. But replication is not always virtuous. Here’s a simple example.
For a few, very exciting, months it seemed as though physicists had detected particles that travelled faster than the speed of light. A group of physicists generated muon neutrinos at the Super Proton Synchrotron (SPS) particle accelerator at the CERN LHC complex in Geneva (Amelino-Camelia 2012). They accelerated these neutrinos down a 1 km beam line toward the Gran Sasso National Laboratory in Italy. Incredibly, neutrinos travelling from CERN to Gran Sasso were measured to make the trip 60.7 nanoseconds faster than light speed. The measurement procedure was very carefully planned. For example, time was measured with GPS timing signals and a cesium atomic clock. The GPS also allowed the physicists to track any small movements in the Earth itself. Even factors such as day vs. night and seasonal trends were taken into account. One of the most important and interesting facts about the methodology is that the experiment was repeated 15,000 times! Additionally, later experiments, checking for systematic error by fiddling with different proton pulse profiles, seemed to replicate the results.
It seems as though each trial (and each replication) serves as a “check” on the other trials. However, this fails to be case of good methodological checking. The reason is the repeated/replicated measurement procedures are not independent. It was discovered (Cartlidge 2012) that the 60 nanoseconds discrepancy came from a poor connection between a fiber optic cable that connects the GPS receiver (used to correct the timing of the neutrinos' flight) with an electronic card in a computer. The connection was tightened and the faster-than-light neutrinos were no more. Each of the 15,000 trials could not serve as a “check” for other trials because they all suffer from the same systematic mistake. To properly check results, we need independence of modes.
The independence of modes is something that we can experiment on in a very simple way. Have a large group of people guess your weight without debating with each other about any of the factors. The likely result is that when you add up and average everyone’s guess, you will get a number very close to your actual weight. The reason why this works is because some people guess low, others guess high, and the error cancels out. But each person’s error is independent of another person’s error. As long as this is the case, then multiple heads are better than one.
But what if you’re not dealing with multiple individuals or multiple sources of information? How does independence of modes apply to you? The title of the post is, after all, about you checking yourself. The next example will give us a simple view of modes and independence.
MacArthur Wheeler robs a bank in broad daylight (Kruger and Dunning 2009). He does not wear a mask and is captured within an hour. When captured, he is in complete shock, “But I wore the juice.” Wheeler believed that putting lemon juice on his face would make him invisible to cameras. He arrived at this conclusion by, what seemed to be, good checking methods. He put lemon juice on his face and took a series of selfie-polaroids. In this case, each photo seems to be an independent mode of measurement. In the results of the polaroids, Wheeler was nowhere to be found. How could all of the photos converge on this result? Simple: What compromises the independence of each measurement process is Wheeler’s systematic stupidity. Wheeler is the engineer behind each measurement process and the interpretation of each result. As such, he is part of the design, implementation, and interpretation of each measurement process. So whatever mistakes he makes, he continues to make, unless an independent engineer steps in and corrects them. This is what is referred to as the Dunning-Kruger Effect: general incompetence produces lack of awareness of incompetence. Think for a moment, though: consistent with the Dunning-Kruger effect, checking yourself is something that everyone can reflect on, and nothing that anyone can truly do for themselves.
We've come to the conclusion that checking requires the independence of modes. We have illustrated what kinds of independence are good for checking. The next step is defining independence of modes. Not only is it important to specify conditions for independence, but it is also important to characterize what kind of independence is sufficient (e.g. partial vs. full independence).
Work Cited:
Amelino-Camelia, G., (2011). OPERA data. Phenomenology of Philosophy of Science. arXiv: 1206.3554v1[physics.hist-ph]
Cartlidge, Edwin, (2012). BREAKING NEWS: Error Undoes Faster-Than-Light NeutrinoResults.
J. Kruger, D. Dunning (2009). Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments. Psychology, 1 (2009), pp. 30–46
Vadim Keyser
Department of Philosophy
Sacramento State
I used to think my research was brilliant before I submitted it for review! I suppose enlisting some independent check before submitting would be a good idea. Vadim, would you recommend I consult and astrologist or a tea leaf reader? Or maybe both, sincell they all seem quite independent :)
ReplyDeleteOn a more serious note, the CERN mix up would likely have gone unnoticed for a long time if coherence with other highly confirmed theories was not important for evaluating new results. Without considerations of coherence, deference to the biggest/newest/most expensive collide might have resulted in physics being led astray for a few years.
ReplyDeleteVadim, thanks very much for this. I wonder if you are using the idea of a mode of measurement to cover two ideas that it can be useful to distinguish in this context. In your second paragraph, you illustrate by pointing out how distinct ways of perceiving a phenomenon, like sight and touch, can conflict. This is mirrored by scientific inquiry, in the sense that we have entirely distinct operations for measuring things like temperature and mass. But in your CERN example, as well as your crowd wisdom and bank robber examples, it seems like what you call a distinct mode might actually be better described as a physically independent measuring instrument operating in the same mode.
ReplyDeleteDan, nice point about coherence. The next question is, is coherence a scientific virtue because of its dependence on independent modes of checking or, is it an independent virtue that can be used as a mode for checking our checking?
ReplyDeleteRandy, that's a solid distinction. I think to make it we need to figure out what constitutes 'independence'. It might be that instruments do not operate without context, and for this reason, they'd still be considered parts of different modes of measurement, depending on the details of the context. Physical trials and agents with cognitive machinery might be like thermometers only if their calibration is somehow standardized and invariant over context of measurement. In the CERN case the standardization seems likely. I don't think it's as simple in the agent case.
Adding to your suggestion, perhaps we could even look at it as a spectrum of modes.