View Single Post
Old 03-14-2015, 03:32 PM   #41 (permalink)
Rosa
Senior Member
47-hour Marathon 2016 Kickstarter Backer
 
Rosa's Avatar
 
Join Date: Jan 2015
Posts: 129
Quote:
Originally Posted by robotparker View Post
Don't mind if I do!

I approached the problem with the mind of a photographer. I got my BFA in Photography and Media Arts and have worked as a freelance photographer for almost a decade, in addition to teaching photography on occasion so I'm no stranger to looking at bad images. When I first saw the original viral image, I immediately pegged it as a poorly exposed, mis-white-balanced image of a white and gold dress. And despite all the explanations I've seen to the contrary (including yours), I've never been able to see it otherwise.

Now, there's no denying that in the original image there is, in fact, blue present in the dress. But I submit that the actual dress isn't blue and only appears to be because of the problems inherent in the low quality sensors and image processors you find in the average cell phone. When I'm touching up and correcting images in a professional context, the first step is always to make sure the white balance is correct. Every light source in a setting affects the overall color temperature of the image, so unless you're working with direct sunlight or controlled studio lighting then your camera is going to have to rely on its best guess and often it's not quite right. Here's that phenomenon in action:

Were people honestly convinced that these were the same dresses? Why can't my mind accept it? I thought that perhaps maybe it was just a matter of complicated lighting and I tried to conceive of what that scenario would look like in practice. Because even if something's black, it would still have some capacity for refection, especially if it were a shiny piece of fabric. If you had a bright enough amber colored light right next to the dress, then it could feasibly result in the black part appearing to be gold in a badly exposed image. And then one night while I was driving, I saw the phenomenon in action. A local bar's sign, comprised of sparse white text and a big black square was being illuminated by the gross orange/amber glow of a street light, and sure enough the highlighted portions of the black did appear to be golden. But the problem was, the white letters also appeared to be golden.

There's simply no scenario where a light powerful enough to make a solid black piece of fabric look golden would also make an adjacent blue piece of fabric appear white. It totally flies in the face of all the practical knowledge I have of photography and studio lighting practices. As for the actual parties involved insisting that the dress is black and blue, I dunno what to tell ya. Maybe there was some sort of miscommunication somewhere that snowballed out of control and/or maybe there's some trolling aspect to it. All I can say is that there's no way a dress can melt steel beams.
SWEEEET. You touch on a bunch of the really interesting nuances in this web.

I dont' even know where to begin, but this is great.

you are totally right, the exposure in this image is whack. thats why this image is 1 in a trillion(?). and why it breaks our brains. it is not realistic, and it is full of ambiguities/conflicting information. it shouldn't be able to happen, given how cameras work. and also as you mention, the material properties of fabric are important. I'm going to punt fabric for a moment, and will come back to it. First, we must address the elephant in the room. we both agree this image should not exist. it should be impossible. May I propose one way that this weird image may have come about (in terms of camera behavior)? your critical feedback would be seriously appreciated on this one...

I wonder if what happened was there was (a) a cool light coming in from behind the dress (that big bright white strip along the side of the image--maybe its a window with bright mid-day daylight, which is blue-biased, pouring in, and (b) there was a bright warm (yellow-biased) incandescent light hanging above the dress, directly lighting it (common in department stores). Then, when the woman pointed her camera at the dress, the camera detected the cool blue light in the background and white balanced for a cool illuminant, even though it should have been calibrating to the warm light that was illuminating the dress from above. If this is the case, it would enhance the amount of warmth left in the image, and would also serve to remove extra blue from the image. So if you had a blue and black dress, the blue would get washed out (pushed in the white direction) and the black would get pushed in the yellow/orange direction. I am an amatuer photographer, so please correct me if I'm wrong on this one, but it feels sensible. One additional reason to believe this could be the case is that the most golden black region of the dress is near the top (neckline), where you could imagine getting that bias from an overhanging light.

OK. Fabric and material properties. I didn't bother touching on this in my earlier post, as it adds a whole additional dimension to the problem. But when I talked about our internal model for the world and the kind of inferences we make and how we weight our various priors, one of the important features that goes into this model is the inferred material properties of the object and our priors about how light interacts with certain material types (which is why it is important to know whether a fabric is matte or shiny). Whatever you infer, this will influence how you interpret the reflectance and the illumination. The appearance of matte and shiny materials should be predictable for shadow vs direct illumination. For this reason, we actually collected data on people's fabric judgments in our study. We indicated four different regions of the dress and had subjects indicate whether they thought the material was matte, sheer, semi-gloss, or high-gloss. We chose a "black/gold" region and a "blue/white" region at the top, and then again a "black/gold" region and a "blue/white" region at the bottom. This is because the illumination cues vary along the vertical axis of the dress... as if the white balance is not consistent across the image. Just to make things extra ambiguous.

The most interesting region to me is the trim around the neckline. This is where the most golden pixels lie. For me, it is this region that conveys the greatest ambiguity about the illumination conditions. When I want to push the white-gold percept, this is where I focus. It is also the region where people’s fabric judgements best predicted if they were Blue-Black (BB) viewers or White-Gold (WG) viewers. In our data, BB folks judged that region to be made of a matte material more often than WG folks, while WG folks judged the fabric there to be more semi-gloss (shinier). If you think the material is matte, then perhaps you attribute the golden glean to being the result of a black matte fabric that is *very* washed out due to being lit by a *very* bright yellow direct illuminant. If so, your brain removes a bunch of the yellow bias in the image giving you the blue/black percept. But if you think the material is glossy/shiny, you think the gold indicates the true color, but that the dress is in shadow (otherwise a shiny material would give off more of a white shine/highlight). If the dress is in shadow, you need to remove some blue (shadows are blue-biased), which enhances the white gold percept.

Here's the kicker: The "black" trim along the neckline is actually sheer, and has some interesting reflective properties that would be worth measuring with a device (see below, or if that pic isn't any good, just take my word for it, because I have the dress in front of me... because I'm such a dork that I ordered it for my Vision Science Conf. in florida this summer... and because I want to take spectral readings under diff. illuminants to see how it behaves). It's the only sheer part of the dress. But most people judge the material there to be the same as the trim everywhere else. But it will necessarily behave differently, so we are probably not modeling that fabric correctly. not sure if this is something that would impact the camera's too... I need to think more about this. if you have thoughts, I'd love to hear them.




To come back to the main point you raised: the exposure in this image is whack. and this is why it breaks our brains. it is not realistic. it shouldn't be able to happen, given how cameras work. But this brings up another interesting feature of the whole phenomenon: because this is a photograph, our brain insists on interpreting it as if it were realistic. and given how weird it is, this ends up being super fucking difficult to do, yielding a very unstable decision threshold that people actually fall on different sides of. this is incredibly rare in color vision. there is no other bi-stable color illusion this compelling in existence (the colored Mach Card is the only one, and that requires actual 3D cues....like a folded corner). SO, given that our brain has to make some decision, it weights very heavily our internal model's priors, which includes the fabric stuff discussed above, but most importantly, information about the typicality of daylight and how it behaves: The impact of our prior over daylight is huge in this illusion. The distribution of pixel values in this particular image (mostly blue and orange/brown.... *brown is dark orange) perfectly aligns with the daylight axis distribution (sunlight measured over the course of the day... you lay a sheet of white paper outside and take spectral readings every hour, then plot the distribution). And the field has good fMRI, ephys, and behavioral data that suggest our neural illumination discounting mechanism is specially tuned to the daylight axis (Lafer-Sousa et al 2012; Pearce et al 2014); we think this is actually very important to understanding why the image is bi-stable and gives two such salient distinct percepts; essentially, you strongly discount one end of the daylight axis or the other; this is the first panel in our figure in the paper, so I can't reproduce it here; but suffice it to say, the two tightly overlap and I think this is something that ends up contributing to how strong the discounting we do is.... I say all of this just so that you fully appreciate that your percept is a) a highly internally biased interpretation of a very ambiguous situation, b) is really quite different from the pixel values (you are doing a lot of illumination discounting) and c) is really quite different from mine, because mine has the same features, but is pushed in the opposite direction along the daylight axis. oh god. i hope any of that makes sense and is useful to you....

I find these figures useful too (made by color vision expert Qasim Zaidi for the NYTimes last week):


Last edited by Rosa; 03-14-2015 at 03:45 PM.
(Offline)   Reply With Quote