Image library concept very disturbing, can you explain?
This topic contains 7 replies, has 2 voices, and was last updated by Olivier 9 months, 3 weeks ago.
September 24, 2017 at 5:48 am #21808
I found out about this website yesterday and while the text library is already amazing I’ve been extremely disturbed by the image library maybe because I’m a photographer. While I understand any image is simply a specific organization of pixels on the screen and one could produce any arrangement and therefore any picture I don’t understand how the search feature works given the amount of data at stake. From what the creator of the library wrote in a post, the “books” are not actually stored but uniquely generated using algorithms and I guess the text search actually perform some specific algorithm selection that matches the entered quote. But what about pictures? Thanks in advance for some explanations, this is really bothering me and I can hardly believe at this stage the photos I upload or web linked is not simply process using some degradation filter.
On another level, I shared my discovery about the library with friends yesterday and they did not seem interested, they would not see the point. To me it is a slap in the face, a huge shock. I can’t really explain why but it is overwhelming. I guess I need to read Borges 😉September 24, 2017 at 11:02 am #21813
Pictures to a computer are just data, like any other form of data. The image archive is made up of every possible images that is 640×416 pixels containing up to 4096 colors.
To describe how pictures are stored on a computer in a nutshell:
Images are made up of pixels, I’m sure you know. If you were to look very closely at your screen, you would see it’s made up of a grid of red, green, and blue lights. In order to display an image on the screen, you need to increase or decrease the brightness of each light individually. Now, in order to represent this data digitally, we encode it into “RGB” values (red/green/blue). Each value is made up of 3 bytes, with a value from 0 to 255. In order to make the color red, it would be stored as <255, 0, 0> meaning 100% red, 0% green, and 0% blue. A picture is made up of millions of these RGB values, one for each pixel.
Once you understand that, all you have to know is that the RGB can be stored in other ways. Instead of 3 separate decimal numbers, you can convert it to hexidecimal. To use the same example of <255, 0, 0>, the color red in hex would be FF0000. Since hexidecimal is base 16, it takes fewer numerals to represent a color (meaning less space). It will always be 6 numerals, whereas the color white for example is <255, 255, 255> (9 numerals). Now all you have to do is imagine a small 3 pixel image of a red, green, and blue pixel to look like: FF0000 00FF00 0000FF. The Library then uses that data the same way it operates the normal text version.September 24, 2017 at 4:38 pm #21817
I guess I’ve got this part. What I don’t get is how the system retrieves the picture I upload in the first place. I suppose my picture is first analyzed and then compared to the library to find the same one and this is the part that strikes me, we are talking here about quite a substantial database to retrieve something from even if each and every picture file does not actually exist. Also there should be thousand, millions of pictures in the library that are almost the same as the one I uploaded (differing only by one pixel in the various tones or the whole picture slightly altered in the tone), I mean there are some many options here. How are the pictures arranged in the library? I see they all have a distinct reference (makes sense). Would it be possible to browse all images that are close to the one the system outputs (iterations of the Mona Lisa painting for example). When I browse the picture around the retrieved Mona Lisa, the previous one is just noise, same for the next one. It would not need face or form recognition, maybe something like a hash value that would make the system able to organize the pictures. Also I was thinking that the difference between a pure random noise picture and one that actually represents something could be detected by trying to compress image, i.e. more noise means lower compression ratio, patterns being easier to compress. But maybe this would require to produce an actual file even if only on the fly and this is not feasible given the number of images here.
This is jaw dropping but it could be a fairly easy hoax (the system just takes the picture you upload, converts it badly and displays it as a result. Honestly I want to believe 🙂 but I also would like to understand. Thanks in advanceSeptember 25, 2017 at 9:52 pm #21834
I’ve written a post on how the normal text Library works, but the same code applies to images. You can read it here, but pay special attention to the 2nd to last paragraph.
If you were to upload the same picture twice, it will be in the exact same location. This is because when you supply a picture, it’s the same as supplying 3200 characters to the text search. Once your image is converted to a text format, it’s fed into a similar inverse algorithm. The only difference is that the location is just one number (really, the hex/wall/shelf/book numbers are all combined in some way to make one number as well). The algorithm has no choice but to always backtrace to the same location number, since it’s purely mathematics. The reason you get noise immediately to the left or right, is that the forward algorithm just continues generating whatever random sequence it was going to continue with.September 25, 2017 at 9:53 pm #21835
(Apologies for the double-post, but apparently there is an enforced character limit)
It’s a little bit much to understand if you’re not a programmer or at least familiar with the concept of pseudorandom number generation (PRNG). Computers by nature are not (normally) capable of natural randomness. They require a mathematical formula that takes an input called a “seed” and then it spits out another number. The new number is now the new seed, and fed right back into the algorithm. This process continues forever. At any point, you can supply your own seed to a PRNG and it will continue with the same sequence it would have. In order to make sure you always get a decently random sequence, you can use the current time in microseconds as your starting seed. This ensures that even if two people booted up the same game for example, it would be nearly impossible for them to encounter the same random sequence of events. The other way is to use true number generation (which is why I said “normally” earlier). This method uses other ways to obtain a random number, such as having the computer measure the temperature of a piece of hardware such as the graphics card or processor. Some TRNGs will have a dedicated device that can just listen to radio static.September 26, 2017 at 12:03 am #21840
The idea makes sense, the execution does not. For instance, these pictures could just be stored millions upon millions of iterations away from the last picture stored, making it highly unlikely any two pictures that people “look for” are close to each other. Anything else is just randomly generated and stored at the time of generation, so maybe you do get a picture out of that… but probably not.
The number of possibilities is so big that it’s extremely hard to imagine that all of it is stored in one place, and beyond that, how is it possible that this system can compare all the images in the database and find a match within seconds? It seems extremely unlikely, and how do I know that these images aren’t randomly chosen over and over again to make it seem like it’s changing? Not only that, but if you take an image smaller or bigger than the normal image size, the image is the same size as it was when you loaded it, and is stored that way. All other images are the same size, debunking it completely.
Now, something that might make sense is to get a hand on the algorithm that builds these random images, and do some comparison on the pixels in the random images to only store the ones that have colors in which might actually mean something. For instance, if i take a picture of something, I’d probably guess that the colors in the picture are more similar to other colors in the picture. Essentially, the more neighboring colors, the more chance that it might be a picture of something, even though it probably isn’t anything.September 26, 2017 at 12:13 am #21841
I’d also like to wonder why it wouldn’t be better to order them, instead of completely random. I feel like it’s a giant waste of time looking at already impossibly random images!September 26, 2017 at 7:10 am #21845
Thank you both for your answers. It is difficult to say (at least for me) where the project may go in the future, or what purpose it may serve, but it is definitely an impressive technical achievement and conceptually or even philosophically mind-blowing.