Secure Surveilance Camera

Have you watched impossible heists on Discovery?

In it a surveillance camera is replaced by one that transmits a fake image by the intruders and the security officers have no clue that they are staring at a fake image. Another trick is to physically hang a picture in front of the camera to fool the security officers.

If the camera has the ability to digitally sign the photo using cryptographic keys embedded within the camera, then a replaced camera can be instantly detected and an alarm generated.

The focusing apparatus of the camera can determine the distance of the object and generate a warning if most of the camera is covered by an image at close distance.

You may also be interested in checking out my earlier suggestion about GPS camera phones.

ViewRank

On the one hand we see continuous exponential growth in computing capabilities of hardware as doubling of clock frequency, doubling of memory, bandwidth, number of cores etc. On the other hand we seem to be working harder and harder to please the computer than the other way round. Yet another amusing fact is that few people realize that the processor is idle most of the time in-spite of all the bloat-ware we run on it.  A search indexing service like “Google Desktop Search” is an example of a program that utilizes this spare processing power to make the life of user a little more pleasant. Here is a suggestion to take that one level further.

Yet another interesting application is dashboard or implicit query as Microsoft calls it. It actually enhances the user experience by proactively suggesting relevant and useful information based upon what the user is doing at that moment.

The computer deals with a huge set of files. It has no clue about what each file actually represents and its importance with respect to the user. Each action performed by a user implies a lot of things about the relevance of a certain piece of data to that person. However such information is typically never captured.

1. The importance of a document depends upon the number of times it was seen by the user. Here ‘seen’ is an important word. If the user does not see it, it does not matter in the imaginary universe of the user.. For Eg a configuration file that the user indirectly uses but has never seen, is irrelevant from the user’s point of view… However a configuration file that was edited by the user is very important to him.


2. The duration for which the document was open indicates its importance to the user.


3. The importance of a section of a document depends on the amount of time that section was visible on the screen.


4. The importance of a section of a document depends on the amount of visible area allocated to it on the screen..


5. The portion of the document that the user edits is more important to him than some portion of a document he just glanced at.


6. Every single keystroke, every single mouse click, every single meter of mouse movement that was physically done by the user is much more precious than Gigabytes of junk he copies in and out of the system.


7. The stuff that he dealt with more recently is usually more important to him than the stuff that he dealt with in the distant past.


8. Error messages that one frequently encounters are more important to the user than a huge list of errors that could possibly be generated.


9. A folder that a person keeps visiting frequently is more important to him than some subfolder in some installation directory he has no clue about. Here visiting could mean navigating through a file open dialog or using the windows explorer or File Manager or other utility or could be a cd using the command prompt.

In this context I introduce a (possibly) new concept. I call it view rank. The above mentioned heuristics at judging the relevance of data could dramatically improve the relevance of the hints and clues provided.

I call the above heuristics as viewrank. Each section of each file has an associated viewrank. When you use Google Desktop Search you could possibly say “Sort in the order of importance to me.” This would actually sort results in the order of the viewrank of the documents.

How many times have you muttered to yourself… I have seen this somewhere on my computer. I don’t know when and where. Now you can have a feature that says “Search in all documents and files that I have seen at least once and arrange it in the order of my viewrank.” It must be obvious to you that the number of files and documents you have actually “spent time with” is a very small subset of all files that exist. Thus you are much more likely to get the perfect hit. For Eg. You would have spent a lot more time on your resume than a leave letter.

Capturing ViewRank

One solution is to ensure that all applications broadcast event information such as document opening, scrolling, editing etc. But this approach has a problem. We will have to wait until all these applications are ready to broadcast this information to the dashboard.

Another approach could be to use screen scraping. Take a print screen of the screen, every second and save it as a compressed image automatically into some free memory or a particular folder in the hard disk. Along with this use a tiny invisible program that stores any keyboard and mouse events for that second. This info can be obtained easily windows hooks…

Analyze the image using primitive OCR software to extract all text from the images. I say primitive because extracting text from a print screen is far easier than extracting the same from a scanned image because of the absence of noise. The co-ordinates of the windows, captured should provide clear set of heuristics for compression searching for text etc. Also we have APIs that return us the DOM of many documents and GetWindowText Function that get us info from many GUI elements. This text would be added to a text index.

Some parts of these snaps shall be extremely compressible.

• For Eg any Windows Explorer window can be just remembered as the set of icons + filenames + attributes + foldernames.


• Only differences in the screen b/w adjacent shots are stored.


• Movies, Games etc. can be ignored.


• Snaps of plain documents, dialog boxes, Web Pages etc are typically highly compressible.


• Windows containing command prompts etc can be primarity remembered as just strings.

Simple back of the envelope calculations show that even if we wish to retain most of the pictorial information it would generate roughly about a 10MB per hour -100 MB per day or 3GB per month - 36 GB per year. Considering the fact that hard drives these days can store terabytes of data, this is really insignificant. By the time the View rank solution is ready, I guess we would have gone thru another pair of Moore’s law cycles.

If we store only text the sizes would be much smaller of course. A lifetime worth of text would definitely fit onto a single CD or DVD.

Using ViewRank

It is like a virtual movie of your entire interaction with the computer. You can travel back in time to any day you choose and see what you had done. All the text that is visible will be actually copy-able. Any
hyperlinks identified in the images will be clickable etc. Remember this would include Email messages, Dialog boxes, Wizards, Folder navigation, interaction with IDEs, the command prompt, telnet sessions etc.

The information thus captured will enable your dashboard to give context sensitive help and advice. For Eg you might have worked on a defect called 23127. You might have info about this on your IM, Mail, and Code etc. When you encounter 23127 you could actually be presented with relevant info in your dashboard.

Your Desktop Search will now also be able to search within everything you have actually “seen” in the specified period. This is very different from whether the string is in the files whose last access date is within a certain range. Also the ordering would be in the order of the time and effort you spent with it.

The Screen Captures usually make it a lot easier to reproduce bugs. In fact one does not even need to redo the action. All one needs to do is to just copy the relevant screen shots and mail it forward. 

ViewRank FAQ

What about privacy?

Your computer has so many programs installed on it. Your sole solution for privacy is installing applications like Antivirus programs, firewalls etc. You trust that these programs themselves are not hostile to you. If the antivirus provider himself steals your info there could be very little you could do. But the logic is that stealing your info would hurt the business of an OS company or Antivirus Company or whatever. So they would not steal your info. After it is just a matter of writing clean code. As far as your privacy concerns you should be no more scared than installing an OS without seeing the source code or installing an Antivirus or Desktop search without looking at the source code.

In fact you can be more confident about your privacy than letting your email reside on a remote server where you are forced to trust what they tell you about what they do with your mail. A Desktop application is very different. If it is sending out your private info, security companies get a chance to shred the applications to bits and examine malicious behavior. So the probability of mischief is much lesser near zero. If it is Open Source it is even better.

What about all the love letters, porn, the hacking sites, all the info I never want captured anywhere?
You can instruct the ViewRank Service to pretend as if some programs were never launched. For Eg. You could have two or more browsers on your system. Use one for all useful work you do for Firefox. Use another browser say IE for all activities you are uncomfortable being recorded. The recording will never capture info/snapshots about the particular application.

Of course in case it does capture any info that u think can cause problems you can always delete it using an interface.

What about efficiency?
The computer can use the wasted CPU cycles to do the processing just like all the Peer to Peer Programs, SETI@Home etc. The user will notice little or no performance degradation as no processing will be done when
the processor is busy.

An Open source application could be written based on these ideas that integrate with dash board. Some company could possibly come up with a product that does all this.

Do you feel there are any gaps in the solution that I suggest here? Would u use it? Can you tell any demerits that you see here? Do you have suggestions for improvements?