Friday, August 12, 2005

Capturing ViewRank

One solution is to ensure that all applications broadcast event information such as document opening, scrolling, editing etc. But this approach has a problem. We will have to wait until all these applications are ready to broadcast this information to the dashboard.

Another approach could be to use screen scraping. Take a print screen of the screen, every second and save it as a compressed image automatically into some free memory or a particular folder in the hard disk. Along with this use a tiny invisible program that stores any keyboard and mouse events for that second. This info can be obtained easily windows hooks…

Analyze the image using primitive OCR software to extract all text from the images. I say primitive because extracting text from a print screen is far easier than extracting the same from a scanned image because of the absence of noise. The co-ordinates of the windows, captured should provide clear set of heuristics for compression searching for text etc. Also we have APIs that return us the DOM of many documents and GetWindowText Function that get us info from many GUI elements. This text would be added to a text index.

Some parts of these snaps shall be extremely compressible.

  • For Eg any Windows Explorer window can be just remembered as the set of icons + filenames + attributes + foldernames.

  • Only differences in the screen b/w adjacent shots are stored.

  • Movies, Games etc. can be ignored.

  • Snaps of plain documents, dialog boxes, Web Pages etc are typically highly compressible.

  • Windows containing command prompts etc can be primarity remembered as just strings.


Simple back of the envelope calculations show that even if we wish to retain most of the pictorial information it would generate roughly about a 10MB per hour -100 MB per day or 3GB per month - 36 GB per year. Considering the fact that hard drives these days can store terabytes of data, this is really insignificant. By the time the View rank solution is ready, I guess we would have gone thru another pair of Moore’s law cycles.

If we store only text the sizes would be much smaller of course. A lifetime worth of text would definitely fit onto a single CD or DVD.