I've been thinking about latent meta data for a long time. A long time. Partly that's because it's such a large and vague topic - the amount of data is large and meta covers everything that you can infer about it.
In this case I've been thinking about how we can write tools to help us understand all the personal data we have knocking around. We have mountains of emails and contacts and web surfing history and conversations and other miscellania and the more we get the harder it is to organise yet perversely there are more rich informational pickings to sift over.
I've written apps that listen on IRC and try and build a view of the world based on what's said. I've written stuff that indexes email corpuses and helps you rotate the data about any point. I've written secretarial bots that act as stenographers and note takers and who do calculations and lookups and go fetch things without you having to context switch, without you having to even ask in some cases, just like a really good PA should. I've written things that crawl Wikipedia and infer and answer questions.
I've talked (ranted, really) about this sort of software alot to my friends, sometimes to the long suffering Tom Insam who seems to bear the brunt of more than his fair share of my insane ravings and half baked ideas.
One of the things I got excited about was Beagle (née Dashboard), the Gnome program that allows you to search all your information from a single interface. I liked its novel use of Clue Packets but in some ways it felt stale - unlike Dashboard you had to go type something in whereas Dashboard would infer from what you were doing. Something about that bothered me - it wasn't new enough I suppose. It was just an evolution of Windows Search and Sherlock and Spotlight. I want a PA, not a reticent knowitall in the corner I have to coax answers out of.
Because he's not a whiny bitch like me and because he seems to have more JFDI than is humanely possible, Tom competely ignored all my frothing and has since come up with his own system - Shelf - which has been getting some heavy weight coverage recently. It harkens back to the Dashboard model and uses polling to workout what you're doing in what apps and heads from there. It's, to be frank, pretty sweet.
But I'm still not totally satisfied and it boils down to this. I get distracted enough - I have IMs and IRC and emails and feeds and phones going all the time and I have to be careful because suddenly it's 3 hours later and the cursor is still blink accusatorily on the blank editor.
What I actually want is something more Exposé like - I want to hit a key combination and everything that Shelf knows about the current context it can gracefully swoosh up in an achingly translucent overlay. The app can then either continuously scan what I'm doing or, for lower spec machines, it can just do its mojo on demand. This also solves the problem that, if you're running the Social Graph plugin it doesn't need to send every url you're looking at to Google (which, in and of itself is part of a wider Seperation of Personas theme of which more sometime in the future).
But not everyone's like me so seperate the frontend and the backend. That way I can have my Exposé mechanism and info junkies can mainline clues and we'll all be happy. Apps could subsume the functionality giving richer integration through a common broker.
Hell for those who just need to know what's going on in the same way Britney needs the limelight, you could make a meta app that streams their Shelf clues, and Twitter streams and RSS feeds and access logs and email notifications and Calendar updates in some sort of context firehose that you get to drink from. Throw in some special self-referential sauce so that it understands itself and Oh look Dave Winer's accessing my site and now he's written a Twitter about it and someone else has written a blog post about it and 3 of my friends have commented defending me and ...
It's the sort of Intertwingularity that gives Ted Nelson a full on chubby.