?

Log in

No account? Create an account

Fri, Sep. 8th, 2006, 03:59 pm
Fear and Loathing in Java Land

Hmm, my first post since I started working for the hallowed Ell Jay nearly two weeks ago. Two weeks that have seen me nose deep in obscure research documents and frequently looking at various stats about the number of users, posts and comments on the system, doing some back of the paper calculations and then blanching deeply before taking a long slug from my hip flask and staring dispondently off into space.

LJ is big. Big. BIG even. Like really big. To paraphrase Douglas Adam - you just won't believe how vastly hugely mind-bogglingly big it is. I mean, you may think that your hard drive has a lot of pr0n on it, but that's just peanuts compared to the amount of data on LJ. Let alone thinking about including Vox.

It's enough to make a transplanted English geek turn to drink.

Oh. Wait.

The code is starting to come together though. Part of it is in Java. Don't ask. I'm hoping it won't stay that way very long because, trust me, I don't like it any more than you do. Strings are so HEAVY. Really heavy. You think that ...

... anyway, so part of it's in Java. I'd forgotten how verbose Java and un-DWIMish it is and how much the library system sucks but, oddly, a night of crippling insomnia (staring into the black holes that are used to store LJs databases will do that to you) actually seems to have helped me get over that. My latest problem was RMI which was giving me some really odd errors


java.io.StreamCorruptedException: Unexpected blockData
at java.io.ObjectInputStream.readObject() (/usr/lib64/libgcj.so.5.0.0)
at gnu.java.rmi.server.RMIObjectInputStream.readValue(java.lang.Class) (/usr/lib64/libgcj.so.5.0.0)
at gnu.java.rmi.server.UnicastServerRef.incomingMessageCall(gnu.java.rmi.server.UnicastConnection, int, long) (/usr/lib64/libgcj.so.5.0.0)
at gnu.java.rmi.server.UnicastServer.incomingMessageCall(gnu.java.rmi.server.UnicastConnection) (/usr/lib64/libgcj.so.5.0.0)
at gnu.java.rmi.server.UnicastServer.dispatch(gnu.java.rmi.server.UnicastConnection) (/usr/lib64/libgcj.so.5.0.0)
at gnu.java.rmi.server.UnicastConnection.run() (/usr/lib64/libgcj.so.5.0.0)
at java.lang.Thread.run() (/usr/lib64/libgcj.so.5.0.0)
at _Jv_ThreadRun(java.lang.Thread) (/usr/lib64/libgcj.so.5.0.0)
at GC_start_routine (/usr/lib64/libgcj.so.5.0.0)
at __clone (/lib64/tls/libc-2.3.4.so)
Couldn't search java.io.StreamCorruptedException: Unexpected blockData



I mean ... WTF?

For future reference the answer turned out tobe in the /usr/lib64/libgcj.so.5.0.0 bit. GIJ (The GNU interpreter for Java) just seems to plain suck. Switching to Sun Java fixed that problem and a bunch of other oddities.

And there ends the first lesson^W post. Have fun slumbering Londonites - I'll be back drinking with you horrible shower in a couple of weeks.

Sat, Sep. 9th, 2006 12:30 am (UTC)
matthew

No one truly understands how big it really is until they get to see it from the perspective that you just did. Turning to drink seems to be a common response. I just have to back it all up every week.
(Deleted comment)
(Deleted comment)

Sat, Sep. 9th, 2006 07:31 am (UTC)
rjw1

have you been properly brain washed yet?

Sat, Sep. 9th, 2006 02:17 pm (UTC)
aca

The horrible shower misses you dude. :) You'll have to swing by the Ship at some point soon to help them celebrate their Good Beer Guide 2007 entry!

Sun, Sep. 10th, 2006 11:30 am (UTC)
pfig

meh. i'd be up in cornhill giving bjs to suits sooner than caught using java, even if that meant working for lj.

but the peep in me likes to look at blackholes. can't you give us some numbers? any numbers, it´s nice to have that kind of info to chat up women.

Mon, Sep. 11th, 2006 08:22 am (UTC)
billyabbott

I just moved offices away from Cornhill and guess what I have to write later this week?

Mon, Sep. 11th, 2006 10:03 am (UTC)
(Anonymous)

a bj how-to?

Mon, Sep. 11th, 2006 09:15 pm (UTC)
deflatermouse

I can't go into too many details but *very* roughly - 4Tb of compressed data representing over a billion items and in the region of 15 new events a second.

In contrast Google and Yahoo claim to have around 19 billion items in their indexes. And large search teams. And thousands of search boxes. All LJ have is me and an old 486DX2 they found lying around and put down in the pit with me. Oh, and the big boys don't have the whole user ACL stuff to deal with either.

Still, I'm sure it will all be ok. I just wish crucially would stop spraying me with the hose and shouting "skynda! skynda! slita!" all the time.

Mon, Sep. 11th, 2006 06:47 pm (UTC)
wildilocks

*waves*

Thanks for add and all the fun chats and drinks over the w/end. Hope if you actually read my LJ it amuses you rather than bores you ^_^

Thu, Nov. 23rd, 2006 03:46 pm (UTC)
pfig

oh, sugar!

if you're a suit, i'll be in cornhill next week giving free head.