Wed, Jan. 31st, 2007, 12:36 pm Ça plane pour moi ...

I think I'm fairly unusual here at LJ towers in that I wasn't plucked from the ranks of habitual LJ volunteers or long term users to come work full time. So in a way I have a sort of unique perspective on the interaction between us and our users - sometimes I kind of feel like the internet's Jane Goodall. Watching the responses to my first news post was a nigh-visceral experience. Friends of mine who've written books have said that they had a similar feelings watching the sales rank and comments on Amazon. It was kind of shocking to see how fast people responded. burr86 reminded me to turn off email notification and I said I'd do it when I came back from making some tea. "Now", he urged, "You need to do it now." By the time I came back from tea there was already 3 pages of comments. People CARE. Because most of my coworkers came from the community I don't think they make the distinction between them and us that I think the community at large does. They get affected. Some of the comments users make hurt them. Deeply. Because I'm often on a different timezone to everyone else I've sat on the end of IM with people who can't sleep out of distress. This is the flipside of The Cluetrain Manifesto that nobody talked about. Part of the problem, I think, is that, no matter how hard you try, you can always communicate better. There's always going to be a filter between us and the users. And there's always going to be people who fear the worst or who willfully interpret every action as being done out of malice or a very vocal few who take their own personal views and ascribe it to everyone. And it's not just LJ. It's everywhere. I have a several friends who work at Flickr - all long term Flickr users to a (wo)man. Today they're dealing with this thread about the deadline to finally move to a combined Yahoo! ID. Like us they've been dealing with accusations of selling out to a larger company (ignoring the obvious that, without the larger company, both of us would have probably ended up shutting down due to overwhelmed resources - technical and personal) and like us with, say, editing comments there are features that users are clamouring for (sets-of-sets is one, of the top of my head, for them) which the users can't understand why the team has not given them. On the one hand I'm baffled by some of the attitudes - logically we must either have a good reason, be lazy, be incompetent or be maliciously withholding a feature because we don't want the users to have it. For some unknown and inscrutable purpose. And some users clearly don't think that it's the first and I think our track record (hopefully) shows that it's not the second or third so they must think it's the fourth. Which just seems ... odd. Do we really come across as malicious? On the other hand I've been on the other side and so I can understand. I can really understand. I'm a geek and a user and a customer and I'm passionate about the things that matter to me. And, amonst many other thing, LJ matters to me. I use it everyday. We use it for work. I feel a burning urge to make it completely awesome. I get defensive about it with other people. When I'm back in England and I'm talking to my (largely LJ using, nay OBSESSED) friends my eyes shine when I talk about it. I really want to finish search now because I've got a really, REALLY cool idea I want to prototype and get signed off which I hope will completely rock everyone's world. I have no conclusion apart from I wish I had a magic way to make the users trust 'us'. Not that I think it needs saying but, naturally, these are my views, not necessarily the views of LiveJournal or SixApart.
Wed, Jan. 31st, 2007 09:53 pm (UTC)
henrylyne

That "burning urge" you have? Might want to get that looked at. ;) Wed, Jan. 31st, 2007 10:23 pm (UTC)
jimramsey

> here at LJ towers I like to think LJ users imagine the Six Apart offices as a Barad-dûr-like fortress with Mount Doom about where the Caltrain station is > Some of the comments users make hurt them. Deeply. Ah yes. After the new update page was launched was not a happy time for me. I felt better after Abe explained that it's an initiation into the land of LJ. Thu, Feb. 1st, 2007 12:28 am (UTC)
rahaeli

See, I really think that when new people start on LJ, someone should sit them down and have the conversation that starts "okay, see these emotional scars here? Yeah, that was October of '04..." (Time frame pulled out of my ass, but I bet that if I looked it up, there would have been *some* drama going on then.) And that's one of the reasons why I think New Business People who are just starting at 6A come in all shiny-eyed, going to Make LJ Better, and all the rest of us come off as being obstacles, because we know that their innovations -- which make perfect sense in the light of whatever dot.com experience they've had -- are going to spark off an epic battle roughly akin to the siege of Helm's Deep. And then we do it anyway, and -- oh, look, there are the orcs. Thu, Feb. 1st, 2007 12:30 am (UTC)
burr86

October '04 was *still* cleanup from September '04. :) Thu, Feb. 1st, 2007 12:35 am (UTC)
jimramsey

what happened in Sept '04? Sun, Feb. 4th, 2007 06:27 pm (UTC)
diffrentcolours

The cleanup from Aug '04. Thu, Feb. 1st, 2007 12:35 am (UTC)
jimramsey

I <3 LOTR metaphors :) Thu, Feb. 1st, 2007 12:45 am (UTC)
rahaeli

It's about the only comparison that's epic enough sometimes :P Thu, Feb. 1st, 2007 09:57 pm (UTC)
kimmi8

beware the orcs! damn, i only have battle scars from July of '05. but i wear them proud, especially in meetings when others laugh at my little scars. they run deep, DEEP, i say! Thu, Feb. 1st, 2007 10:05 pm (UTC)
rahaeli

But that's the beauty of working here! There's always another battle to scar you. (I'm still smarting from April of '02, just a little, deep down. heh.) I will never laugh at your scars! You earned them with honor. *g* Thu, Feb. 1st, 2007 12:21 am (UTC)
rahaeli

I wish I could make this required reading before commenting in news. Thu, Feb. 1st, 2007 05:14 am (UTC)
anildash

this is so, so good. do you mind if I link to this publicly? Either on my personal blog or maybe on 6A.com? Thu, Feb. 1st, 2007 07:27 am (UTC)
deflatermouse

Err, sure. Go for it. Either/both is fine. Thu, Feb. 1st, 2007 10:08 am (UTC)
susumu
"Ignoring the obvious that, without the larger company, both of us would have probably ended up shutting down due to overwhelmed resources - technical and personal"This is the key part of it for me. If everyone who bitched about these "conspiratorial" sell-outs, changes or limits actually left the service they were moaning about for an alternative, how long would they have before that one either implodes or attracts the attention of Google's chequebook? It's nice to feel like you're part of something slightly underground and individual but, realistically, popular things cost money to run. I came to LJ later than many (although still before it was bought by Six Apart), but I am Flickr "old skool". Yeah it's pretty annoying that I need to link my Flickr account to a Yahoo ID now, especially since my Yahoo ID I've had forever is already linked to my secret other Flickr account, but if it means I can carry on paying £13 a year for literally more Web 2.0 than I know what to do with, I'm happy. Thu, Feb. 1st, 2007 11:30 am (UTC)
boyofbadgers

Sadly I don't think there is a way to make all your users trust you. When you've got the size and variety of userbase that LJ has, whatever you do is going to do piss someone off. All you can do (I mean you as the LJ hivemind) is give people fair warning that something is going to happen and pay attention when they complain. From my own perspective, the thing that currently annoys me most about LJ is that very little apparent effort is going into making the basic functionality work properly. The proportion of comment posts that go awry, or poll answer submissions that don't work is getting ridiculous, and nothing seems to be being done about it. All we ever hear about via news is exciting new feature X, and when X = getting a bunch of flowers on yr userinfo, it rankles all the more when my next comment disappears into the ether. Just telling us what you're doing to fix stuff would make an awful lot of us a lot happier. Oh dear. This comment seems to have turned into an instance of what you were talking about. I'll shut up now. Thu, Feb. 1st, 2007 06:47 pm (UTC)
deflatermouse

So there's two points there - the giving people a warning and whether to work on new features (trivial or otherwise) or work on scaleablity. So the warning thing. Fair cop. Our only defense is that announcing new features before they've been finalised can lead to overpromising but really, yeah, we can do better. So, the second thing - the scaleability problems. It's unfair to say we're not working on it. We work on it every day. We've had to invent new technologies (mogilefs, Perlbal, memcached) just to keep up which we then release for free and are used by other large sites. We had to invent our own mail solution to deal with the amount of outbound messages we deal with. It's not so much that we're not working on it it's more that every time we give ourselves some breathing space we get more users and more traffic and ... You say basic functionality, we say a huge cluster of DB and Web machines, massive amounts of bandwidth and working on the cutting edge of what some of the technologies we use can do. If we weren't working hard on scaling then LJ would have fallen over a long time ago. You've also got to remember that we're small. From the outside it may seem like we're some huge conglomerate with battery pens of programmers slavishly working away. But if you look at the holiday post picture then that's it. That's LJ. And two of those people have moved ointo bigger and better things. And technically I don't even work for LJ any more - I work for the internal RnD group. So that leaves, err, 5 engineers. Things like vgifts - they're a bit of fun and, in the grand scheme of things they don't take very long and they let the team do something a bit silly which stops them going insane. Flickr got the same thing with the whole "Ho Ho Ho Hat" malarkey over Christmas. One complaint we get is over LJTalk. What people forget is that brad and crucially wrote DJabberd in their spare time. Integrating it into LJ was really trivial (basically an authentication and roster plugin which used existing LJ apis) and, of course, for the people who don't want to use it then they never even have to see it. So for us it rankles a little when people complain about it. However, despite all that, you're right (in a way at least) in that you have every right to get knarked when you post a comment and it just vanishes. We can only promise to try (even) harder (THE BEATINGS WILL STOP WHEN MORALE IMPROVES!). Thu, Feb. 1st, 2007 10:09 pm (UTC)
boyofbadgers

Ah, man, reading my comment back, I didn't emphasise that 'apparent' enough. I really wasn't trying to accuse you guys of not working on scalability (tho it does read like that, sorry) - my main point was that this work goes unannounced and so is invisible to most users. And that this lack of visibility then leads to people assuming that all the dev time is being spent on Rad! New! Features! rather than improving the services that they paid for. Which is an untrue and unfair assumption, but given the relative lack of info, an understandable one. Thu, Feb. 1st, 2007 11:57 pm (UTC)
deflatermouse

Ah, yes. Re-reading I get what you're saying. Actually, that's kind of a tricky question - should we post to news and say "So, we squeezed another 3% speedup from the DBs this month. Not that you'll notice but, hey"? That seems like it'd get real old, real fast but on the other hand then at least there'd be a sense we were doing something. I shall have a ponder. Fri, Feb. 2nd, 2007 03:34 pm (UTC)
julietk

FWIW I think this would be a Good Thing. A monthly roundup of the dull-&-boring (but useful & practical!) changes would I think make it clearer that stuff *is* going on behind the scenes, and would increase tolerance. Maybe also an update on just how *difficult* the whole thing is ("the servers are 3% faster but we've also had a 5% increase in post volume" or whatever), as I don't think people realise that either. I certainly wasn't aware (although I guess it would have occurred to me if I'd thought about it for any length of time) that LJ is running so close to the technological edge. Making that more obviously public might generate a bit more goodwill as well. It seems to me a bit like train delays: if one is stuck just outside of Middlesborough or whatever then obviously that's going to suck whatever. But it does IME suck a lot *less* when you have the driver explaining what's going on (even when they're saying "I'm not quite sure what the problem is, but I'll keep you updated" or "There's a lorry stuck under a bridge and we're waiting for the engineer to haul ass over here & I'm sorry but I have no idea how long it'll take - but I'll let you know when I hear more!" (why yes, that *did* happen to me) ) than when there's just radio silence. Mon, Feb. 5th, 2007 04:19 pm (UTC)
2shortplanks

Maybe this kind of dull stuff wouldn't be any use to the average LJ user. But to me? I'd be interested. As would all of the other people who spend all their time working on scaling sites would to. Wed, Feb. 14th, 2007 11:56 pm (UTC)
searchtools

Actually, yeah, being a bit more open (current buzzeword: transparent) tells people where the effort is going. It educates us about the problem and gives us a feeling for how difficult these giganto-scale issues are. Context is powerful! Thu, Feb. 15th, 2007 03:32 am (UTC)
matgb
What people forget is that [info]brad and [info]crucially wrote DJabberd in their spare time The thing is? I follow a lot of news and meta LJ discussion comms. That's the first I heard it was a spare time project. If that had been said, then it would probably have come across so much better, because Brad's known for doing that sort of thing. Ah well, water under the bridge. Thu, Feb. 15th, 2007 07:41 pm (UTC)
deflatermouse

It didn't go in the official LJ news and meta discussions because, well, it wasn't an official LJ thing ;) Most of the info was on brad's journal - the initial announcement was here I think and you can follow the development using the 'djabberd' tagBut yeah, when we announced it we probably should have mentioned that it was something they'd done in their spare time. Thu, Feb. 1st, 2007 12:25 pm (UTC)
natalief

Now I feel bad for the tone of my comment on that news post. It is just that with MS and ADD, any change is painful whether it be for the good or not... Thu, Feb. 1st, 2007 07:01 pm (UTC)
peshwengi

Wow, it's strange how demanding the users can be. Personally I like it and I don't expect to see more features considering how... err... free LJ is. I support and write software for traders - if anything goes wrong or they don't have the "features" they need, they can be losing thousands of dollars a minute.... but they are very reasonable about their demands. Which also surprises me! Thu, Feb. 1st, 2007 07:01 pm (UTC)
revmischa

i hope after search you can work on lj, not just cross-platform stuff Thu, Feb. 15th, 2007 08:46 pm (UTC)
ladydiana

I think you have many good points, but from my perspective as a user, "FLASHY BRIGHT FLASHY STYLE WITH LOUD AND OFFKEY MUSIC THAT LOOKS LIKE A BOTTLE OF PEPTO BISMOL EXPLODED OVER A KITTEN!" (for example) does *not* constitute "improving the site", and that's pretty much what some of the styles look like to me. I know there are others that disagree with me. That's perfectly fine. But to say "we've improved the site because we've added #xyz styles!" - and have things like, say, full-text searching, which people have been *begging* for for five years, and a supported backup program, which people have been *begging* for for four years - it's like, your (collective) priorities are for as much flashy bright flashy as possible, and mine are as little flashy bright flashy as possible, and of course there's going to be a disparity there. The Valentine's Day thing didn't really bother me, but it looked TRULY, VIOLENTLY DISGUSTING with my colour scheme. Fine, I got over it, after wincing and cringing. Telling people that "you can't opt out of these schemes because we don't want you to have the right", on the other hand, is not so cool. Some of the things in suggestions, which I've lately referred to as the "black hole of LJ" (and don't even bother submitting there any more because it's a waste of my time), were "WNBI"'d FIVE YEARS AGO because the architechture at the time couldn't support it. Or because ONE PERSON (usually brad) either didn't like the idea or didn't personally have a use for it. Has anyone thought to revisit those suggestions, and see what can be implemented now? And what effort is being given to make those which *can't* be implemented now be available at a later date? That might draw some attention away from what *some* users are complaining about. Thu, Feb. 15th, 2007 11:58 pm (UTC)
deflatermouse

Hmm, excellent points. I'll try and deal with them vague in order. So, yes, the styles thing is, well, subjective. The Valentines Day scheme look disgusting with my theme as well. Search - well, I'm working on it now. The problem is that full text search is a huge problem. It would have been far too much work to dedicate to up till now - there's no way that the team could have spared someone full time to come up with a solution especially since very few people have any experience with large search engines. What people might think is easy ( "Why don't you just use MySQL's built in search") actually isn't. Backup - not sure. I'll ask though. It's a strong possibility that it just hasn't reached the top of the TODO list. Suggestions - it's a good idea and despite what you may think we do pay attention (look at the last news post for example) but it's hugely time consuming to try and keep up and reply to everything. And that is basically, the crux of the problem. The LJ team isn't big and the proportion of users who pay us in some way (permanent, paid or plus accounts) is small so we can't get much bigger. So we can't do everything so we have to make choices. Sometimes the choices we make aren't the right one. Actually since there's always some people who want the feature and some people who don't we're always going to be wrong in someone's eyes. We could do more but that would probably involve getting rid of free accounts, increasing advertising or increasing fees. And we don't want to do any of those. If we get some time it's not a bad idea to go over some of the old suggestions and see what's already been done or could be done. I'll try and look into that. Sat, Feb. 17th, 2007 01:36 pm (UTC)
sasuko
I know I'm not part of your friends list, and this is your journal and you are speaking as an individual rather than as part of the company, but there were some thoughts that I had on the issue which I'd like to share, and I'm hoping it's all right? Heck, just in case, I'll put it on my own journal here, so that you can choose if you want to read it (less it borders harassment). I guess the main thing that I've really talked about there is my feelings on the policy of having plus accounts (and I'd admit that I'm still not able to talk about that in a nice manner). At any rate, I just thought I'd thank you for making this post, because it helped me see things a little clearer at any rate. So thank you for this, and for any harsh words that I might have said to you or the team for my misconceptions, I just thought I'd apologise. Tue, Feb. 20th, 2007 08:16 pm (UTC)
deflatermouse

First off, no worries about commenting or anything. It's why I write these sorts of things[*] - as evidenced by other comments to this post keeping the community informed about stuff is hard and there's a fine balance but, at least this way, the information is out there. So there's two things I think you're bringing up (I'm tired from driving back from Tahoe yesterday so if I've missed anything then let me know). I'll deal with them in no particular order. First - Plus accounts. It sucks that we had to break the social contract. Yes Brad said "No adverts". But that was when LJ was small. Things are different now. It sucks. It really, really sucks. I wasn't here for the buy out of LJ nor for the decision to create plus accounts so I really can't speak authoratitively for anyone. My understanding is that without the sale to SixApart LJ would have gone under a long time ago. Put simply - the emotional and financial strain was just too much for Brad and Danga to bear. And LJ needs to make money to pay for staff salaries and pay for bandwidth and hardware costs. And paid accounts are a relatively small proportion of people. As far as I know the choices were - adverts, increase paid account fees or have limited time span free accounts. All of those suck but the last two would have had far more of a deleterious effect on the social dynamic of the site. So it had to be the first (I suppose there could have been other options like asking people to pay on a scale depending on income or something but my guess would be that they just wouldn't work). In fact I think the solution is both unique and innovative. Nothing changes unless you happen to want more features. Then you have to give us money. Either flat out or by watching adverts. So until then if you didn't wnat to see ads then you wouldn't. Just like the old days. Then two bones of contention - sponsored communities and ads on the front page. Sponsored Communities - this one took me by suprise. As far as I know the companies who wanted a sponsored community could have just bought a paid account like anyone else and then that would have been that. But, instead, they gave us more money which we can then put back into the site. So the end result is the same except that we've got more money to spend and we get to mark it as commerical so people can avoid it. I thought that was a win-win situation but, after the announcment I started to see why people were upset. I still think it's an ok thing to do but I can see why people don't. Ads on the front page - again. They suck. But only a tiny percentage of paid users will ever see them because most stay logged in all the time. And when they see them it's only for a second. But they get us money which, unfortunately, we need. Of course, both my arguments rely on the fact that I feel ads are a necessary evil. And this is from me. I left a previous company due to the fact that I didn't want to be involved in internet advertising. I've ranted and raved on mailing lists, at conferences, in letters to magazines. I've flamed marketing people till bile came out of my eyeballs and I gave myself an ulcer. But the alternative is shut down LJ or get more people to sign up for paid accounts. Second part continues below. [*] Well, actually 90% of the content of this journal used to be in a private diary that nobody else could read apart from me. When i was writing my dissertation I found that writing a diary as if addressing an imaginary audience helped me sort stuff out in my brain - sort of like debugging with teddy - when I came to work at LJ I transferred them all over because I figured they may be useful. Tue, Feb. 20th, 2007 08:17 pm (UTC)
deflatermouse

Secondly - opt out stuff. Most of the stuff we do is opt out. But some isn't. And it's not like we're doing it just to spite the users. Making somehting opt out creates more complexity in the code and soon the number of combinations of features becomes exponential. And when that happens the code becomes brittle. Adding new features or refactoring the code becomes more complicated. The sit ebecomes slower. And you get more bugs. You get more bugs because the number of subtle interactions goes through the roof and code is a slippery eel-esque substance and subtle interactions tend to go wrong. See for example this post on threading on multicore CPUs. Not only that but bugs get through because programmers are humans and humans are soft fleshy meat sacks that make mistakes. Those mistakes are usually caught by testing but testers are humans as well and the more combinations of features we have the more likely it is that a bug will slip through. And when we do find a bug the code is so complex that it takes for ever to fix. And the fix may introduce more bugs and so on and so on and so on ... So, we try and keep down the combinations so that LJ isn't slower and buggier than it is. "But why?!!" you scream, "Why add more features, we like it just the way it is!". Because, well, because people actually like new features - it's actually a relatively small vocal minority who don't. And without new features the site stagnates and when it stagnates it starts to die off and eventually we'd go out of business. Also, the more options we give people the more complicated the site is to use for new users. And it's already complicated enough. I'm terrified about rolling out search. I want to give people exactly the level of privacy that they want ranging from all out paranoia "thou shalt never index any of my posts at all" to personal "Only I can search my posts" all the way to "what the hell, let any body search" but I'm worried that the more options there are the more likelyhood there is that someone will put the worng level of privacy on and leak something they didn't wnat or worse (from our point of view) we'll introduce a bug because of complexity and suddenly we'll have a mass privacy issue on our hands. What do I do? Control or potential for disaster? I lie awake at night worrying about this sort of stuff. It's killing me. Possibly in a literal way if insomnia and stomach ulcers can kill. Let me know if there's anything I haven't answered or you have any other questions or you want to call bullshit on any of my points. Simon Wed, Feb. 21st, 2007 01:30 am (UTC)
sasuko
Hi, thanks for your reply. I've made my response back here if you're interested (if not, no worries). I guess I prefer it to be at a location where I have control over (because the last thing I want is a flame war between myself and some unknown entity. If the issue started between you and I, I would hope to keep it that way instead of getting friends involved. That's the reason why I typically keep all comments except my own screened, by the way, in case you're wondering). At any rate, thank you for your response thus far, and I wanted to tell you that it's been appreciated. Wed, Feb. 21st, 2007 01:20 am (UTC)
solitudinarian
"Nothing changes unless you happen to want more features. Then you have to give us money. Either flat out or by watching adverts.
So until then if you didn't wnat to see ads then you wouldn't. Just like the old days."You can keep repeating this claim just like gets done in all the official posts, support answers and other staff comments. It still doesn't make it true, though. Fri, Feb. 23rd, 2007 10:32 pm (UTC)
deflatermouse

I'm not sure which bit you disagree with. Unless you're a plus user or logged out then you shouldn't see ads (unless you count a sponsored community as an ad, but that's a different kettle of fish alltogether). Yes, the ads on the front page are a (fairly recent) bone of contention - is it that much of an inconvience to you or are you just morally opposed to any advertising at all (and I'll accept that as a valid argument)? Sat, Feb. 24th, 2007 12:08 am (UTC)
solitudinarian

Um, that's not how the account levels and ads work. If it really was only plus and logged out users then I'd think a lot less people would have had a problem with it. I know I wouldn't really care... As it is, I can't even follow discussion in some of the official communities like news or feedback without being subjected to your advertising. The same with every plus user's or community's page. So, how exactly is that "just like the old days"? Sun, Feb. 25th, 2007 09:36 am (UTC)
lusercop

That you're working for LJ/SixApart explains why you've disappeared. Still, let me know next time you are around Central London. :-) Sun, Mar. 4th, 2007 08:02 am (UTC)
dimajazz: Volunteer
I'd like to search or research if I could understand more clearly exactly what you're looking for. If you could leave a definition of what you are looking for please do not hesitate to leave a comment in my journal. Diana Marie Chapman dimajazz I only heard about Live Journal a few days ago, liked what I saw & started an unpaid account. Diana |
|