I've been wanting to play back the writing of an essay for years. Since Etherpad saves every keystroke, I convinced the founders to add a way to play them back. "Startups in 13 Sentences" was the first essay I wrote on Etherpad. Now I'm going to write all of them on it.
Playback is just one little feature of Etherpad, but think of the implications of this alone. Among other things it will make cheating impossible in classes where students write papers, because now you can finally "show your work" in writing the way you do in math.
I convinced the founders to add a way to play them back
You often talk about releasing early, building something people want, etc. Since you carry a lot of influence, and (I guess) invest in Etherpad, and they've adjusted it to your personal request - do you worry that this effect might give you a distorted view of how well Etherpad specifically, and YC startups generally, are building what people want?
Look at the attention this is getting Etherpad. I'm pretty sure this was a pretty high ROI day of work!
To further drive that thought home, my company hereby offers to add any 1-day-effort feature request made by any popular tech figure, provided they're willing to share our work with their audience. (Sites for those would be http://bug.gd or http://www.yumbunny.com or any other site in my profile.)
Seriously, they'd be silly not to have taken pg up on this even if he wasn't helping to back them.
Convincing someone of something you believe in and abusing your influence are two very different things.
I don't think pg implied that he bullied them into implementing this, only that he talked them into it. And I can't imagine that YC would last long if pg got into the habit of bullying its start-ups into implementing his personal requests.
Valid point, but I think it's worth looking at the potential benefits. The thing I always hate about writing is that I'm constantly saving copies off to the side. At the very least I would think snapshots would be invaluable, but once you have playback, you basically already have snapshots with high granularity.
While solutions like DropBox or Sharepoint allow versioning and collaboration it's at a higher level. Maybe I've been spoiled by VMWare, but I love having branches of specific versions to fall back on and test things out. Why should writing be any different?
This is an incredible thing to watch! Can you do this with any piece of writing on Etherpad?
What a great way of bringing attention to a start-up: make something that's inherently fascinating due to features. This is unlike anything online I've seen before because the functionality just doesn't exist anywhere else. So it's a great thing to watch and it also tells you about the service.
I like that you keep constantly going back to eliminate unnecessary language. Something tells me that's the first and most obvious trait of watching good writers write: at the governor's school, when the writers would write on paper instead of on the computer, you could read a draft and see that emerging pattern of phrase after phrase, each building on the last. It's interesting seeing it happen on the computer.
Can you do this with any piece of writing on Etherpad?
Anything you write with it now you will be able to play back like this soon. Etherpad is already saving all the keystrokes. They just haven't released the player yet. (This is a one-off prototype.)
This site shows how some fairly good illustrations are drawn (brushstroke playback). Unfortunately, not all drawings can be animated and the ability to search for playback-capable drawings is limited:
It would be interesting to have the timestamp data (if it is recorded) of the keystrokes (adjusted to author's timezone - privacy withstanding) so you can see a clock/calendar [day|night icon] when the author wrote it (was it 3am in the morning? did they put it aside for a couple hours or was it a period of days?).
However, it should be somewhere hidden and available only if people really want to be able to replay. Else it is going to be an over kill on their servers.
This is great, and the article itself is excellent. Regarding "going back to eliminate unnecessary language", as Stephen King said, "2nd Draft = 1st Draft – 10%".
Might I suggest, from the watcher's perspective, it'd be handy if the final feature sets the vertical scrollbar such that the currently playing change is on the screen?
It might be kind of jerky during the editing stage, but it's better than scrolling around trying to find out where the action is. ;)
What I learned from this is that I write very differently from the way you do. My first draft is almost straight stream of consciousness, and then I go back and reread and edit several times. You seem to make several attempts at each sentence before finding one you like, and then edit the whole thing.
Of course, I may just not realize how I actually write, since I don't have a playback of anything of length to watch.
What I'd really be interested in watching is some famous programmers working on code with this kind of playback. Not sure if it'd actually be enlightening, but it would be fun to experiment with.
I really liked this sentence I caught. "Launching lets you observe the collision between your presuppositions and the needs of actual users."
I know, every word should tell, but I wish it had made it.
That feature is neat and gives insight into the way people write. Would be just as interesting to see a playback of versioned source code files in SVN / CVS..
That would be an amazing training tool. I have learnt the most by observing great engineers code, than reading tons of material. It just strikes at you the way they fork functions, how they name variables, logic, the way they would cut out/undo unnecessary statements and combine them to optimize code. It just reflects the way good programmers think when they code.
If this were only available when you guys were writing code for Viaweb, and when you were writing On Lisp -- what a wonderful thing it would be to learn from. It would show the thought process behind what went into writing what you wrote!
"I doubt everyone will abandon their word processing software and user Etherpad."
They will probably give up Word for something, eventually. Especially among the current generation, writing is something done online; a blog, an email, a tweet, writing on someone's wall. At some point, it stops making sense to have a word processor distinct from the web. Right now, Word has more features that people like than online writing tools, but that will change over time. Online writing tools are a more natural fit for collaboration and integration with words' final destinations.
Microsoft could make the online tool that replaces Word, but it is probably too dangerous to their current revenue stream. I don't think they can get the same revenue for an online version of Word, and it would be difficult for them to rationalize trying a new business model that may never regain the revenue they get now.
(And yes, I'm basically just making the Innovator's Dilemma argument for word processors.)
Going off of the "show your work" idea, is it possible to analyze the writing style to prove that it was -you- who wrote it? Or is there too much variation? There was a bad episode of Numb3rs where they used someone's "typing profile" to identify someone, so it just got me thinking :)
If high-resolution timestamps for each keystroke aren't too much of a burden it seems like this would be straightforward to implement, given an algorithm like BioPassword's.
It looks like BioPassword works best on short, often repeated phrases like login names and passwords. I'm not sure how it would work for larger, unpracticed blocks of text. Etherpad would also need to store information about the timing of when each key is pressed, and based on the demo it doesn't look like they're doing that.
There are already non-webapp mouselog/keylog and replay applications. And besides, think of the implications of letting students be on the internet while they are writing essays.
It's different when it's built into a generic, web-based editor. Imagine the difference between
a) telling students to install a keylogger, write an essay with it turned on, then submit the essay plus the logs, and then the teacher figuring out how to replay the logs, versus
b) telling students to write their essay at etherpad.com, they submit the url, teacher clicks on "replay."
As for the matter of students being online, that's going to have to get solved anyway, because all the apps are moving online.
I think that pendulum has almost reached the end of it's arc. There are a number of trends that are starting to pull in the opposite direction. From concerns about privacy and security to the proliferation of mobile computing, intermittent disconnection from the cloud has it's uses.
Tools like Google Gears and the continued blurring of the line between desktop applications and web applications mean that it's easy to build applications where being connected enhances the experience considerably, but the essential functions are still there even if the network goes away.
"There are a number of trends that are starting to pull in the opposite direction. From concerns about privacy and security"
Ironically I think it's the tech crowd/geeks (us) who are most resistant to webapps. We can see the privacy,security,reliability issues. The average users however, cannot. They're already embracing webapps.
That might actually be evidence that the pendulum has almost reached the end of its arc. The tech crowd first embraced the web just as average users were embracing the PC. Windows 95 was the first really usable version of Windows, and it was also the year of Netscape's IPO. Mainstream users adopt last decade's technology just as the early adopters are finding a new playground.
The big question is what's next. I'm guessing it'll be mobile - the geeks are embracing it just as the mainstream is finally becoming comfortable with last decade's technology. But it's a bit too early to tell.
There are pendulums in fashion, but not, as far as I know, in technology. If you get pendulum-like behavior for a cycle, you have to assume it's a coincidence.
Cheating is a difficult and important problem, but I'm not sure this solution would be worth the inconvenience it would cause. If it only inconvenienced cheaters that would be fine -- but I think it would also cause some honest students and teachers a lot of trouble.
First, there's the simple privacy issue. If a professor told me to install a keylogger on my computer or if she told me to use etherpad to write an essay - either way I'd feel like my privacy was being violated.
Then there's the practical issue of how this system would affect writing ability. I personally do my best writing when I'm able to ignore the critic in my head and focus on the words. Being self-conscious about someone seeing all the stupid things I wrote along the way would make it much more difficult to get into this state of concentration.
Finally, there's the issue of making teacher's not only read two dozen boring essays but also watch replays of them being written. I personally think their time could be better spent.
One thing that I'd find problematic with that is the inability to unplug to write. There are a lot of tasks (and as I recall you said the same in one of your essays) that I find I concentrate better with the internet unjacked.
'Letting' students use the internet while they're writing essays? I think I'm missing something. I've never written an essay anywhere but on a computer with regular stuff around, including a web browser, etc.
script -t 2> script.demo.time script.demo
xemacs -nw
# M-x tetris
# ; play tetris now
# C-x C-c ; exit emacs
exit
# watch tetris being played:
scriptreplay script.demo.time script.demo
script can create a "screencast" of any terminal session. This works pretty well, but probably is not the right solution for the education market pg is talking about :-)
Indeed, I've written all of my posts on a combination of Darkroom and Etherpad. Etherpad lets me involve a couple editors who can make copy edits and suggestions while I'm still the draft.
It's also really interesting to watch a good writer go about editing. This is certainly a way to teach writing, besides the obvious method of practice.
That's a great quote. Seems like the original version is by John Godfrey Saxe though, and reads "Laws, like sausages, cease to inspire respect in proportion as we know how they are made."
Even without the playback, if legislative documents were versioned and the revisions were public, we could at least watch the diffs and call people on the pork and poison pills they insert into bills.
Maybe, but only if you had a larger sample than one person. You'll find that there are at least several common ways to write - most quite unlike each other. That would probably be the biggest lesson.. that there is little you can learn from watching this process except "just do it your way".
I found it quite surprising to watch. I thought it was a genetic algorithm writing an article somehow at first with all the constant deleting and rewriting.
Once I realized what was really going on I was struck by what an odd writing style it was (to my experience). I'm firmly in the "get it all out then come back and tighten it up" clan. There are, I suspect, at least several groups though.. I'm intrigued what they all are (perhaps the "just write and don't ever edit" group, the "refine as you go" group, the "headings then fill the gaps" group...)
I'd been kicking around the idea of doing something like this in Javascript for a while. My working title was 'TimeWriter', but I never got beyond some notes on delta formats and UI, and discussions with friends.
There's lots more potential in the graphical dimension: sliders, text color/weight that varies with age/volatility; timelines of activity (gross keystrokes or net adds/deletes) that go by either keystroke-time or clock-time.
Also an interesting thing to think about: how to build an efficient text index that can find any word or phrase that was ever present, even momentarily (or as the prefix of another word mid-typing) -- and plot its positions and lifespans.
Of course with smallish works just replaying every tick and grepping every frame could work. However, my initial inspiration for the TimeWriter idea was hosting an indefinitely lengthy personal journal. Each day, you could edit with abandon, knowing that instant search could find any fragmentary thought (and surrounding context) you'd ever jotted. The freedom of a clean sheet of paper, whenever necessary, with the confidence of perpetual perfect recall, when desired.
Thanks for that. I built this same thing into Twiddla (to replay meetings), but never released it because I didn't like the interface.
I had always assumed that nobody would get it unless I built a silly little mp3/cd player timeline with sliders and buttons. This changed my mind. Simple works just fine. Gimme a couple days and I'll polish it up and throw it live.
Great site. Interesting biz related side topic: Have sites like this trended too far towards the policy of "no account required"?
This is a service I actually want to sign up for and I wasn't able to find a sign up link after looking around for 30 seconds or so. Surely there would be some benefit to EtherPad to have me sign up for an actual account and collect data on me.. which in this case I am happy to have them do since they are providing a solid service..
This might be a silly question, but could anyone recommend a solid resource on case studies of fairly recent startups?
...wait a second, a stupid detector is going off in my head, let me check something real quick... (googles "founders at work")... well that's the one -- curiously I hadn't read it's synopsis before...
I would still be interested in further reading suggestions though if other books come to mind.?
Being able to watch how I write. It's like seeing a photograph of oneself for the first time. I had no idea how often I restarted sentences, for example.
There's other stuff I want to see that should be even more interesting. When it's replaying, I want to be able to see characters that make it into the final version in one color, and others in another. Then I'll be able to tell when I'm writing well and when I'm writing badly.
I agree, I'm an obsessive editor and usually make multiple revisions to comments I leave on various blogs, HN, forums, etc. I should probably get in the habit of writing in a doc first, but I don't really feel like putting in the extra step for stuff that's usually rather trivial. If I was an active blogger, I would likely try to use this tool on a regular basis and link to it at the end of every post.
Thanks for sharing, very awesome to watch. My only request would be the ability to adjust the playback speed.
I tend to frequently restart sentences as well. I have found that pushing myself to quickly get an extremely rough draft down makes me more efficient. This rough draft helps me focus my thoughts in the beginning on what I want to say. I can come back later and change how I say it.
If I don't do this--and I frequently don't--I find my output grinds to a halt with my brain both creating and editing sentences at the same time.
I do something similar; get a first draft out quickly so I'm not inhibited by grammar, sentence structure, logical flow etc.. then perform a second parse to tighten everything up. Interestingly, I sometimes find I almost completely reverse the passage I've written! It's as if my brain performs some sort of initial stackdump to convey my thoughts.
There's other stuff I want to see that should be even more
interesting. When it's replaying, I want to be able to see
characters that make it into the final version in one
color, and others in another. Then I'll be able to tell
when I'm writing well and when I'm writing badly.
How about coloring text based on their age in the work? That is, characters added in the last edit are bright blue (or green), and as they age without changing, they fade to black. That would be extremely cool to watch.
Was it Ian Fleming that said you should never start writing a sentence unless you know how it's going to end? It seems we're now in an age where sentences are started without knowing how they're going to start.
I really like this evolutionary-way of writing. Small little improvements and changes until it sounds better and fits better to the rest of the paragraph/text/code.
I found out that if i am disciplined enough when writing code this way the result is much much better.
When i was working in corporate world there was always some distraction (boss, time constraints , ...) to break this flow resulting in not so good code.
i thought about doing something like this in javascript for the comment or feedback boxes on websites, so you could see what people really think
This web site sucks and you are a moron\b\b\b\b\\b\b\b\b\b\\b\b...\b\b\b\b\b\\bb\bGood day sir, I can't seem to find how to cancel my account. Thank you and I hope you are having a pleasant evening.
This is interesting to watch. I notice you type something, you delete it, and then type something different quite a bit. When you write on paper, how do you deal with this constant deletion?
I think more before writing when it's on paper. The medium has more friction. But I haven't written anything for publication on paper since 1991, when I finished writing On Lisp on a stack of legal pads.
That's interesting. I'm the opposite way: I find I think less before writing and edit a lot more when I write on paper. Words on a screen look as flawlessly rendered as words in print, and to me that makes them seem more permanent; I feel like I have to really think hard before I can tell these perfect letters what to do. Stuff written in my own hand feels more tentative and more malleable.
I mean, I guess people wrote books before there were computers, so I shouldn't be that surprised, but I just realized I can't imagine writing anything longer than a couple pages on paper.
I remember arguing with English teachers in middle school that I thought better while typing. They insisted that I hand write my rough draft anyway, and I hated it.
J. K. Rowling did most of her draft work on lined paper:
"There is only one thing that annoys me about living in Edinburgh - well, two, but I'm pretty much resigned to the weather now. Why is it so difficult to buy paper in the middle of town? What is a writer who likes to write longhand supposed to do when she hits her stride and then realises, to her horror, that she has covered every bit of blank paper in her bag? Forty-five minutes it took me, this morning, to find somewhere that would sell me some normal, lined paper. And there's a university here! What do the students use? Don't tell me laptops, it makes me feel like something out of the eighteenth century."
-J. K. Rowling
I was completely dumbfounded when I first saw that stack of paper because I can barely manage to write a one line note to my wife without scratching something out. To undertake something as large as the Baroque cycle trilogy is really beyond my comprehension.
I don't know if he wrote Cryptonomicon this way also but my first edition is full of typos. Something I imagine could happen if you turn over several hundred hand written pages with edits to a typist.
Writing on paper has one advantage: because you can't revise things, you're forced to finish the first draft before you start on the second.
So, what you produce may not be as good as it will get later, but at least it's complete.
The major disadvantage of writing on paper is of course when you have no idea what you're going to write ahead of time, because then your first draft is going to be very, very low quality.
You're probably thinking of an emacs mode that does this on its own, but it would also be excellent to have an emacs minor mode that integrates with Etherpad. Ideally, you just do M-x etherpad-buffer and emacs tells you a URL that you can then send around and let other people watch / co-edit whatever you're editing in emacs.
To make this work, Etherpad would need to provide an API, though not necessarily a well-documented one. A supported API might make sense for Etherpad, though. Imagine being able to load a Wordpress plugin that enables playback of your blog posts. Or something that lets you co-author blog posts on etherpad. Oh, and greasemonkey scripts that let you co-edit any elements in a web form. OK... let's not get ahead of ourselves...
Not so sure; if I'm not mistaken Lynx javascript support is rudimentary (if not non-existent). Doing Comet on that would be somewhat challenging, I guess.
A few years ago I stumbled on a wonderful art project where people used a similar technique to record world class poets as they wrote for 15 minutes. The site is still live at http://quickmuse.com/ if you want to see a poet laureate or two write poems. Robert Pinsky's poems are particularly fascinating to watch because he never makes a single edit or mistake.
Many people. There's even a moderately popular firefox plugin to automate the process (the name escapes me) - it invokes your editor for you, then when you save and exit it copies the temp file into the textarea.
As others have mentioned - a great experience to watch PG writing essays. However, I was put off by the speed. Is this because PG was typing so fast or is it the player? Is it possible to control the speed (other than the Prev/Next link at the top)?
I'm self-critical enough as it is just realizing that someone else is going to read the finished product, let alone everything I wrote before exposing it. I would have a hard time letting my thoughts flow.
In addition, for school papers I would type in sarcastic or snarky sentences just for fun about what I really think of the subject matter or whatever book I was forced to write an essay for at the time. Sometimes I'd go so far that I would save a different copy of the paper just so that I could go back and laugh at it years later.
It looks like a great way to gain insight into how you or someone else writes, but I think it should always be opt-in.
Interesting - this is exactly how journal recovery used to work on VMS. I remember watching my mother edit documents in the MASS-11 (http://www.stanq.com/blog/?p=11) word processor on a VT-100 terminal. If her session fell off she'd ring an operator and he'd play her session back on the screen; it looked exactly like this!
It's so hard to guess. I remember it taking about 3 days, but not working all day. I know there were 5400 half-second intervals when I typed at least one keystroke (that's what the numbers at the top mean).
It would be really interesting to see timestamps, particularly in longer works.
Two reasons: 1) You could see when the Eureka! moments occur. 2) To easily get a real measure of how long it really takes to write.
I have a real problem with 2), both with underestimating how long it will take to do things I think will be easy to write, and overestimating things I think will be hard. Of course, there's a lot more to writing than typing, but it could provide important information that is tedious to track at a high level and impossible to track at the level of granularity that this app could.
5400 half-seconds is 45 minutes. So, assuming you take 2 min to think for every 1 min of typing, that's still working on the essay just 45 minutes everyday (for 3 days).
I think (from comment above: "not working all day") you took much longer than the figure above? Perhaps a lot more thought goes into an essay than I imagine?
I'd like to see this applied as an embeddable feature in hacker news. see who writes comments, which comments get commented on and what people really have to say before they hit that 'add comment' button
This is very cool stuff. I wish it had existed before. One small wish/request: can we have a slider that we can use to control the speed? I'm guessing that would be very helpful.
This is really amazing. It's interesting to watch you experiment with many PG-esque phrasings before settling on the final one, especially in the introduction.
There's huge potential here, especially if there's some sort of even imperfect high-level 'glomming'. How often have you written a clever turn of phrase, an insightful paragraph, or a poignant witticism, gone back and removed it to fix flow, and then re-wrote it, not getting the phrasing quite right?
The ability to fix those types of changes, or to revert back to a paragraph "somewhere in the middle" would be huge.
For now they just hacked together something custom for me. However, Etherpad saves every keystroke, so once they enable this for everyone, you'll be able to replay stuff you wrote with it.
Incredible. I was talking about my teacher about how a transition away from paper to tech is great and all but we have lost the sense of keeping drafts. Drafts have a very high value. I don't know how you are connected with the folks @ Etherpad but if there is anyway I can help, let me know.
Playback is just one little feature of Etherpad, but think of the implications of this alone. Among other things it will make cheating impossible in classes where students write papers, because now you can finally "show your work" in writing the way you do in math.