Because your documents are under subversion or a similar version management system, then you can “diff” the documents to see what your collaborators changed. You can setup emails to get notifications when something changed something.
(1) What you describe is not collaborative writing. Merely appending comments on someone’s document is review, not writing.
(2) Why on Earth would you want to have lines much longer than 80 characters? Your troubles with diff are the least of your worries. Many text editors, such as vim, work poorly with very long lines.
I think we are getting sidetracked. My statements were “every single graduate student should learn to write without a word processor” and “serious science students should learn LateX”.
I did not write “everyone should learn LaTeX”, and I certainly did not write “everyone should use LaTeX”.
All I am saying is that you should consider other writing tools than Microsoft Word. If, after trying out alternatives, Microsoft Word is best for you, then fine.
I have written several research papers and reports without LaTeX.
I prefer lyx but people I know use texmaker and kile too. To each his own 🙂
People I admire know one editor completely in and out – vim or emacs. IMO, learning a editor such as vim/emacs is the first thing any serious developer/gradstudent/scientist should do.
@Pavan I keep hearing about git. Isn’t it just another version control tool?
Ragib Hasansays:
I use Textpad for almost all my editing. I’ve integrated latex with it, and can compile tex files from it directly.
It’s not as fancy as TeXMaker, but certainly way faster.
Neilsays:
How do you do collaborative writing, e.g. comments on papers in progress? I’ve always found Word to excel in this area. For LaTex it seems to boil down to PDF markup.
Neilsays:
That’s true, but I maintain the MS Word collaboration features are far superior. Especially with people – *cough* advisors *cough* – who don’t edit Latex.
Finally, diff with Latex has never been useful for me, since diff is line-based, and I have soft-wrapping – therefore each paragraph is one line.
This leads to much manual searching to find the change. Have you found something better than traditional diff?
Edsays:
Don’t use soft wrapping and do put every sentence on its own line. Then diff works great.
Neilsays:
I don’t get it. All your text documents contain carriage returns or line feeds after X characters? Seems like the wrong solution to the problem.
The right solution is a better diff. Diff was designed for code, where lines >80 chars made it impossible to read on the terminals of the day (and still a bad idea for readability). But we don’t have these restrictions nowadays.
As for the extent to which advisors and grad students ‘collaborate’, I’ll leave that to Daniel. I still maintain that Word’s least appreciated feature (that I haven’t seen anywhere else) is “Track Changes” and comments. Sure, it can be duplicated with merge/diff, but that answer doesn’t work for a lot of people.
I agree with all the comments about GIT. git is very well designed you may like to watch this video by our cocky friend Linus Torvalds http://www.youtube.com/watch?v=4XpnKHJAok8
@Steven I also use extensively wikis to collect ideas when approaching new topics. I should have included it in my blog post.
Caoimhin O'Broicsays:
Well, I’ve followed your blog for quite a while and this is really the first item I take issue with …
I think you’ve jumped into the “gave the GUI a go, didn’t do what I wanted, I’ll just code it” trap that pervades in the world of mathematical / computational science.
While it’s true that a quick-and-dirty attempt to generate complex documents in Word is most often frustrating and infuriating, the fault lies in the lack of initial document design rather than with the application. I recently completed my PhD in computational biology and, in line with your current topic, I spent considerable time exploring the writing options available to me – yes, I learned LaTeX !! What I found though is that if the document is correctly planned and implemented from the start, then the simplicity with which downstream complexity can be added in a Word document is impressive. This is especially true of the most recent Word incarnation. Collaboration is also remarkably simple if correctly implemented with the track changes and revision functionality. Just for the record, I’m far from a Microsoft proponent, in fact, linux is my OS of choice and I also use openoffice every day for “quick and dirty” documents.
Ultimately, I’d stick with your ‘check out all the options’ advice, but would tag on ‘learn the fine details of MS Word ‘ as well. For my money, complex documents implemented correctly in Word are traceable, managable, simple, and time efficient and beat LaTeX hands down …
git is much like svn with one crucial difference: it is a distributed VCS. That is, commits are made locally and changes can be pushed to one or more servers. It may seem like a small difference but it has made a big difference to the way I manage my own code and collaborate with others.
Definitely worth trying. If you don’t like git’s style, mercurial (hg) is another distributed VCS I have heard good things about.
I support wholeheartedly the push for Latex in research. However, as the previous commenter points out, collaborative writing with Latex is not obvious. I would recommend mercurial over subversion for this kind of use, even for personal version control (no need for a server).
Otherwise, I am an Emacs guy.
JSsays:
Regarding something better than the traditional line-based diff. Here is the wrapper that I am using (together with svn): http://www.iki.fi/suo/color-wdiff/
@Neil, I used to believe as you – that a better diff was called for – then found that end result is not necessarily better. The simple act of placing an end-of-line at the end of every sentence is oddly semantic. When you review changes, you are interested in added, deleted, or modified sentences. Adjacent modified fragments in two sentences are more likely distinct changes, not one change. The added line break makes that distinction impossible to miss.
With content laid out one-sentence-per-line it seems a bit easier to scan through changes, as the line breaks add just a bit of semantics to the layout. Also, overlong sentences REALLY stand out – kind of a side benefit. 🙂
Stevensays:
I write a lot of my personal notes (not shown to others) in a personal wiki. When a good idea for a paper forms, I do a first draft of the abstract and/or introduction with pen and paper. If it looks good, I type it up in Latex, rewrite a little, and share with collaborators. At that point, we start writing the paper.
Although I don’t write my papers in my wiki, it is one of my most important writing tools for developing ideas.
I’ll add my recommendation to texmaker, although I do still enjoy an occasional Emacs loaded up with every TeX mode and extension in existence. I tried Lyx for a while and hated it, but each to his own. Also, I’d really love an editor that was like Scrivener, but with a lot of extra LaTeX functionality built in.
I second the recommendation to use a DVCS rather than svn. It is so easy to set up a local repository for your own stuff even for quite trivial things without the need for a central server. Then you can throw stuff onto the server when it gets serious or when it is time for collaboration. Also, being able to commit when you don’t have a network connection is very handy. The only advantage that svn has for LaTeXing is that someone wrote a package that prints the svn revision number and other details on the .dvi or .pdf file, which can be very useful for keeping track of drafts. I don’t know of anything similar for any of the DVCS systems.
Personally I prefer Mercurial (hg) as a DVCS for LaTeXing because I think it is much easier to parse than git for people who don’t do a lot of coding. As a physicist, I have to collaborate with a lot of very unhackerish types and it is hard enough to persuade them to use any VCS at all, so ease of use is definitely at the top of my list of requirements. Also, I love how easy it is to throw a repository up on the web via https or http with the included hgweb scripts and because it is Python I find it very easy to write extensions/customizations for hg. I think git is ultimately a lot more flexible and probably faster, at least in its current implementation and undoubtedly contains features that are essential to complex projects, e.g. the Linux kernel. It is definitely the tool of choice for serious hackers but if you are just going to use it for a few LaTeX documents and a few small coding projects then life is a lot easier with hg.
You have mentioned using Eclipse for programming, so you might want to use it for scientific writing with the TeXlipse plugin: http://texlipse.sourceforge.net/
I have this setup with SVN for years and no problems so far.
I have used maxima in the preparation of a couple of papers, including one upcoming journal article. It is not as fancy as Maple or Mathematica, but it gets the job done very nicely. And there is a great community around it.
But honestly, it is very rare that my research will lead me to the kind of algebra that requires a computer algebra system. Usually, for me, it is a sign that something went badly.
Itmansays:
Never, never, never use Word for a scientific publication. I had an experience with a survey. You simply cannot get the work done.
1) References are awful.
2) Tables and charts are awful.
3) Most important for surveys and large articles: bibliography is almost impossible!!!
Itmansays:
One more question: do you use any computer algebra system?
Yes, git is yet another vcs but efficient and productive. I know quite a few people who always keep their home directories in git 🙂 (did i mention that they keep some jar files also in git)
If one is interested in a graphic editor I’d say winedt (no i) for MS OS and Kile for linux.
Megansays:
LaTeX, Google Docs, OpenOffice, Vim, and Eclipse… definitely a few of my favourite things. I would put a good word in for Mercurial too.
I have recently heard of writing wikis to assemble notes and I love the idea. I saw some comments here about it and would be interested to hear about the wiki workflow.
Neilsays:
I use TiddlyWiki to store my personal thoughts and notes. Not a true wiki in the sense of collaboration and versioning, though, more of a notes application.
I spent years writing Latex with simple editors. It was ok.
One day I tried LyX and never looked back.
I wrote all my papers and my entire PhD thesis with LyX.
http://www.lyx.org
LyX is unique on his WYSIWYM approach.
http://en.wikipedia.org/wiki/WYSIWYM
Comments in LaTeX are trivial. For one thing, LaTeX has already comments in its language (begins a line with “%”).
If you prefer something fancy, you can setup commands like this one:
newcommand{neil}[1]{textbf{{small{color{magenta}DL}: #1{color{magenta}$circ$}}}}
(This uses the package color.)
Because your documents are under subversion or a similar version management system, then you can “diff” the documents to see what your collaborators changed. You can setup emails to get notifications when something changed something.
(1) What you describe is not collaborative writing. Merely appending comments on someone’s document is review, not writing.
(2) Why on Earth would you want to have lines much longer than 80 characters? Your troubles with diff are the least of your worries. Many text editors, such as vim, work poorly with very long lines.
@Neil
I think we are getting sidetracked. My statements were “every single graduate student should learn to write without a word processor” and “serious science students should learn LateX”.
I did not write “everyone should learn LaTeX”, and I certainly did not write “everyone should use LaTeX”.
All I am saying is that you should consider other writing tools than Microsoft Word. If, after trying out alternatives, Microsoft Word is best for you, then fine.
I have written several research papers and reports without LaTeX.
I prefer lyx but people I know use texmaker and kile too. To each his own 🙂
People I admire know one editor completely in and out – vim or emacs. IMO, learning a editor such as vim/emacs is the first thing any serious developer/gradstudent/scientist should do.
I prefer vim with git.
@Pavan I keep hearing about git. Isn’t it just another version control tool?
I use Textpad for almost all my editing. I’ve integrated latex with it, and can compile tex files from it directly.
It’s not as fancy as TeXMaker, but certainly way faster.
How do you do collaborative writing, e.g. comments on papers in progress? I’ve always found Word to excel in this area. For LaTex it seems to boil down to PDF markup.
That’s true, but I maintain the MS Word collaboration features are far superior. Especially with people – *cough* advisors *cough* – who don’t edit Latex.
Finally, diff with Latex has never been useful for me, since diff is line-based, and I have soft-wrapping – therefore each paragraph is one line.
This leads to much manual searching to find the change. Have you found something better than traditional diff?
Don’t use soft wrapping and do put every sentence on its own line. Then diff works great.
I don’t get it. All your text documents contain carriage returns or line feeds after X characters? Seems like the wrong solution to the problem.
The right solution is a better diff. Diff was designed for code, where lines >80 chars made it impossible to read on the terminals of the day (and still a bad idea for readability). But we don’t have these restrictions nowadays.
As for the extent to which advisors and grad students ‘collaborate’, I’ll leave that to Daniel. I still maintain that Word’s least appreciated feature (that I haven’t seen anywhere else) is “Track Changes” and comments. Sure, it can be duplicated with merge/diff, but that answer doesn’t work for a lot of people.
I agree with all the comments about GIT. git is very well designed you may like to watch this video by our cocky friend Linus Torvalds http://www.youtube.com/watch?v=4XpnKHJAok8
@Steven I also use extensively wikis to collect ideas when approaching new topics. I should have included it in my blog post.
Well, I’ve followed your blog for quite a while and this is really the first item I take issue with …
I think you’ve jumped into the “gave the GUI a go, didn’t do what I wanted, I’ll just code it” trap that pervades in the world of mathematical / computational science.
While it’s true that a quick-and-dirty attempt to generate complex documents in Word is most often frustrating and infuriating, the fault lies in the lack of initial document design rather than with the application. I recently completed my PhD in computational biology and, in line with your current topic, I spent considerable time exploring the writing options available to me – yes, I learned LaTeX !! What I found though is that if the document is correctly planned and implemented from the start, then the simplicity with which downstream complexity can be added in a Word document is impressive. This is especially true of the most recent Word incarnation. Collaboration is also remarkably simple if correctly implemented with the track changes and revision functionality. Just for the record, I’m far from a Microsoft proponent, in fact, linux is my OS of choice and I also use openoffice every day for “quick and dirty” documents.
Ultimately, I’d stick with your ‘check out all the options’ advice, but would tag on ‘learn the fine details of MS Word ‘ as well. For my money, complex documents implemented correctly in Word are traceable, managable, simple, and time efficient and beat LaTeX hands down …
git is much like svn with one crucial difference: it is a distributed VCS. That is, commits are made locally and changes can be pushed to one or more servers. It may seem like a small difference but it has made a big difference to the way I manage my own code and collaborate with others.
Definitely worth trying. If you don’t like git’s style, mercurial (hg) is another distributed VCS I have heard good things about.
I support wholeheartedly the push for Latex in research. However, as the previous commenter points out, collaborative writing with Latex is not obvious. I would recommend mercurial over subversion for this kind of use, even for personal version control (no need for a server).
Otherwise, I am an Emacs guy.
Regarding something better than the traditional line-based diff. Here is the wrapper that I am using (together with svn): http://www.iki.fi/suo/color-wdiff/
@Neil, I used to believe as you – that a better diff was called for – then found that end result is not necessarily better. The simple act of placing an end-of-line at the end of every sentence is oddly semantic. When you review changes, you are interested in added, deleted, or modified sentences. Adjacent modified fragments in two sentences are more likely distinct changes, not one change. The added line break makes that distinction impossible to miss.
With content laid out one-sentence-per-line it seems a bit easier to scan through changes, as the line breaks add just a bit of semantics to the layout. Also, overlong sentences REALLY stand out – kind of a side benefit. 🙂
I write a lot of my personal notes (not shown to others) in a personal wiki. When a good idea for a paper forms, I do a first draft of the abstract and/or introduction with pen and paper. If it looks good, I type it up in Latex, rewrite a little, and share with collaborators. At that point, we start writing the paper.
Although I don’t write my papers in my wiki, it is one of my most important writing tools for developing ideas.
I’ll add my recommendation to texmaker, although I do still enjoy an occasional Emacs loaded up with every TeX mode and extension in existence. I tried Lyx for a while and hated it, but each to his own. Also, I’d really love an editor that was like Scrivener, but with a lot of extra LaTeX functionality built in.
I second the recommendation to use a DVCS rather than svn. It is so easy to set up a local repository for your own stuff even for quite trivial things without the need for a central server. Then you can throw stuff onto the server when it gets serious or when it is time for collaboration. Also, being able to commit when you don’t have a network connection is very handy. The only advantage that svn has for LaTeXing is that someone wrote a package that prints the svn revision number and other details on the .dvi or .pdf file, which can be very useful for keeping track of drafts. I don’t know of anything similar for any of the DVCS systems.
Personally I prefer Mercurial (hg) as a DVCS for LaTeXing because I think it is much easier to parse than git for people who don’t do a lot of coding. As a physicist, I have to collaborate with a lot of very unhackerish types and it is hard enough to persuade them to use any VCS at all, so ease of use is definitely at the top of my list of requirements. Also, I love how easy it is to throw a repository up on the web via https or http with the included hgweb scripts and because it is Python I find it very easy to write extensions/customizations for hg. I think git is ultimately a lot more flexible and probably faster, at least in its current implementation and undoubtedly contains features that are essential to complex projects, e.g. the Linux kernel. It is definitely the tool of choice for serious hackers but if you are just going to use it for a few LaTeX documents and a few small coding projects then life is a lot easier with hg.
You have mentioned using Eclipse for programming, so you might want to use it for scientific writing with the TeXlipse plugin: http://texlipse.sourceforge.net/
I have this setup with SVN for years and no problems so far.
Check LatexDiff if you want to automaticically produce pdf document with word-by-word diff. This works great for me
http://www.pppl.gov/~hammett/comp/tex/latexdiff.html
@Itman do you use any computer algebra system?
I have used maxima in the preparation of a couple of papers, including one upcoming journal article. It is not as fancy as Maple or Mathematica, but it gets the job done very nicely. And there is a great community around it.
But honestly, it is very rare that my research will lead me to the kind of algebra that requires a computer algebra system. Usually, for me, it is a sign that something went badly.
Never, never, never use Word for a scientific publication. I had an experience with a survey. You simply cannot get the work done.
1) References are awful.
2) Tables and charts are awful.
3) Most important for surveys and large articles: bibliography is almost impossible!!!
One more question: do you use any computer algebra system?
Daniel,
Thank you for the clarification.
Prof @Lemire: I missed your question.
Yes, git is yet another vcs but efficient and productive. I know quite a few people who always keep their home directories in git 🙂 (did i mention that they keep some jar files also in git)
I leave it to the following URL to purport my stand:
http://whygitisbetterthanx.com/
If one is interested in a graphic editor I’d say winedt (no i) for MS OS and Kile for linux.
LaTeX, Google Docs, OpenOffice, Vim, and Eclipse… definitely a few of my favourite things. I would put a good word in for Mercurial too.
I have recently heard of writing wikis to assemble notes and I love the idea. I saw some comments here about it and would be interested to hear about the wiki workflow.
I use TiddlyWiki to store my personal thoughts and notes. Not a true wiki in the sense of collaboration and versioning, though, more of a notes application.