Open Notebook

Despite having had the experience of having some of my work copied from publications of initial findings, I remain committed to the idea that science and scholarship should be affairs conducted, insofar as it is practicable and ethical to do so, in the open, available to others for inspection and consideration. To some extent, this website cum blog, which I originally called a logbook, was/is an effort to do some of that. But blogs are now best understood as more oriented toward chronological arrangements, which runs counter to what is often the slow accumulation of notes and ideas that are fundamental to science and scholarship.

While software like WordPress includes the options for more static pages, it is not necessarily easy to set up nor is it necessarily easily integrated into other workflows — that is, being able to use a an always available application that can capture notes and scribblings wherever you happen to be reading or writing. (I suspect Automattic’s purchase of SimpleNote is one step to solving this problem, and, like many others, I will be watching to see how things develop. For now, publishing from within SimpleNote simply creates a URL which one can send to others, like Dropbox or Google Drive.)

I have been following Caleb McDaniel’s experiments in open notebook scholarship for a while now, waiting for a moment where I might be in a position to follow suit. McDaniel’s system is based on Gitit, a Pandoc-based wiki that, obviously, uses Git for publishing results to a web server — it’s not quite clear to me, at this stage of my reading, how much of the Gitit infrastructure is copied to the web.

This past semester, I tried out a Python-based system, MkDocs, which I chose principally for the simplicity of its folder structure and the fact that it seemed least “fussy.” (Obviously such an evaluation comes with a rather large block of salt.) The advantage of Gitit, as I understand it, is that it uses Pandoc to create HTML, but Pandoc can be used within any setup — so long as you have it installed — as can Git. McDaniel’s system is, like so many others, including my own, founded on markdown.

The really exciting part of his note system, I think, is that instead of building one giant Bibtex file, each bibtex entry is at the top of the document that also contains his reading notes and quotations. I have been using Papers for reference management, and while my experience of it has been mixed: the automatic determination of the likely citation information is quite good; my experience of its ability to keep track of associated files — sometimes hard-won PDFs — has been lackluster. The experience of taking notes on electronic documents is okay, and, obviously the auto-generation of quotations from highlighted material is exceptional — but also dependent on the quality of the PDF. (Why, oh why, do so many humanities journals and materials generate such poor PDFs? Is this a workflow issue? Would a LaTeX-based workflow generate better PDFs? What’s going on here?)

It’s this generation of Bibtex on-the-fly, which I find truly exciting, and McDaniel has released the Pandoc filter which does that work. As I move forward with my own system, probably based on MkDocs, I’ll be looking to see if I can either re-use his filter or re-create it within Python. I am not alone in exploring this territory, Johannes Grassler has released [mkdocs-pandoc][], a Python module “contain[ing] a set of filters for converting mkdocs style markdown documentation into a single pandoc-flavored markdown document.”

Why do this? Open science/scholarship is hard, and, as I have experienced, possibly opens you up to some unpleasant experiences. First, there is the principle of openness itself, which is something which one aspires to, knowing that it’s probably never fully achievable because, first, an individual researcher from a public university simply doesn’t have the resources to make it happen, and, second, because human. Second, there is my own reticence to commit everything into a file system that remains outside my control, and, especially, open to corruption — and here I simply mean the dangers of having a system break ungracefully. In the early aughts I had a Windows 2000 machine crash — when the power supply friend, it took out part of the hard disk. The part that remained could not be logged into, leaving several years of work, which, yes, should have been backed up, unaccessible. Since then, I am more active in terms of having multiple copies and having those copies in a format that degrades with some grace. Plain text can’t be beat in this regard.

For the record, I wrote The Amazing Crawfish Boat in Scrivener, and I will probably do the same with the next book, especially now that Scrivener has an iOS companion. But Scrivener is a place where I focus on writing, not necessarily on compilation of notes and ideas — indeed, when I have used it as such, the writing suffers. (To be clear, the app itself can handle pretty much anything you throw at it. That dump everything and sort it out in the writing approach simply doesn’t work for me.)

This business of compiling notes has remained one of those things I wish I could sort out. At present, I have a few Devonthink databases as well as a number of Evernote notebooks, and that’s not counting the collection of plain text notes I have sitting in at least one directory in my Dropbox account — and some sitting in a Ulysses notebook on my iCloud account. I don’t think all these notes necessarily need to come together in one place, but reducing the places isn’t a bad idea, and, just as importantly, finding a place for scientific and scholarly notes that facilitate productivity is one piece of the puzzle that I think can be solved sooner rather than later.

Winter Break 2016-2017 just got a little more interesting.

2 thoughts on “Open Notebook

  1. Towards an Open Notebook Built on Python – John Laudun

Leave a Reply