Now that I’ve introduced the format of my todo.txt file, let’s look at what we can do with it. I have a few retrieval commands that I use for pulling information out of my todo.txt file.

Reviewing NA Format

First, it’s been so long since my last post on this topic, how about a little refresher? Here’s the format of a next action line:

[priority] [project] NA [recurrence] [pigpog] context enterdate [startdate] [duedate]

where

1. [priority] is an optional priority for the NA. I use (!) as my standard priority (brackets are required) and (@) as the priority I associate with GTD-specific next actions, e.g., weekly review. Almost any single character is useful, although I don’t make use of #, ~, or x because these characters mean other things to me.
2. [project] is an optional project name for the NA. I keep track to two kinds of projects, trackable and non-trackable. (This comes into effect when I use the birdseye.py script discussed by Gina Trapani here.) Trackable projects have names like [p:Blog], [p:FixCar]. The square brackets mean “project” and the p: means “trackable”. Non-trackable projects just have the square projects, e.g., [TV], [Tweak].
3. NA is the next action. Every line must have a next action, otherwise, what’s the point. Almost anything goes here, although I avoid using double quotes because they cause havoc with one of my scripts, but single quotes are fine.
4. [recurrence] is an optional reminder for me that a particular NA has a recurrence; I use the format (recur: every…) for this. There’s no specific format, i.e., I don’t have any scripts looking for “recur:”, but it’s a nice format for me.
5. [pigpog] is an optional reminder for me that a particular NA is part of a larger sequence of actions, in the fashion of pigpog. Again, there’s no specific format, but I use (pigpog: …).
6. context is the context of the next action. All contexts must start with @, e.g., @anywhere, @online, etc. Every line must have at least one context; I tend to use only one, but you could put more if you want.
7. enterdate is the date that this NA was created, in the format e:2006-08-23. I use the entry date in order to keep track of how long an NA has been around - it’s good for determining exactly how many days I’ve been waiting for something, or how stale some NAs are getting. FYI, my stalest NA is 99 days old. GASP. The ironic part? It’s an NA telling me to read the first chapter of a Perl Weekend Crash Course Book!
8. startdate is optional; it the earliest possible date that I could start this NA, if that date is later than today. The start date is in the format s:2006-08-25. I use the start date to hide NAs that can’t be started yet - very useful because my todo.txt file also stands in as part of my Tickler system.
9. duedate is also optional, it is the due date of the particular NA. Yeah, I know, deadlines are frowned upon, but I prefer to have deadlined tasks so that I can see most, if not all, of my NAs in one particular place, and that includes ones with deadlines. Examples of NAs with deadlines are some of my recurring tasks, like downloading transactions to balance my checkbook, washing the car every month, etc.

Retrieval Functions

These are my most commonly used retrieval functions:

• get: get all the NAs that are current for right now. What makes a current next action for me?
  • It’s startable, i.e., either there is no start date, or the start date is today or earlier.
  • It’s either got no due date, or a due date within the next week.
  • These two criteria are built in so that I can use my todo.txt file as a tickler file as well, e.g., a reminder to backup my computer once a month. The system will still work for you even if you don’t want to use start or due dates.
  • It’s not a @someday/maybe item. I don’t want to see my @someday/maybes all the time; I’ll look for them specifically.
• getall: this is similar to the get command, but it returns everything, including those NAs that might be on the @waiting list or @someday/maybe list.
• getpri: yeah, yeah, I know, priority is evil. But I use them, for extra special things, or to focus myself. You don’t have to use them. But if you do, you can use this command to find those items that are prioritized.
• getdated: this function doesn’t take an argument, but rather, returns all NAs that have a due date, and sorts them according to the due date. There are several categories: overdue, due today, due in the next week, due in the next month, or due after that.

Now, each of these command can have up to three arguments, i.e., three search terms, connected by “AND”. In other words, if I look for “pear”, I’ll find NAs with “pear” in them. If I look for “pear partridge”, I’ll find things with “pear” and “partridge”. I’ll explain this more in the running example below.

Running Example - Retrieving NAs from the todo.txt File

For the running example, I’m using a fictitious todo.txt file, that hopefully gives you an idea of how flexible this system can really be. I’m going to just pasted in the results of some command line playtime with it, interspersed by my comments.

First, let’s see what NAs I have due soon for my blog. Remember that the get command returns things that are startable, and due no later than a week for now (or don’t have a due date at all.)

Get with One Argument

F:MiscellaneousGTDToDoBlog>get blog=====| blog |=====[ 7] 69 | | | [p:Blog] Create fictitious todo.txt @laptop- e:2006-08-22[ 8] 69 | | | [p:Blog] Post about hotkey @online- e:2006-08-22[ 9] 69 | | | [p:Blog] Update Text Fetish #0 with links (pigpog: Publish post) @online- e:2006-08-22 

Now, unfortunately, the spacing shown here is less than optimal. I’ve got the script set up so that the columns are spaced nicely. I’ve even got it set up so that extra long NAs wrap, so they look good in my console window. This is done by way of an argument passed into the script, saying how much wrapping I need. Here’s an image showing what it should really look like:

What do the Columns Mean?

Well, that’s an easy one. First, although I don’t have unique identifiers for individual NAs in the text file, I need to be able to refer to them with other commands. So, every time output gets displayed to me, a line number is shown, e.g., [7] in the image above. You must note though that [7] doesn’t always equal that line; it’s really only good for the current output. So, if I were to delete [7], say by using the “done” command, which I’ll introduce later, then really line [8] becomes [7] because the numbers refer to where that line is in the text file, right now. I just don’t want people thinking that these are unique identifiers, like in a database. This is more transient.

The second column, here, showing 69, is how many days it’s been since I entered this NA into the text file (based on the e:\d\d\d\d-\d\d-\d\d). So, yesterday, when I printed this example out, it was 69 days since 22 Aug 06. This is a way for me to see how stale things are. Unfortunately, these three NAs are particularly stale! Now, if I were to have a start date (s:\d\d\d\d-\d\d-\d\d) in an NA, then this number would be how many days since that NA was startable. So, if startable, then how many days since I could have started it. If no startable, then how many days since I entered it. If you don’t want to use either startable or enter dates, then just an empty column.

The third column, blank, shows how many days until the deadline. These NAs have no deadlines, so empty columns. If the NA were overdue, then the number would be negative, showing how many days overdue.

The next column, blank, would be where priorities would be shown.

The last column is the entire text of the NA. Notice how it wraps prettily :) Notice also that I put extra information in some NA, e.g., (pigpog:…) or (recur:…). This is a way of reminding myself that a particular NA is part of a chain, or a recurring NA that I want to repeat over a certain period.

Get with Two Arguments = AND

F:MiscellaneousGTDToDoBlog>get blog hotkey=====| blog AND hotkey |=====[ 8] 69 | | | [p:Blog] Post about hotkey @online- e:2006-08-22 

Remember earlier, when I said that you can have three arguments, and they’re combined with “AND”. Here, I’m searching for both “blog” and “hotkey”, returning the one NA that has both of these terms in it.

How about ORing Two Arguments?

Is it possible to combine arguments with the OR connector? Why yes. Here, you just have to cater to the fact that inside the script is just a test to see if a line matches some text. What you put in your text is your business. For example, when I want to OR two terms, I just feed the script one term, “hotkey|fetish”. When it comes to the test, Perl sees this as asking, does this line have either “hotkey” or “fetish” in it?

F:MiscellaneousGTDToDoBlog>get "hotkey|fetish"=====| hotkey|fetish |=====[ 8] 69 | | | [p:Blog] Post about hotkey @online- e:2006-08-22[ 9] 69 | | | [p:Blog] Update Text Fetish #0 with links (pigpog: Publish post) @online-               e:2006-08-22 

Most likely, it’s possible to do other fancy things, like other regular expression goodness, in your search term(s). Me, I’m happy with one term, two terms and’d, and two terms or’d.

Getting All NAs for a Context

To find all the (current) NAs for a context, you can just use the context as your search term, e.g., “get @calls”. Here, I’m showing that I can get everything from my @waiting list.

F:MiscellaneousGTDToDoBlog>get @waiting=====| @waiting |=====[ 5] 0 | | | [TV] Waiting for next Hell’s Kitchen @waiting e:2006-10-30 

Someday/Maybes

Similarly, I can look for someday/maybe NAs. However, I can’t use the get command. Remember above, when I said that the NAs returned by get were those that I could do now, and weren’t someday/maybes? So, this command will get me nothing.

F:MiscellaneousGTDToDoBlog>get @someday=====| @someday |===== 

Instead, I move to the getall command, which will return all NAs, regardless of whether or not they’re startable, whether or not they are due soon, whether or not they are someday/maybes.

F:MiscellaneousGTDToDoBlog>getall @someday=====| @someday (ALL) |===== 

=====| Waiting |===== 

=====| Someday |=====[ 2] 0 | | | Learn to play piano @someday/maybe e:2006-10-30 

Note that there’s also a Waiting section here. Sure, I can get @waiting with the get command, but I use getall normally when I’m looking a project. So, I want all of the NAs for a particular project, and I just want a quick separation between actually do-able NAs, vs. waiting-fors, vs. someday/maybes.

For example, let’s see all of my NAs for my [TV] project.

F:MiscellaneousGTDToDoBlog>getall TV=====| TV (ALL) |=====[ 4] 68 | -68 | | [TV] Get Eureka’s next episode (recur: every wed) @online- e:2006-08-22                  s:2006-08-23 d:2006-08-23 

=====| Waiting |=====[ 5] 0  |     | | [TV] Waiting for next Hell’s Kitchen @waiting e:2006-10-30 

=====| Someday |===== 

This past printout is a good example to show the due date magic. Line [4] has a due date of 23 Aug 06. I’m a little behind, 68 days to be exact.

Don’t use [ ]

It’s important not to use the square brackets [ ] in your search terms. That’s because in Perl, square brackets are used to denote that any one of the characters inside makes a good match. In other words, when I search for [TV], that means that any NA with a “T” or “V” (or lower case, too I think) is a good match. Not very useful.

F:MiscellaneousGTDToDoBlog>getall [TV]=====| [TV] (ALL) |=====[ 1] 69  |     | (!) | Win lottery!!! @anywhere e:2006-08-22[ 4] 68  | -68 |     | [TV] Get Eureka’s next episode (recur: every wed) @online- e:2006-08-22                      s:2006-08-23 d:2006-08-23[ 6] 69  | -61 |     | [p:Bills] Download transactions and balance books @online+ e:2006-08-22                      d:2006-08-30[ 7] 69  |     |     | [p:Blog] Create fictitious todo.txt @laptop- e:2006-08-22[ 8] 69  |     |     | [p:Blog] Post about hotkey @online- e:2006-08-22[ 9] 69  |     |     | [p:Blog] Update Text Fetish #0 with links (pigpog: Publish post) @online-                      e:2006-08-22[ 10] -1 |     |     | [p:Test] Test #1 Start Date @laptop- e:2006-08-22 s:2006-10-31[ 11] 69 | 14  |     | [p:Test] Test #2 Due Date @laptop- e:2006-08-22 d:2006-11-13[ 12] 69 |     |     | [p:Test] Test #3 No Dates @laptop- e:2006-08-22 

=====| Waiting |=====[ 5] 0   |     |     | [TV] Waiting for next Hell’s Kitchen @waiting e:2006-10-30 

=====| Someday |=====[ 2] 0   |     |     | Learn to play piano @someday/maybe e:2006-10-30 

Looking at Projects

So, when looking for a specific project, you could get very specific and even use the p: notation, if you used it in your NAs. Here, I’m searching for current NAs for the p:Test project. See, there is one NA that I can do right now.

F:MiscellaneousGTDToDoBlog>get p:Test=====| p:Test |=====[ 12] 69 | | | [p:Test] Test #3 No Dates @laptop- e:2006-08-22 <

But what about other NAs? Am I missing something? So, let’s look at all NAs associated with this project.

F:MiscellaneousGTDToDoBlog>getall p:Test=====| p:Test (ALL) |=====[ 10] -1 |    | | [p:Test] Test #1 Start Date @laptop- e:2006-08-22 s:2006-10-31[ 11] 69 | 14 | | [p:Test] Test #2 Due Date @laptop- e:2006-08-22 d:2006-11-13[ 12] 69 |    | | [p:Test] Test #3 No Dates @laptop- e:2006-08-22 

=====| Waiting |===== 

=====| Someday |===== 

This printout tells me that line [10] is an NA, but it’s not startable until tomorrow. Line [11] is another NA, but it’s not really current, because its due date is two weeks away. Line [12] is the only “current’ NA.

Seeing High-Priority NAs

It’s also possible to pull out those high-priority NAs in your system.

F:MiscellaneousGTDToDoBlog>getpri=====| (.) |=====[ 1] 70 | | (!) | Win lottery!!! @anywhere e:2006-08-22 

Seeing All NAs with Deadlines

One of my favourite commands is getdated, which retrieves all NAs with a due date (that are startable), and sorts them into a pretty format for me.

F:MiscellaneousGTDToDoBlog>getdated=====| OVERDUE |=====2006-08-23 [ 4] 68 | -68 | | [TV] Get Eureka’s next episode (recur: every wed) @online- e:2006-08-22                             s:2006-08-23 d:2006-08-232006-08-25 [ 3] 69 | -66 | | Wash Car @errands e:2006-08-22 d:2006-08-252006-08-30 [ 6] 69 | -61 | | [p:Bills] Download transactions and balance books @online+ e:2006-08-22                             d:2006-08-30 

=====| DUE TODAY (2006-10-30) |===== 

=====| DUE IN NEXT WEEK (on or before 2006-11-06) |===== 

=====| DUE IN NEXT MONTH (on or before 2006-11-27) |=====2006-11-13 [ 11] 69 | 14 | | [p:Test] Test #2 Due Date @laptop- e:2006-08-22 d:2006-11-13 

=====| DUE LATER (after 2006-11-27) |=====

So, I’ve got three overdue NAs. Nothing due today (well, yesterday, actually, since that’s when I printed this). Nothing due in the next week. One NA due in the next month, and nothing due later on. Note that any NAs with due dates, which also have start dates later than today, are NOT shown. This is because I use this system as my tickler too, and I don’t want to see stuff that I can’t start yet.

Conclusion

the point of this post is to introduce you to the script(s) I use for retrieving information from my todo.txt file. Right now, everything is in one huge perl script, and I don’t know if I want to break it up into individual commands (faster loading) or leave it in one file. So, you don’t get to see the script yet. But, I’m thinking of peeling off several of the commands and uploading the scripts, so you can see what they look like, and maybe even adjust them to your system, if you choose.

Earlier Posts in this Series:

• I’ve got a Text Fetish
• Text Fetish: Todo.txt Format (AKA NA Format)

Categories: GTD, todo.txt

I received an email the other day that struck me as the perfect opportunity for a walk-through demonstration of EverNote Template Creation. I’m going to seriously paraphrase here:

Hi GTDW, I know that you’re an EverNote Template Goddess. I suck really really bad at XML but have a cool template idea that I need help with. I’ve made a really pretty HTML table, based on the Reading Template, but I don’t know squat about turning it into an XML template for EN. Can you, in your wonderful beneficence, help me out? Please? Pretty please?

Now, note that I did say I was paraphrasing. The real email was more along the lines of:

I finally got around to making a Conference/Seminar Tracker. I did the basic HTML stuff for it. I based it on the Reading Template. I was confused by the lines:

I am not sure what there usage is. I am sending you the HTML part of the Template. Could you Templify It for me.

Either way, the request is valid. And a perfect opportunity to teach people about how easy it is to make an EverNote template.

How to Make an EverNote Template from an Old One

Step 1. Find Old Template. Find an existing template that you like, in this case, the Reading one (luckily available in the EverNote Template Catalogue here - top of the page).

Step 2. Look at the template’s xml file. Here’s what the xml look’s like for the reading template. An EverNote template always has the same structure, expressed in tags: EVERNOTE > TEMPLATES > TEMPLATE > CONTENT. The meaty part is in CONTENT.

Step 3. Find the table. As I said, the meaty part is in CONTENT. Specifically, most templates that we see around are based on a table (some are not, but they’re in the minority). Find the table tag, it goes something like this:

….

Step 4. Put table into HTML. Now, what you want to do, when modifying another template is find the table, paste it into a blank html document (to do that, open your favourite html editor, say new html document, then paste the table into the body, i.e., between the and tags. This lets you see the table in its full glory, play around with rows, columns, colours, etc. So, paste table into a new html document. If you’re lucky, you have access to an html editor that lets you see the code and a preview at the same time, say, like Dreamweaver.

Here, I’ve opened Dreamweaver, started a new HTML document, and pasted the table into the body. You can see part of the code at the top, and the table preview at the bottom.

Step 5. Modify the table. If possible, stick with the same variables, e.g., those things that are showing up as little ASPs in the table. If you look at the code for one of them, it looks like: - these are the variables that the template is using. Now, I’m not going to modify this table myself, my original requester has already done that. So, all I’m going to do, is paste his fancy html table on top of this one.

Now, it looks like this:

Step 6. Copy XML template file. Now it’s tricky. I have to turn this pretty html table back into an EverNote template. The easiest way to do this is to open the original template that you were looking at, and immediately save it as a different file name. That way, you’ve got a nice template to use to work on your template.

Here’s what the xml file looks like, in Dreamweaver:

Step 7. Paste table. First things first, replace the table in the xml file with the nice html table that you’ve just created.

(a) Select the table in xml

(b) Paste html table in its place

Step 8. Fix variables. Now, I have to be careful here, because I’m dealing with somebody else’s html table. Remember up above, where the person didn’t know what the

lines were for? This is what we’re going to figure out right now.

Actually, I’m not even sure what the does for sure, but if it’s in the original template, I just leave it there. On the other hand, the

lines are basically just variable declarations. These are going to be variables that hold information that you type in when the template is being used in EN. If it’s a variable (i.e., an editable variable), that means you can type in that field/cell when you use the template to create a note in EN.

So, the tricky part is that I have to look at what variables have been used in the table, and make sure that they’ve all been declared in this line that we’re discussing. Let’s see.

As I run through the table I see the following variables used:

• title
• author
• whenLocation
• pm_yes
• pm_no
• fullurl
• abstract
• contents
• notes

In the variable declaration line, I see: title, author, fullurl, remarks, notes, abstract, contents, date whenDownload, whenPrint, whenRead, checkbox done.

So, let’s just double-check the bullet-list and this list: title, author, fullurl, remarks, notes, abstract, contents, date whenDownload, whenPrint, whenRead, checkbox done

• title
• author
• whenLocation
• pm_yes
• pm_no
• fullurl
• abstract
• contents
• notes

Okay, so there’s some mismatch. What we need in the template is the following:

• Strings - these are basically things that you type, like names, notes, etc. So, string title, author, fullurl, abstract, contents, notes.
• Dates - these are date boxes. None in this new table, so I can actually just get rid of the date variables that have been declared, i.e., date whenDownload, whenPrint, whenRead.
• Checkboxes - these are, well, checkboxes. The new table has pm_yes and pm_no, but no done checkbox, so we’ll have: checkbox pm_yes, pm_no. Update #1: Actually, I’m not going to treat this as checkboxes, because I don’t want them to have any effect on the “to do/done” state of a note. After all, they just indicate that there are or are not printed materials.

In addition, there’s another string in the new table, called whenLocation, which I would have thought was a date (hence, the when). I’m just going to rename it to “location”, and put it in the list of strings above. (And rename the field the table so it’s “location” and not “whenLocation”.)

So, our variable declaration is going to look like this:

<% string title, author, fullurl, abstract, contents, notes, location;
checkbox pm_yes, pm_no %>

Step 10. Fix the title/heading of your template. Specifically, fix the line that says

Reading to something more appropriate for your new template. In this case, the template is going to be titled Conference/Seminar Tracker.

Step 11. Housekeeping of template header. There’s all sorts of information at the top of the template’s xml file: base, path, date, template created, id, name, etc.

I actually ignore almost all of it, except for two fields:

• id=”[00000000-0000-0000-0000000000000001]”
  • This is an identification number. Now, if you use these identification numbers, each template has a unique identifier. Then, when you make changes in the future to your template, and reimport it, the changes will be promulgated through your old notes. Very sweet. (Unless you make major changes like changing variable names, or removing variables, or something else that will screw up your old notes. But, if you’re like me and just like to change the format once in a while, or add new variables, then it’s a good thing.)
  • Now, it is true that you don’t need to use an identifier; you could just remove the tag. But, if you do this, and then change your template later, you’ll end up with two templates with the same name, but maybe different formats. Your old notes will not be updated with any template changes. But, you also won’t wreck any old notes by accident.
• name=”Reading”
  • This is the name of the template that you’ll see when you import it, or when you call on it in EverNote. You can name it whatever you like. Here, we’re going to name it Conference/Seminar Tracker.

I’m going to change the id number here to #8, which is the next one on my list. Note: if you’re going to import this template into your EN database, you want to make sure you don’t already have one with an id number of 8. (Odds are very low, unless you’re into scripting your own templates and have started with the low numbers like me.)

So, here’s what the header looks like now:

Step 12. Save the file. Save to an xml file with a useful name. Here, I’m going to save it as Conference Seminar Tracker.xml.

Step 13. Test out the template. I always test out a new template in a different database, just because my normal database is just to valuable to risk. Import the new template into your test database, and see how it looks.

Well, it doesn’t look too bad, but I don’t like the line-wrapping. Obviously, I need to tweak the size of a couple of the columns to get them to fit better.

Step 14. Fiddle. This is the fiddly bit. Keep playing with the table column sizes, e.g., the first cell is Conf/Seminar Title, and it’s info in html looks like:

defines a row, and defines a cell in that row. Here, it says that this column should be 15% of the overall width. Too small. I’m going to push that to 25%.

Also, for the cell with “printed materials”, it says 10% for that cell, then 10% for the column with the check boxes. If you look at the code for that whole row,

you’ll see 20, 10, 10, etc., for widths for some of the columns. I’m going to remove the width requirement for all of the columns in this row and see what happens.

To summarize, I’ve changed 15% to 25% for the Conf/Seminar Title cell, and removed all the width requirements in the busy row. Save the template xml file. Go back to the test database. Re-import the template. Check it out.

Well, no line wrapping now, but not too pretty either. So, back to fiddling. I’m not going to bore you with the details, but I’m going to just reiterate over this twitching the width sizes until I get it done pretty…

Took me three tries, but now I have this:

Oooh, pretty.

Summary

And that’s about it. It’s a painstaking process, but really, it’s not that hard. It’s taken me about an hour to write this post, while doing the fiddling, so, once you get the initial learning curve out of the way, it’s not that hard to create new templates based on old ones.

Links

Here’s the xml for this Conference/Seminar Tracking template.

Also, there are some older walk-throughs that I created when I was first learning templates in EverNote:

• What Does a Template File Look Like
• Anatomy of a Template
• Downloading Templates
• Saving Templates on Your Computer
• Creating a Template from Scratch
• Importing a New Template into EN

That’s it class. Any questions?

Update: After spending just over an hour lovingly crafting this blog in Windows Live Writer, I was dismayed to discover that the post couldn’t be uploaded. Stupid WLW, not even a useful error message. Sigh. That means that I had to recreate this post in Blogger. Sure, cut and paste is a wonderful thing, but there are 15 images in this blog. That means finding each image in my clipboard program, exporting it to a jpeg, putting them in the right order (since the preview in WLW was a bit smaller than readable), uploading the images through Blogger’s image upload interface, five at a time. Then, since Blogger stupidly pastes images at the top of the post, figuring out which image when where and moving it. ARGH. It’s taken me 45 minutes, including crashing WLW once and FireFox twice. This post had better be worth it ;)

Every now and then, I find a web page that just begs to be shared with you. PC World has an article entitled, “A Brief History of Computers, As Seen in Old TV Ads“. The author, Harry McCracken, seems to have spent a weekend trolling through YouTube clips, finding commercials featuring home computers. He pieces them all together chronologically, with some nice colour commentary. If you like watching old TV ads, and really, who doesn’t, check it out.

By the way, my first ever computer was a Commodore 64. I’m embedding the ad for that right here, just because I can :)

Update 2006-10-19 11:16: Fixed the missing link. Thanks J.

 

2007-05-14 09:35 Update: Updating links to the Perl script and exe file referenced herein. Unfortunately, the images have been lost forever.

One of the recurring complaints, at least from my perspective, is that you can’t search for a particular category. Anything in the category list is not included in the search results. Couple this lack of searchability with a category listing numbering more than a thousand, and you can see why it’s an annoying problem.

Over at the EverNote support forum, lcdEVN posted a couple of posts about categories. lcdEVN is requesting information on how to create a hierarchical list of category links. I loved the idea and was interested in a solution. In addition, lcdEVN’s first post provides enough information that I figured I could maybe pull something off with a simple Perl script.

It turns out that you can export the category list (as opposed to all your notes). The export results in an xml file, which contains enough information (with some tweaking) to create links to each individual category. The end result? A way, albeit painful, to create a note containing each and every category in linkable form. Then, you can search for a particular category name, find it in the note, and then follow the link to the actual category. Sweet. Here’s how it works.

Exporting the Category List

The first thing you need to do is export your EverNote database, but only the links. In EverNote, go to the file menu, select Export… and then make sure that only the Categories check box is checked.

Notice how many categories! Eeek. Selecting OK produces an xml file, which contains content that looks like this:

Ugly, but the information is there. See, any line that looks like this:

Can be turned into a link in an HTML document. That link can be copied and pasted back into the EverNote database, and can be clicked and followed to the category.

Converting the XML to an HTML List of Categories

I created a simple script in Perl that can read in an XML file and produce an HTML file containing a numbered list of categories. These categories are sorted alphabetically (by name of the category, not by identity of the link) and duplicates are removed. So what you end up with, is something that looks like this:

You’ll notice that there are two “Done Reading” categories. These are, in fact, completely different categories in different parts of the hierarchy.

How to Run the Perl Script

If you have a Perl execution installed on your computer, you can download the script and run it from the command line. Simply use the following command:

perl enalpha.pl “f:\\path\\endb.endb” enimport.xml result.html

The script is called enalpha.pl. The first parameter is the the path and filename of your EverNote database. I put it into quotes because my path has spaces. And notice the double backslashes. The second parameter is the file containing the exported category list. The second parameter is where the resulting code will be put. This command assumes that the perl script is sitting in the same directory as the xml file.

For those of you who don’t have perl installed, you can use this exe file in the same way.

Bringing the HTML Back into EverNote

All you have to do after running the script is open the html file in your browser, then copy the contents, and paste into a new note.

Searching for a Category

You can now search for anything in your database, and your category list will be included in the results. I’ve tagged my list with @categories so that I can first narrow down to the category list only, and then search in it. The only problem I have is that I have over a thousand categories, so the note itself is rather long. So, I keep it minimized and use the F3 button to toggle through any particular lines in the note that suit my search criteria.

Yeah, But Why No Hierarchy?

Originally, I wanted to create an actual hierarchy, just so that it looked the same as the category list on the left. However, the exported xml was too complicated. Sure, the hierarchy information is in there, but it would require multiple passes through the file, keeping track of where a category name showed up, and figuring out what subcategories needed to be inserted there. Also, my category list includes many duplicates, which means that my new note would be even longer than 1000+ lines. On the other hand, putting every single category into an array, sorting the array alphabetically, and then removing the duplicates was painfully easy. And, it results in the most concise list possible. Finally, it actually makes sense to sort the categories alphabetically, because then, I could scroll through the alphabetized list, which I sometimes want to do. And, why do I need to duplicate the hierarchy any way? It’s not like I need to be able to figure out exactly where a category sits in the list in order to navigate to it - that’s what the link is for!

I’m working my way, slowly, through a backlog of questions via email. Here’s one that I wanted to think about a bit before replying.

I was asked the following:

What if one day you decide to switch to Linux or Mac and 5000 notes and other information that you have collected with EverNote is stuck on windows.

EverNote is nice and has many features, but what are you going to do if one day you will need all your info on a different system where EverNote is not designed for?

The author also comments on the fact that he prefers dealing with mostly text files, maybe with some html, since it is eminently portable. Not only is is software agnostic, but operating system/platform as well.

Excellent question to ask. I’ve never migrated to a completely different operating system, but I have migrated my notes from one software program to another. Here is my experience in the matter.

First, any time you decide to make use of an application that stores data in some format, you have to weigh the benefits and risks. When I first started using a word processor, it was a WordPerfect version, with the blue screen. (Actually, my first word processor was whatever the Commodore 64 offered; I did my first year of university on a 64, but that’s another story.) Then, at work, we upgraded from WordPerfect to WordPerfect, until I was an expert in it. When it came time to migrate to Office, I was extremely reluctant. I actually dragged my feet for over a year. And, really, there weren’t even any compatibility issues - everything I had ever written in WordPerfect was readable in Word. No, I just didn’t want to learn a new application.

Switching Gears to Data/Information Management

I’ve always been a bit of a pack rat when it comes to keeping digital information. I remember a decade ago or so, I decided that the best way to keep track of everything that was crossing my desk was to create an Access database. So I did. It even had a front end on it; items were categories with a primary and secondary category, start date, due date, etc. I could search along several dimensions, print out what was relevant to a project, etc. It was extremely useful at the time. When I went back to school I investigated using it still, but the note taking for research is much different from the day to day of an administrative job.

So, I scrapped the database. I scouted around, and found TreePad. It was an amazing program in my mind, especially 5 years ago - I had never seen anything like it. I used it diligently until just over a year ago. At that point, I had almost 60MB in my most current database, with various other or archive databases littered about. Last summer though, I decided that TreePad was a bit too static for me. It seemed that I kept putting information in, but not getting anything out of it. To find something, I had to navigate through my category hierarchy, or search for specific terms. And the search was actually quite slow. So, I started looking around.

Enter EverNote (and OneNote)

I discovered both EverNote and OneNote at about the same time. I really liked the look and feel of OneNote, but EverNote won me over with its flexibility. Automatic categories. Multiple categories per note. Hierarchy as big as I want, even if I couldn’t search the category list itself. Easy clipping from the web.

Along the way, I also dabbled with other possible solutions, namely NetSnippets, Onfolio, etc. Only EN and ON have managed to stay in my stable. OneNote is where I keep things that, if they were on paper, I would put into a binder. For example, afghan patterns, recipes, latex code snippets, etc. Things that I don’t need to search very often, but when I want to see them, I just go to a notebook and flip through, looking for what I want. EverNote is where almost everything else goes. (Don’t tell anyone, but I also make great use of my RSS feed reader, GreatNews, to keep track of interesting posts.)

What’s the Point?

What’s the point of this long ramble? Well, last summer, I switched completely, cold turkey, from my TreePad database to a new EverNote one. I had to think about this decision. Theoretically, I could have copied every single note in my TreePad file into a new one in EN, but did I really need to? No. At that point in time, I was kind of in the middle of big projects for my research, so I wasn’t actually going to be splitting up my research material too badly. There were a few things I wanted to keep access too (e.g., list of warranty dates, passwords, etc.) The rest, I could just leave in TreePad format, and if I found a need, I would start it up and dig out what I needed. Now, some 15 months later, I recall only having to do that a handful of times. Of course, I’ll keep the database around, but to be honest, it’s sitting on another machine, in a backup folder. I’ve completely switched over to EN.

What about the Future?

This email made me contemplate the future with respect to EN. What if I decided I didn’t like the program anymore, or found something even better? What if the program was discontinued? What if I changed operating systems? Well, the last question is not much of a fear for me. If I do every move to a mac, I’m going to have bigger issues than dealing with my data - I’ll have a whole new paradigm to figure out.

Okay, what about if the program is discontinued? Well, I just keep my current copy of the software. I figure the operating system would have to drastically change (a la DOS to Win 3.1 kind of change) before the application wouldn’t run for me anymore. That may be pretty far in the future. And I’ll probably have to buy a new kind of media player (over DVD) before that happens.

The most likely drastic event would be me finding something new. I won’t say that I deliberately look around for new stuff, but I’m not actively searching, that’s for sure. There are a few things that I would like to see improved in EN, but not enough to make me switch to anything else. Besides, you just can’t beat the flexibility.

But what if I decided that EN was no longer for me. And I’ve already got something like 80MB invested in a database. And really, it’s not like it’s that exportable. Sure, you can export notes to HTML, or even text, but a lot of the beauty of my database is the categories, and the ability to autocategorize, etc. Even if I exported every single note as a text note, and imported them into the next great application, I would still be missing a key component of the information - categories, tags, relationships between items. It would be extremely expensive, time-wise, to recapture all of that. So, I guess, if I do move on, I’m going to have to do when I switched from Access to TreePad and from TreePad to EverNote - go cold turkey. Scary thought. But, it’s doable, as I’ve already proved to myself.

Text Fetish Alternative

Now, the person who wrote me the original email suggested that the best way to go is to just use text and maybe html to keep everything. I agree that this is a very versatile, albeit clunky solution. Sure, I could keep all of my information in text files: contact lists, calendar (I do love Remind), to do lists (my text fetish Perl thingy), shopping lists, Christmas lists, etc. etc. I could even keep research information in text files, say notes for a particular article in one file, with the article in the same directory, etc. etc. However, there are a couple of downsides to that:

1. Visual recognition. Text is all fine and dandy, but sometimes you want to clip something from the web, and you want to see it in your notes as it looked online. I believe that we remember things by how they look, not just by what they say. For example, if I’m skimming through a text book, or looking through notes, I tend to remember sort of what it looked like, e.g., font size, colour, shape of the text, etc. You lose all of this information when you paste to straight text.
2. Images. I’m a big fan of clipping images: screen shots in particular. Can’t do that in text. I guess I could create an html file with a link to the image, but then I’m storing two separate documents, and it’s a pain to do that, when I could just be pasting away.
3. Searching. One you break things up into a gazillion smaller files, you need either a great organization system, or a great search program, or both. I could probably do this, just using Copernic Desktop Search (and I do love it), but that’s a little heavy maybe. I also like knowing that everything important is in one or two files, instead of one or two thousand.
4. Relationships between data. One of the great things about EN, or TreePad, or OneNote, is the ability to group like things together. All notes on passwords can be in the same category, or in the same subtree, or in the same section of a notebook. The only way to categorize on your hard drive is with directories/folders. I’ve heard something about Mac people being able to add keywords to files, and maybe it’s also possible in Windows, but that, at the very least, would involve right-clicking on each and every file, and selecting properties, and clicking somewhere, and typing something, etc. etc. Much pain. Plus, EN (and even ON), offer the ability to ‘tag’ things. For example, I use a @next tag for any note in EN that has work that I have to do. ON offers flags that can do the same thing. In text files, I guess you could go with the same @next as a text addition to your file, and then use your desktop search program to find it, but still, clunky.

Conclusion

The moral of the story is that before you get involved in inputting any large amount of data into a software program, you need to be sure that this is the program for you, and that you’ll be happy with it for the foreseeable future. Just be aware of the fact that one day, you might be switching.

If you find that you are leery of committing yourself to one application, and deal mostly with text (no images, pen-writing software, etc.), then you could seriously consider going the text file way. In that case, spend some time thinking about how you want to organize things (e.g., directory structure, adding specific keywords to files in order to group them) and invest in a very good desktop search program.

Either way, remember that it’s the information that you’re collecting that’s really important, not the way you’re collecting it :)