Every couple of weeks, I have to do what I think of as “refilling the well.” It’s something between relaxing and recharging, pouring equal parts inspiration and motivation into the well of my brain so I can keep writing, keep creating.

My primary means of refilling the well is going to bookstores (my current personal favorites being Mysterious Galaxy and Villainous Lair). I’m as big a bibliophile as anyone else, enjoying the smell, the touch, the weight of books, even the sight of them, but it’s not just that.

I look around the store, and marvel at all the different books that got published. Books about the history of the telegraph. Books about a fictionalized War of the Roses set on a world that’s not Earth. Books about minotaurs and paladins going on quests together.

Seeing so much getting published, across so many different genres and styles, reminds me that there’s room for what I want to write. There’s room for my characters, for my worlds, for my stories. Every one of the books on the shelves started out as someone’s pencil scratch of an idea, and they found room and space to be made. If they can do it, I can too.

Warning: Spoilers ahead.

I’m not sure I’ll be back for Season 6 of Game of Thrones. I feel like this last season was the weakest one so far. I’m still processing why, but I suspect it’s because of the following things, mistakes that I’ll try to avoid in my own writing:

1) Focusing on the wrong things.

But because they did decide to show us the assault itself, they weren’t able to show us other things, like Sansa trying to work out different ways to escape, or talking to the different servants to find out which ones she could rely on. They couldn’t show us the preparations for a siege at Winterfell, with Sansa trying to take advantage of the chaos to send a raven to Littlefinger or study the walls to remind herself of the best way over them.

I think it was a similar mistake to insist on showing us the full extent of Cersei’s humiliation, including the entire walk of shame. I didn’t want to see it, I didn’t need to see it – seeing her at the next small council meeting, head shaved and face cut, shaking as she reaches for her wine, is enough – and it prevented them from showing me other things, like Kevan trying to get her back, or the whole of them dealing with the aftermath.

I’ll admit that GoT takes place in a nasty world, where nasty things happen. But I didn’t need to see Craster actually rape his daughters to know he was a nasty man and understand what was happening there. I didn’t need to see King Robert’s sexual orgies to know the humiliation his antics caused Jaime and Cersei. And I didn’t need to see Viserys force himself on his sister to know she lived in fear of him.

2) Moving characters around instead of letting them move.

Take Jon Snow going to Hardhome. Why was this necessary? I understand that without Jon Snow there, there’s no perspective character to show us the assault of the army of the dead. But it would have made more sense for Aliser Thorne to have gone instead of Jon: he’s First Ranger, and known to hate the Wildlings more than Jon. Wouldn’t the oath to give them safe passage have been more impressive coming from an old and known enemy?

Jaime and Bronn going to fetch Myrcella also didn’t make sense to me. I mean, I understand wanting to show a buddy knight trip between the two of them, but Jaime has little reason to go and Bronn has less, and their presence didn’t affect the outcome at all. If they hadn’t been there, the Sand Snakes would have tried to kidnap Myrcella, failed, and any messenger from Cersei asking to see her daughter would have given Doran the excuse he needed to send Myrcella away to safety.

Finally we have Jorah. His decision to sign up for gladiator combat the first time made sense, since it gave him a chance to see Daenerys again. But submitting to slavery a second time after being banished again? Only made sense as a way to place him near her during the Sons of the Harpy attack. For the character, it didn’t make sense at all.

3) Trying too hard for big moments.

I think the first two mistakes, made often enough over the course of the season, robbed the emotional high points of any impact. Instead of caring that Brienne finally got to confront Stannis, I just saw a knight come upon an old wounded man in the forest, tell him her name, and deliver a killing blow. Instead of dying a little inside at seeing Jon bleeding out in the snow, I knew from the moment Olly came to fetch him that he was about to be ambushed, and the circle of knives was way too much “Et tu, Brute?” to make me do anything other than shake my head.

And Drogon saving the day . Well, of course he saved the day, then dumped Daenerys in the middle of nowhere instead of somewhere else in the city. How else were the writers to setup Daenerys being standard in the wilderness, needing her two bravest knights to come save her (groan)?

None of it worked for me, and the parts that did deliver an emotional impact – Sansa’s assault, Cersei’s humiliation – were entirely negative. For me, this season was a set of lessons in what not to do. Here’s hoping I take them to heart.

Outline’s not quite done. I keep bouncing back and forth between the plot and the characters, each change rippling out and making me re-arrange chapters and scenes.

It feels harder this time, and I can’t tell if that’s because it’s such a different book, or because I’m simply afraid of not being able to write a second novel, or because the idea’s not as good as I think it is.

The only thing I can do is keep plugging away at it, pushing the outline around until I have enough of a plot shape to start writing. I tell myself that all it takes is time, and I should be patient, but that doesn’t change the fact that my last outline took me two weeks while this one is a month and counting.

Maybe I should just dive in and start writing, outline be damned? Maybe what worked for the last book isn’t going to work for this one.

It might come to that. In any case, I’m setting a deadline for myself of July 1st. Outline or no, I’m going to start writing the first scene on or before then.

Incredibly long, but eye-opening. So much more of the news makes sense to me now, like I’ve been given a set of mental footnotes for each story that mentions anything related to energy.

The book could’ve used some serious editing, though. I found numerous typos, misspellings, repeated phrases, even whole paragraphs that echo each other.

In addition, the book starts out at a good pace, but begins to feel like a slog somewhere around Part 4 (where he discusses renewable energy, then spends Part 5 going into more detail about renewables). Had to push myself to finish this one.

That said, I learned an incredible amount, including:

• In 2011, the US was 78% self sufficient in overall energy: natural gas, oil, coal, nuclear, renewables, etc
• Cap-and-Trade as a solution to carbon pollution was pioneered by the lead permits issued to refineries in the 80s to eliminate lead from gasoline
• In 2009, newly discovered reserves equalled the amount of oil produced over the entire lifetime of the industry

Thanks to the good advice from L.D. Parker in the comments, I resolved last week’s plotting dilemma by deciding to interweave the narrative from the trigger character and protagonist’s point of views. I’ll start with the initial “hook” scene I have in mind for the first chapter, then do an intro chapter with the protag, then alternate back and forth throughout the book.

I tell myself that even if it doesn’t work the way I want it to, I can go back and do whatever I need to fix it (probably drop the trigger character and focus solely on the protag), so long as I make it through the first draft.

With that problem solved, I finally started writing up the outline, character sketches, etc for the new book in Scrivener. This means taking all the notes I’ve jotted down – some handwritten, some typed up in Evernote, some dictated into my phone – and bringing them together in one place, and imposing some sort of order on them.

It’s the last step before I actually start writing, and it gives me a visual indication of holes in the plot, of weak points in the story that’s developing. For example, I can already see that I’m going to need a lot more background for my protag than I have worked out so far, simply because the number of their scenes aren’t balanced against the trigger character.

With luck, I’ll have the initial plan written up and into Scrivener by the end of next week, and then I’ll be ready to plunge back into the blank page and start swimming toward the finish line.

I’ve had two generations of Nook ereaders. I liked holding them better than the Kindles that were available, I wanted to feel good about buying ebooks after browsing at my local Barnes & Noble, and I didn’t like the way Amazon was waging war against book publishers (and, as a consequence, on authors).

But B&N hasn’t updated their Nook in almost two years. Their last Nook’s screen resolution is good, but still not as good as a printed book. It has an annoying habit of setting the margins so wide that the text forms a three-word column down the screen, and then locks me out of making any adjustments. It doesn’t sync the last page read between the ereader and my iPhone. Its illumination is noticeably uneven. The covers for it are terrible and expensive, so when I travel I put it back inside the box it came in. I have to re-adjust the fonts and margins everytime I open (or re-open) a book, because it doesn’t remember my settings.

None of which are show-stoppers, for sure, but over time they add up. The final straw was when B&N locked users out of downloading our ebooks to our computers. I used to do this on a regular basis, so I could save backups of the books to Dropbox. That changed a few months ago, when they took down the download link next to all the books in their users' Nook online libraries.

So I went shopping for a new ereader. I worried that I might have to go with a Kindle, since they seemed to have the best screen resolution out there.

Then I heard about the Kobo Glo HD. I knew Kobo already, since they stepped into the breech left behind by Google dropping its ebook partnership with independent bookstores. I knew they produced ereaders, since I’d seen them for sale at Mysterious Galaxy (as part of their collaboration with Kobo). The reviews I found of them were generally positive, and the Glo HD - which hadn’t come out yet - promised a screen resolution as good as the Kindle, and at a cheaper price.

I couldn’t find one locally to try out, so I took the plunge and ordered it. I’m very, very glad I did; I’ve been using it for a month now, and I can honestly say this is the ereader I’ve been waiting for.

The screen resolution is sharp enough that it looks like a printed book when I set it down on a table to read. And unlike the Nook’s dark screen, the Glo HD’s is bright enough that I don’t feel the need to turn on the reading light during the day.

Syncing? My bookmarks sync between the Kobo app on my phone and the ereader no problem, easy as pie, even for books that I didn’t buy from Kobo.

Sideloading was a little more complicated than I’d like. I had to use Adobe Digital Editions to connect to the reader and transfer books over, but it moved all 142 of my backed up B&N books without a hitch, and they all showed up in my Library on the Glo just fine.

I still have to adjust the fonts sometimes between books, but I no longer care. I don’t care because the options for tweaking are incredible: I’ve got a dozen different fonts, sliders for font size, line spacing, and margins, as well as the ability to set justification to full, left, or simply off. And I’ve yet to encounter a book that locks me into reading a certain way. I’ve got full control over how the book looks, and it’s about freaking time.

Even the case they sell for it is amazing. Its the first case for any portable device – Nook, iPad – that actually makes the original device better. It doesn’t add to the Glo’s weight, closing it puts the reader to sleep and opening it wakes it up, and when its open it folds back behind the reader to make it feel even more like a book in your hand. Oh, and it kept the screen scratch-free in my backpack over four cross-country flights.

So this is one gamble that’s completely paid off. It’s the first ereader that I prefer reading on to a paper book, so much so that I have to stop myself from buying ebook versions of the hardcovers on my bookshelf just so that I can read them on the Glo HD.

Had a minor hiccup in outlining the new book while I was traveling: I decided to change the main character. Thought for a few days there I’d have to split the books off, and use the previous set of characters for the plot I had been working on while thinking up a different plot for the new protagonist.

Thankfully my split-brain didn’t last long; I realized I could rework the old plot for the new characters, and keep everything I liked about both.

So now I’m laying out the narrative for the new book. I’ve got the antagonist’s motives and moves down, and the same for what I’m thinking of as the “trigger character,” the one that starts things moving.

Having some trouble deciding when and how to bring in my protagonist though. I’ve got several different routes to take, some of which use the initial scene I want to have and some that don’t. The problem (at the moment) is that the routes that don’t use that scene make more sense than the ones that do.

So do I let go of this little darling scene of mine? Or do I brainstorm until I can find a way to keep it and have a protag intro that makes sense?

Haskell and Power Series Brought to Life

• not interested in convergence
• laziness lets you handle infinite series
• head/tail great for describing series
• operator overloading lets you redefine things to work on a power series (list of Nums) as well as Nums
• multiplication complication: can't multiply power series by a scalar, since they're not the same type
• could define negation as: negate = map negate
  • instead of recursively: negate(x:xs) = negate x : negate xs
• once we define the product of two power series, we get integer powers for free, since it's defined in terms of the product
• by using haskell's head-tail notation, we can clear a forest of subscripts from our proofs
• reversion, or functional inversion, can be written as one line in haskell when you take this approach:
  • revert (0:fs) = rs where rs = 0 : 1/(fs#rs)
• can define integral and derivative in terms of zipWith over a power series
• once we have integrals and derivatives, we can solve differential equations
• can use to express generating functions, which lets us do things like pascal's triangle
• can change the default ordering of type use for constants in haskell to get rationals out of the formulas instead of floats
  • default (Integer, Rational, Double)
• all formulas can be found on web page: ???
  • somewhere on dartmouth's site
• why not make a data type? why overload lists?
  • would have needed to define Input and Ouput for the new data type
  • but: for complex numbers, algebraic extensions, would need to define your own types to keep everything straight
  • also: looks prettier this way

How to Learn Haskell in Less than 5 Years

• Chris Allen (bitemyapp)
• title derives from how long it took him
  • though, he says he's not particularly smart
• not steady progress; kept skimming off the surface like a stone
• is this talk a waste of time?
  • not teaching haskell
  • not teaching how to teach haskell
  • not convince you to learn haskell
  • WILL talk about problems encountered as a learner
• there is a happy ending: uses haskell in production very happily
• eventually made it
  • mostly working through exercises and working on own projects
  • spent too much time bouncing between different resources
  • DOES NOT teach haskell like he learned it
• been teaching haskell for two years now
  • was REALLY BAD at it
  • started teaching it because knew couldn't bring work on board unless could train up own coworkers
• irc channel: #haskell-beginners
• the guide: github.com/bitemyapp/learnhaskell
• current recommendations: cis194 (spring '13) followed by NICTA course
• don't start with the NICTA course; it'll drive you to depression
• experienced haskellers often fetishize difficult materials that they didn't use to learn haskell
• happy and productive user of haskell without understanding category theory
  • has no problem understanding advanced talks
  • totally not necessary to learn in order to understand haskell
  • perhaps for work on the frontiers of haskell
• his materials are optimized around keeping people from dropping out
• steers them away from popular materials because most of them are the worst ways to learn
• "happy to work with any of the authors i've critized to help them improve their materials"
• people need multiple examples per concept to really get it, from multiple angles, for both good and bad ways to do things
• doesn't think haskell is really that difficult, but coming to it from other languages means you have to throw away most of what you already know
  • best to write haskell books for non-programmers
  • if you come to haskell from js, there's almost nothing applicable
• i/o and monad in haskell aren't really related, but they're often introduced together
• language is still evolving; lots of the materials from 90s are good but leave out a lot of new (and useful!) things
• how to learn: can't just read, have to work
• writing a book with Julie (?) @argumatronic that will teach haskell to non-programmers, should work for everyone else as well; will be very, very long (longer than Real World Haskell)
• if onboarding new employee, would pair through tutorials for 2 weeks and then cut them loose
• quit clojure because he and 4 other clojurians couldn't debug a 250 line ns

Production Web App in Elm

• app: web-based doc editor with offline capabilities: DreamWriter
• wrote original version in GIMOJ: giant imperative mess of jquery
• knew was in trouble when he broke paste; could no longer copy/paste text in the doc
• in the midst of going through rewrite hell, saw the simple made easy talk by rich hickey
• "simple is an objective notion" - rich hickey
  • measure of how intermingled the parts of a system are
• easy is subjective, by contrast: just nearer to your current skillset
• familiarity grows over time -- but complexity is forever
• simpler code is more maintainable
• so how do we do this?
  • stateless functions minimize interleaving
  • dependencies are clear (so long as no side effects)
  • creates chunks of simpleness throughout the program
  • easier to keep track of what's happening in your head
• first rewrite: functional style in an imperative language (coffeescript)
  • fewer bugs
• then react.js and flux came out, have a lot of the same principles, was able to use that to offload a lot of his rendering code
  • react uses virtual dom that gets passed around so you no longer touch the state of the real dom
• got him curious: how far down the rabbit-hole could he go?
  • sometimes still got bugs due to mutated state (whether accidental on his part or from some third-party lib)
• realized: been using discipline to do functional programming, instead of relying on invariants, which would be easier
• over 200 languages compile to js (!)
• how to decide?
• deal-breakers
  • slow compiled js
  • poor interop with js libs (ex: lunar.js for notes)
  • unlikely to develop a community
• js but less painful?
  • dart, typescript, coffeescript
  • was already using coffeescript, so not compelling
• easily talks to js
  • elm, purescript, clojurescript
  • ruled out elm almost immediately because of rendering (!)
• cljs
  • flourishing community
  • mutation allowed
  • trivial js interop
• purescript
  • 100% immutability + type inference
  • js interop: just add type signature
  • functions cannot have side effects* (js interop means you can lie)
• so, decision made: rewrite in purescript!
  • but: no react or flux equivalents in purescript (sad kitten)
• but then: a new challenger: blazing fast html in eml (blog post)
  • react + flux style but even simpler and faster (benchmarked)
• elm js interop: ports
  • client/server relationship, they only talk with data
  • pub/sub communication system
• so, elm, hmm…
  • 100% immutability, type inference
  • js interop preserves immutability
  • time travelling debugger!!!
  • saves user inputs, can replay back and forth, edit the code and then replay with the same inputs, see the results
• decision: rewrite in elm!
• intermediate step of rewriting in functional coffeescript + react and flux was actually really helpful
  • could anticipate invariants
  • then translate those invariants over to the elm world
  • made the transition to elm easier
• open-source: rtfledman/dreamwriter and dreamwriter-coffee on github
• code for sidebar looks like templating language, but is actually real elm (dsl)
• elm programs are built of signals, which are just values that change over time
• only functions that have access to a given signal have any chance of affecting it (or messing things up)
• so how was it?
  • SO AWESOME
  • ridiculous performance
  • since you can depend on the function always giving you the same result for the same arguments, you can CACHE ALL THE THINGS (called lazy in Elm)
  • language usability: readable error messages from the compiler (as in, paragraphs of descriptive text)
  • refactoring is THE MOST FUN THING
  • semantic versioning is guaranteed. for every package. enforced by the compiler. yes, really.
  • diff tool for comparing public api for a lib
  • no runtime exceptions EVER
• Elm is now his favorite language
• Elm is also the simplest (!)
• elm-lang.org

An odd mix of spot-on observations and disturbing cruelty. Bryson nails the incomprehensibility of the Glasgow accent to American ears in one place, then in another compares an overweight family eating dinner to a group of slow-witted bovines.

It’s refreshing to hear a travel writer be frank about the parts of the country they don’t like, and after living in the UK for two decades Bryson seems to have plenty to complain about. He also doesn’t flinch from talking about his own rudeness, relaying scenes where he browbeat a hotel owner, or insulted a woman’s intelligence because her dog got too close to him. Perhaps to a different audience, in a different time (the book was published in 1995) it comes across as bracing honesty, or even funny, but to me it just made him seem like a bit of a jerk.

Still, his writing voice is strong and pleasant, even when he’s not, and I went through the book quickly. I did manage to learn a few things:

• Blackpool's beach isn't a beach. In a stroke of ketchup-is-a-vegetable genius, Thatcher got around EU beach cleanliness regulations by decreeing that Britain's resort beaches weren't beaches at all!
• As recently as the 70s Liverpool was the second busiest port in the UK, and it used to be the 3rd wealthiest town in Britain.
• Glasgow won European City of Culture in 1990

An excellent antidote to the normal narratives of invention and progress.

But Edgerton isn’t a Luddite, or a cynic that doesn’t believe in progress. Instead, he sets out to fill in the stories that normally get glossed over in normal histories: the importance of horsepower to the modern armies of World War II, the communities in West Africa that have grown up specifically to maintain the cars and trucks they inherit from the developed world using local materials, the resurgence in whaling in the 1920s and 1930s driven by demand for whale oil to be used in margarine. It’s fascinating, incredibly readable, and it changed the way I read stories of technical progress and achievement.

Three facts in particular stood out to me:

• India and Taiwan produce more bicycles each year than the entire world did in 1950.
• In 2003, the largest R&D spenders weren't in biotech or the internet; they were car companies: Ford, Daimler Chrysler, Toyota, etc.
• The rickshaw, which I always assumed was an old tech lingering in the modern world, was in fact only invented in 1870, in Japan.

A quick, enjoyable read. Indiana Jones crossed with John Le Carré sprinkled with some Inception-like plotting.

Presents itself as a regular sci-fi novel, but the first half is almost completely filled with flashbacks, a series of nested stories, one inside the other, each level going one step further back into the past. Macleod pulls it off by having the same narrator tell most of it, then uses interrogation transcripts and letters to fill out the rest.

It’s nested all the way down, with the novel’s big ideas woven into the structure of the narrative itself. Ultimately works it way back to the very beginning, the first story, closing the loop in a very tidy (but not too tidy) way.

It’s the best method of infodumping I’ve seen in a long time.

Macleod may have carried the nesting too far. By the time I reached the end of the book (and back to the first level of nested story) I had to re-read the beginning to remind myself of what was going on there, and I’m not sure the details between the two endpoints match up.

Still, it’s a lesson in how to present a lot of backstory (~100 pages worth!) to the reader without it feeling shoved down their throat.

I’ve decided to stick with the plan of working on the new novel first, and then going back to edit the last one.

That means I need to get cracking on a serious outline for the new book. Got some travel coming up next week, which should give me some dedicated time to start hashing that out. I’ve got the three main characters already and the basic plot, so it’ll be a matter of working out the beats of the story as well as taking a first stab at how it will end. That ending will probably be completely different by the time its written, but I need a star to steer by if I’m going to get in the boat.

With luck — and work, let’s not forget time-in-chair — I’ll have the outline done in two weeks, so I can start writing in earnest by June 1st. That’ll give me six months of runway to be finished before the end of the year.

Let’s hope it’s enough.

Medieval-level fantasy with goblins and elves, airships and intrigue, and race relations and gender politics and multiple sexual orientations. In a word, awesome.

Vivid and rich and alive in unexpected ways. The plot is rather basic — outsider unexpectedly inherits the throne, has to learn to rule people that look down on him — but the characters are so interesting, so well fleshed-out, that it held me all the way through. I might just read it again.

The big writing lesson for me from this book is exactly that: well-written characters that you want to spend time with will compensate for a lot of other shortcomings. For The Goblin Emperor, those shortcomings would normally compel me to stop reading.

I gave up trying to pronounce many of the fantasy words and names it introduces. The glossary of terms, which I found while desperately searching for some sort of help in keeping terms and titles and characters straight, proved to be worthless. Many of its definitions are either self-referential or refer to other terms which are. There’s also no map, so I had no idea of the relative size or placement of any of the cities and nations mentioned in the book. As some of the intrigue involves trade relations among neighboring realms, this was frustrating.

But I ultimately didn’t care. I cared about the main character from the first chapter, and cared about the others almost as quickly. I skipped over names, I couldn’t keep any of the titles straight, I had no idea where anything was, and I didn’t care. The main challenges of the book were people, and I wanted the main character to succeed with all of them. Everything else faded away.

Even while outlining the new novel, I keep wanting to go back to the old.

Especially now that I’ve gotten some reader feedback. Makes me think maybe my time would be better spent editing and polishing the novel I’ve got, rather than writing a new one.

Or perhaps I should switch to outlining and writing the sequel to the last novel? That’d be easier: I’ve already got the characters and the world in place, and some history established. I should be able to get off and running on that book faster than the first, right?

It’s hard for me to tell whether this is my normal flightiness or a sensible course correction.

For now, I’m going to stick with the plan: outline and write a new book with completely different characters, then come back to the first novel and edit it into shape. With that done, I’ll be able to alternate periods of new writing with periods of editing, and hopefully get into a rhythm that’ll sustain me all the way to publication.

Genre fiction has always been aimed at the Outsider, at the person with enough distance from the dominant culture to think critically about it.

It’s just that our definition of Outsider has expanded.

When I was a kid, I felt like an Outsider because I was clumsy and nerdy and socially awkward. The school’s hierarchy enforced that status: football players were in, science geeks were out. Genre fiction was pitched directly at me, giving me an escape from social rejection and poverty and feeding into the sense of wonder I held about the world around me.

I never thought about the fact that, as a white male, the ladder I felt myself to be on the low rungs of was already placed far over the heads of other groups.

As an adult, I no longer feel like an Outsider. Though I undoubtedly am an Outsider when in certain company — I’m an atheist, which puts me out from most of the American populace, and a programmer, which makes my work boring to most people — I don’t feel like one day in and day out.

I’ve come to realize that there are other people who feel much more like Outsiders than I ever did, and that while my Outsider-status has diminished with adulthood, theirs has likely only been enhanced, as their life experiences diverge from what’s considered acceptable in wider society.

These people — women*, people of color, the LBGT community — deserve genre fiction that speaks to them, that talks about their experiences as Outsiders (and Insiders**), that addresses their issues and their needs. I’m glad to see my favorite section of the bookstore embracing them, proud to see us growing up as a subculture.

I still enjoy this fiction, even though I’m straight, and white, and male. Because I remember being the kid that didn’t fit in, that no one wanted to play with, that adults felt uncomfortable around and kids didn’t want to talk to. Sci-fi and fantasy was there for me, and it can and should be there for others, as long as there are outsiders that need it.

The Imagination is a big place. there’s room for all of us.

* Which, holy shit, that half of the population should be sidelined in pop culture for so long is mind-boggling ** Everyone that belongs to a subculture outside the norm is automatically an Insider for that subculture

Wonderfully written re-discovery of the world of 1913 via a tour of its major cities. Manages to give a feel for each without dwelling too long on any one city.

Ends on a haunting note, with the assurances and questions of 1913 obliterated by the war of 1914. 1913 comes to seem an extension of the 19th century, rather than the beginning of the 20th, a different world that had a different future, once. Final chapter quotes a German intellectual returning home after the war to see everything preserved as if 1913 had been frozen in time: the British books, the Persian cigars from French friends, the Russian plays, all transformed, all changed now that the internationalism of 1913 had been dismantled by four years of war.

Three of the many things I learned about the state of the world in 1913:

• The Ottoman Empire was still in the midst of the reforms and changes brought about by the Young Turks and the new parliamentary government they had brought back
• Woodrow Wilson originally ran on a platform of domestic reform, and hoped that his presidency would leave him free from foreign policy crises so he could focus on it.
• Non-European Algerians were french subjects, not citizens. They could become citizens only by renouncing Islam and applying for citizenship

First reader review of the novel draft is in! And it’s generally positive!

True, it’s from a good friend of mine, who’s definitely biased. And yes, he had a list of suggested edits for me, from grammar mistakes to confusing descriptions to scenes that dragged on too long (all of which he’s right about). But overall he liked it, and he wants to read more.

Maybe it’s not as bad as I feared, after all.

Or perhaps it is, and he’s seeing past the mistakes to how good the novel could be, if edited into shape.

Either way, it’s encouraging for me to hear. If I can entertain one person with the first draft, I can entertain more with the second, and even more with the third. It’ll take work to get there, but this kind of validation, however biased it may be, makes me feel like it’ll be worth it.

Everything Will Flow

• Zach Tellman, Factual
• queues: didn't deal with directly in clojure until core.async
• queues are everywhere: even software threads have queues for their execution, and correspond to hardware threads that have their own buffers (queues)
• queueing theory: a lot of math, ignore most
• performance modeling and design of computer systems: queueing theory in action
• closed systems: when produce something, must wait for consumer to deal with it before we can produce something else
  • ex: repl, web browser
• open systems: requests come in without regard for how fast the consumer is using them
  • adding consumers makes the open systems we build more robust
• but: because we're often adding producers and consumers, our systems may respond well for a good while, but then suddenly fall over (can keep up better for longer, but when gets unstable, does so rapidly)
• lesson: unbounded queues are fundamentally broken
• three responses to too much incoming data:
  • drop: valid if new data overrides old data, or if don't care
  • reject: often the only choice for an application
  • pause (backpressure): often the only choice for a closed system, or sub-system (can't be sure that dropping or rejecting would be the right choice for the system as a whole)
  • this is why core.async has the puts buffer in front of their normal channel buffer
• in fact, queues don't need buffer, so much as they need the puts and takes buffers; which is the default channel you get from core.async

Clojure At Scale

• Anthony Moocar, Walmart Labs
• redis and cassandra plus clojure
• 20 services, 70 lein projects, 50K lines of code
• prefer component over global state

Data Science in Clojure

• Soren Macbeth; yieldbot
• yieldbot: similar to how adwords works, but not google and not on search results
• 1 billion pageviews per week: lots of data
• end up using almost all of the big data tools out there
• EXCEPT HADOOP: no more hadoop for them
• lots of machine learning
• always used clojure, never had or used anything else
• why clojure?
  • most of the large distributed processing systems run on the jvm
  • repl great for data exploration
  • no delta between prototyping and production code
• cascalog: was great, enabled them to write hadoop code without hating it the whole time, but still grew to hate hadoop (running hadoop) over time
• december: finally got rid of last hadoop job, now life is great
• replaced with: storm
• marceline: clojure dsl (open-source) on top of the trident java library
• writing trident in clojure much better than using the java examples
• flambo: clojure dsl on top of spark's java api
  • renamed, expanded version of climate corp's clj-spark

Pattern Matching in Clojure

• Sean Johnson; path.com
• runs remote engineering team at Path
• history of pattern matching
  • SNOBOL: 60s and 70s, pattern matching around strings
  • Prolog: 1972; unification at its core
  • lots of functional and pattern matching work in the 70s and 80s
  • 87: Erlang -> from prolog to telecoms; functional
  • 90s: standard ml, haskell...
  • clojure?
• prolog: unification does spooky things
  • bound match unbound
  • unbound match bound
  • unbound match unbound
• clojurific ways: core.logic, miniKanren, Learn Prolog Now
• erlang: one way pattern matching: bound match unbound, unbound match bound
• what about us? macros!
• pattern matching all around us
  • destructuring is a mini pattern matching language
  • multimethods dispatch based on pattern matching
  • case: simple pattern matching macro
• but: we have macros, we can use them to create the language that we want
• core.match
• dennis' library defun: macros all the way down: a macro that wraps the core.match macro
  • pattern matching macro for defining functions just like erlang
  • (defun say-hi (["Dennis"] "Hi Dennis!") ([:catty] "Morning, Catty!"))
  • can also use the :guard syntax from core.match in defining your functions' pattern matching
  • not in clojurescript yet...
• but: how well does this work in practice?
  • falkland CMS, SEACAT -> incidental use
  • POSThere.io -> deliberate use (the sweet spot)
  • clj-json-ld, filter-map -> maximal use
• does it hurt? ever?
• limitations
  • guards only accept one argument, workaround with tuples
• best practices
  • use to eliminate conditionals at the top of a function
  • use to eliminate nested conditionals
  • handle multiple function inputs (think map that might have different keys in it?)
  • recursive function pattern: one def for the start, one def for the work, one def for the finish
    • used all over erlang
    • not as explicit in idiomatic clojure

Life of a Clojure Expression

• John Hume, duelinmarkers.com (DRW trading)
• a quick tour of clojure internals
• giving the talk in org mode (!)
• disclaimers: no expert, internals can change, excerpts have been mangled for readability
• most code will be java, not clojure
• (defn m [v] {:foo "bar" :baz v})
• minor differences: calculated key, constant values, more than 8 key/value pairs
• MapReader called from static array of IFns used to track macros; triggered by '{' character
• PersistentArrayMap used for less than 8 objects in map
• eval treats forms wrapped in (do..) as a special case
• if form is non-def bit of code, eval will wrap it in a 0-arity function and invoke it
• eval's macroexpand will turn our form into (def m (fn [v] {:foo "bar :baz v}))
• checks for duplicate keys twice: once on read, once on analyze, since forms for keys might have been evaluated into duplicates
• java class emitted at the end with name of our fn tacked on, like: class a_map$m
• intelli-j will report a lot of unused methods in the java compiler code, but what's happening is the methods are getting invoked, but at load time via some asm method strings
• no supported api for creating small maps with compile-time constant keys; array-map is slow and does a lot of work it doesn't need to do

Clojure Parallelism: Beyond Futures

• Leon Barrett, the climate corporation
• climate corp: model weather and plants, give advice to farmers
• wrote Claypoole, a parallelism library
• map/reduce to compute average: might use future to shove computation of the average divisor (inverse of # of items) off at the beginning, then do the map work, then deref the future at the end
• future -> future-call: sends fn-wrapped body to an Agent/soloExecutor
• concurrency vs parallelism: concurrency means things could be re-ordered arbitrarily, parallelism means multiple things happen at once
• thread pool: recycle a set number of threads to avoid constantly incurring the overhead of creating a new thread
• agent thread pool: used for agents and futures; program will not exit while threads are there; lifetime of 60 sec
• future limitations
  • tasks too small for the overhead
  • exceptions get wrapped in ExecutionException, so your try/catches won't work normally anymore
• pmap: just a parallel map; lazy; runs N-cpu + 3 tasks in futures
  • generates threads as needed; could have problems if you're creating multiple pmaps at once
  • slow task can stall it, since it waits for the first task in the sequence to complete for each trip through
  • also wraps exceptions just like future
• laziness and parallelism: don't mix
• core.async
  • channels and coroutines
  • reads like go
  • fixed-size thread pool
  • handy when you've got a lot of callbacks in your code
  • mostly for concurrency, not parallelism
  • can use pipeline for some parallelism; it's like a pmap across a channel
  • exceptions can kill coroutines
• claypoole
  • pmap that uses a fixed-size thread pool
  • with-shutdown! will clean up thread pool when done
  • eager by default
  • output is an eagerly streaming sequence
  • also get pfor (parallel for)
  • lazy versions are available; can be better for chaining (fast pmap into slow pmap would have speed mismatch with eagerness)
  • exceptions are re-thrown properly
  • no chunking worries
  • can have priorities on your tasks
• reducers
  • uses fork/join pool
  • good for cpu-bound tasks
  • gives you a parallel reduce
• tesser
  • distributable on hadoop
  • designed to max out cpu
  • gives parallel reduce as well (fold)
• tools for working with parallelism:
  • promises to block the state of the world and check things
  • yorkit (?) for jvm profiling

Boot Can Build It

• Alan Dipert and Micha Niskin, adzerk
• why a new build tool?
  • build tooling hasn't kept up with the complexity of deploys
  • especially for web applications
  • builds are processes, not specifications
  • most tools: maven, ant, oriented around configuration instead of programming
• boot
  • many independent parts that do one thing well
  • composition left to the user
  • maven for dependency resolution
  • builds clojure and clojurescript
  • sample boot project has main method (they used java project for demo)
  • uses '--' for piping tasks together (instead of the real |)
  • filesets are generated and passed to a task, then output of task is gathered up and sent to the next task in the chain (like ring middleware)
• boot has a repl
  • can do most boot tasks from the repl as well
  • can define new build tasks via deftask macro
  • (deftask build ...)
  • (boot (watch) (build))
• make build script: (build.boot)
  • #!/usr/bin/env boot
  • write in the clojure code defining and using your boot tasks
  • if it's in build.boot, boot will find it on command line for help and automatically write the main fn for you
• FileSet: immutable snapshot of the current files; passed to task, new one created and returned by that task to be given to the next one; task must call commit! to commit changes to it (a la git)
• dealing with dependency hell (conflicting dependencies)
  • pods
  • isolated runtimes, with own dependencies
  • some things can't be passed between pods (such as the things clojure runtime creates for itself when it starts up)
  • example: define pod with env that uses clojure 1.5.1 as a dependency, can then run code inside that pod and it'll only see clojure 1.5.1

One Binder to Rule Them All: Introduction to Trapperkeeper

• Ruth Linehan and Nathaniel Smith; puppetlabs
• back-end service engineers at puppetlabs
• service framework for long-running applications
• basis for all back-end services at puppetlabs
• service framework:
  • code generalization
  • component reuse
  • state management
  • lifecycle
  • dependencies
• why trapperkeeper?
  • influenced by clojure reloaded pattern
  • similar to component and jake
  • puppetlabs ships on-prem software
  • need something for users to configure, may not have any clojure experience
  • needs to be lightweight: don't want to ship jboss everywhere
• features
  • turn on and off services via config
  • multiple web apps on a single web server
  • unified logging and config
  • simple config
• existing services that can be used
  • config service: for parsing config files
  • web server service: easily add ring handler
  • nrepl service: for debugging
  • rpc server service: nathaniel wrote
• demo app: github -> trapperkeeper-demo
• anatomy of service
  • protocol: specifies the api contract that that service will have
  • can have any number of implementations of the contract
  • can choose between implementations at runtime
• defservice: like defining a protocol implementation, one big series of defs of fns: (init [this context] (let ...)))
  • handle dependencies in defservice by vector after service name: [[:ConfigService get-in-config] [:MeowService meow]]
  • lifecycle of the service: what happens when initialized, started, stopped
  • don't have to implement every part of the lifecycle
• config for the service: pulled from file
  • supports .json, .edn, .conf, .ini, .properties, .yaml
  • can specify single file or an entire directory on startup
  • they prefer .conf (HOCON)
  • have to use the config service to get the config values
  • bootstrap.cfg: the config file that controls which services get picked up and loaded into app
  • order is irrelevant: will be decided based on parsing of the dependencies
• context: way for service to store and access state locally not globally
• testing
  • should write code as plain clojure
  • pass in context/config as plain maps
  • trapperkeeper provides helper utilities for starting and stopping services via code
  • with-app-with-config macro: offers symbol to bind the app to, plus define config as a map, code will be executed with that app binding and that config
• there's a lein template for trapperkeeper that stubs out working application with web server + test suite + repl
• repl utils:
  • start, stop, inspect TK apps from the repl: (go); (stop)
  • don't need to restart whole jvm to see changes: (reset)
  • can print out the context: (:MeowService (context))
• trapperkeeper-rpc
  • macro for generating RPC versions of existing trapperkeeper protocols
  • supports https
  • defremoteservice
  • with web server on one jvm and core logic on a different one, can scale them independently; can keep web server up even while swapping out or starting/stopping the core logic server
  • future: rpc over ssl websockets (using message-pack in transit for data transmission); metrics, function retrying; load balancing

Domain-Specific Type Systems

• Nathan Sorenson, sparkfund
• you can type-check your dsls
• libraries are often examples of dsls: not necessarily macros involved, but have opinionated way of working within a domain
• many examples pulled from "How to Design Programs"
• domain represented as data, interpreted as information
• type structure: syntactic means of enforcing abstraction
• abstraction is a map to help a user navigate a domain
  • audience is important: would give different map to pedestrian than to bus driver
• can also think of abstraction as specification, as dictating what should be built or how many things should be built to be similar
• showing inception to programmers is like showing jaws to a shark
• fable: parent trap over complex analysis
• moral: types are not data structures
• static vs dynamic specs
  • static: types; things as they are at compile time; definitions and derivations
  • dynamic: things as they are at runtime; unit tests and integration tests; expressed as falsifiable conjectures
• types not always about enforcing correctness, so much as describing abstractions
• simon peyton jones: types are the UML of functional programming
• valuable habit: think of the types involved when designing functions
• spec-tacular: more structure for datomic schemas
  • from sparkfund
  • the type system they wanted for datomic
  • open source but not quite ready for public consumption just yet
  • datomic too flexible: attributes can be attached to any entity, relationships can happen between any two entities, no constraints
  • use specs to articulate the constraints
  • (defspec Lease [lesse :is-a Corp] [clauses :is-many String] [status :is-a Status])
  • (defenum Status ...)
  • wrote query language that's aware of the defined types
  • uses bi-directional type checking: github.com/takeoutweight/bidirectional
  • can write sensical error messages: Lease has no field 'lesee'
  • can pull type info from their type checker and feed it into core.typed and let core.typed check use of that data in other code (enforce types)
  • does handle recursive types
  • no polymorphism
• resources
  • practical foundations for programming languages: robert harper
  • types and programming languages: benjamin c pierce
  • study haskell or ocaml; they've had a lot of time to work through the problems of types and type theory
• they're using spec-tacular in production now, even using it to generate type docs that are useful for non-technical folks to refer to and discuss; but don't feel the code is at the point where other teams could pull it in and use it easily

ClojureScript Update

• David Nolen
• ambly: cljs compiled for iOS
• uses bonjour and webdav to target ios devices
• creator already has app in app store that was written entirely in clojurescript
• can connect to device and use repl to write directly on it (!)

Clojure Update

• Alex Miller
• clojure 1.7 is at 1.7.0-beta1 -> final release approaching
• transducers coming
• define a transducer as a set of operations on a sequence/stream
  • (def xf (comp (filter? odd) (map inc) (take 5)))
• then apply transducer to different streams
  • (into [] xf (range 1000))
  • (transduce xf + 0 (range 1000))
  • (sequence xf (range 1000))
• reader conditionals
  • portable code across clj platforms
  • new extension: .cljc
  • use to select out different expressions based on platform (clj vs cljs)
  • #?(:clj (java.util.Date.) :cljs (js/Date.))
  • can fall through the conditionals and emit nothing (not nil, but literally don't emit anything to be read by the reader)
• performance has also been a big focus
  • reduced class lookups for faster compile times
  • iterator-seq is now chunked
  • multimethod default value dispatch is now cached

The presentation’s done, the conference is over. I can start turning my attention back to writing.

But writing what? The book I’m outlining now is near-future science fiction, something that’s usually difficult to pin down. It’ll almost certainly be thought of as a prediction of what’s to come, instead of what it really is: an excuse to indulge some of my programming daydreams without moving too far away from the known and familiar.

So how do I balance the whimsy I want to put in there and the reality-grounding I know it’ll need? How do I gel a coherent story out of all the ideas bubbling around in my head for such a setting?

There’s no way to know except to write it, to set something down and then work with it until it reaches the shape I want. So I’m outlining, sketching characters and situations out, building up the scaffolding I’ll need before starting to write in earnest.

Here we go again.

I was happy to find that the first movie still holds up. I think part of why it works is because it is largely set in a 1999 that is frozen in time. It also helps that the basic structure of the movie is classic: naive youngster is shown a wider world, told of a prophecy where they will save the world, then begins to fulfill that prophecy.

The relative roles of the other main characters in the story bothered me this time, though, where they didn’t before. Morpheus is still amazing, but step back a bit and he’s one more wise black man guiding a white kid to a greatness that he can’t achieve. Trinity kicks ass, but squint and she’s a kung-fu wielding female who’s only there to fall in love with and support the male hero. For a film set in the future with a nominal theme of breaking down mental boundaries, these elements feel distinctly old and out of place.

The second and third movies are still complete failures, though. I enjoyed the second movie at the time, and remember hating the third one along with everyone else. But rewatching them showed me how much the two final films are really one film, and it’s not a good one.

I think a large part of the problem with the last two movies is that they violate the narrative expectations set by the first movie. The trilogy sequence setup in The Matrix was: a nobody becomes special (first film), then learns more about their specialness (second film), then pursues and achieves the mission for which they became special (third film).

But they skipped the second step, and padded the third step out over two movies. Mistake.

There’s a whole chunk of story missing, where we’d normally see Neo rescuing people from the Matrix – maybe getting frustrated that he can’t convince more people it’s fake? – and learning about what he can and can’t do. For example, he can’t do the Keymaker’s trick with doors: if we saw other people do it for an entire movie, but didn’t know how they were doing it, the Keymaker reveal would have a lot more punch.

Skipping that piece of the story prevents us from watching Neo learn and grow, and drops an opportunity to deepen the characterization of the crew of the Nebuchadnezzar along the way.

Given the current structure – and the large narrative hole in it – the last 2 movies should have been compressed into one. Cut out Zion, the attack on Zion subplot, the scenes with the last stand on the docks, etc. Stick to the thread of Neo and his crew chasing down the Keymaker and getting to the Source, then Neo taking a ship to the Machine City and ending the war.

Everything else – the machines attacking the docks, the sabotage of the ships by the Smith-infested human – can come to us as reports that Morpheus relays to the crew. This lets us keep the focus on Neo’s story, since we don’t have time to give the other plots and characters their due. Trying to squeeze them in – like the second and third films do – weakens Neo’s plot and doesn’t deliver any emotional heft. There’s simply not enough screen time.

The best course would have been to make the second bridging movie that’s missing, and then made the trimmed down third movie to wrap it up. Instead, we get one and a half good movies: the original Matrix, and the half a movie buried inside the latter two.

Not writing is starting to get under my skin.

The two weeks I was going to take off has turned into a month.

At first it was so I could catch up on all the house work I’ve been putting off: hardscaping the front yard in response to the drought, cleaning up the back yard, fixing both fences.

More recently I’ve held off writing because of a talk I’m giving at a conference for work next week. I knew starting on any novel would quickly occupy whatever spare head space I have, and I wanted to keep that free to work on the presentation.

Both good reasons. But it doesn’t stop me from missing it. When I was working on the novel, I felt like I had a purpose, a mission to fulfill. Without another novel to work on, I feel more relaxed, true, but also a little empty, a little directionless, a little smaller now that I don’t have any characters spouting dialogue into my head.

I’m trying to be patient, to keep notes on the books I’m debating working on, to stay focused on the other goals I’ve set for myself for this month. But I miss the work, and need to get back to it.

Background:

terribleminds.com/ramble/20…

whatever.scalzi.com/2015/04/0…

grrm.livejournal.com/420090.ht…

I think if these tactics had been used to ensure that only women got nominated for the Hugos this year, or that only PoC did, the Sad Puppies wouldn’t see that as right or fair.

I also think that they had – still have, I guess – a chance to act on their feelings of rejection in a positive way, by starting their own convention. No one could fault them if they started a Con that promoted the authors they prefer, nor would anyone be this mad if they’d launched their own awards at that Con.

First draft of novel’s done, writing vacation is winding down.

I’ve got an urge to start editing the novel now, to go back and fix the mistakes I know are there, and find the ones I don’t yet know about.

But I’m holding off. I’m not ready to treat it objectively yet. While printing it off for my wife to read I read a few pages, and liked it a little too much.

I need fresh eyes on it, eyes that haven’t seen anything but the words on the page, and so will notice if something’s missing or inconsistent or out of tune. Thus the printing run for my wife, so she can read it while soaking in the tub. And thus my new search for beta readers, for those willing to slog through the mess that is the first draft.

Wish me luck.