What Makes an Extraordinary Physician

06 May 2013

Day 9 (improved version)

I wrote this essay for my Art and Science of Medicine I course, and decided to share it here since the topic is general and reflects a broad sampling of what I’ve learned in the last couple years. Names of doctors and patients and some personal details are redacted to preserve privacy.

The duty of doctors is widely recognized as a profoundly high standard: every day, encountering disease, using conversation and examination to reveal details inscrutable to most people, and applying science and compassion to restore a patient to health. To “first, do no harm” is difficult enough when a patient lies mysteriously sick and the number of possible maladies is endless; to find the right treatment to cure or relieve them is an ordeal of another magnitude. In order for doctors to perform these duties, society places a remarkable amount of trust in them, allowing them to touch and manipulate the flesh of people and hear their most intimate secrets. We need doctors because we often encounter them at our weakest, when the natural functions we expect from our bodies suddenly fail either due to accident, the environment, or defect. To restore that function is an extraordinary power and an extraordinary privilege. To perform these duties extraordinarily is even more superlative. Superlative doctors inspire everyone in the practice of medicine to reach for that standard that seems so profoundly high. To find compassion for a patient that may not summon it naturally, to make time for an extra conversation with family after an exhausting shift, to always place the patient first in spite of potential conflicts with our own interests—these are deeds that encourage other physicians to improve their own qualities in the course of the profession.

In my experience, an extraordinary physician must have at least four core characteristics. Firstly, there is the ability to listen. It would seem that every doctor does this, but few do it perfectly. Is it necessarily their fault? Usually, no; clinics and hospitals are busy places: there are phones ringing, intercoms screeching, machines beeping, emails inbound, computers with forms that need to be filled. But to shut out all of that noise and distraction, and stare a patient in the eyes and listen to and feel every word that they are saying—that is something that every patient not only deserves, but often needs in order to become healthy again.

One physician I shadowed before coming to Sinai—in fact, the first one that inspired me to consider medicine as my calling—was an exceptional listener. As soon as Dr. K entered the room, the door closed, and he would lighten the mood with a few jokes, mostly at his own expense. (They were usually the same jokes over and over, but the different patients wouldn’t have known, and so they always worked.) He was a cardiologist; his patients almost universally had chronic heart failure, and he needed to change their daily habits to save their lives. That meant understanding everything about them: when they got up, what they ate, what made them stressed, how the medicines made them feel, what they expected from their new life post-cardiac incident. Talking about this, therefore, was the crux of his role in their healthcare team. I never saw him touch a phone or a computer in the presence of the patient. They trusted him because he really listened, eyes on their face, sharing himself with them, and only taking notes after the conversation was over.

He once told me, somewhat surprisingly to my ears, that if he had to choose between all the labs and tests that he could order and the ability to talk to them he would pick the personal history every single time. I heard his patients open up to him about everything—a father whose son who just committed suicide, a man struggling with cocaine, a mother who was frightened she would outlive her daughter. It was startling and humbling that I was even allowed in the room when it happened. All of these things mattered because without them, how could a physician realistically plan out the rest of these people’s lives? Would a battery of pharmaceuticals, taken regularly or not, make a dent in a patient’s sclerotic arteries in the face of the kind of life-altering stress that these patients had? A change involving the fabric of a patient’s life, like what they ate, thought, aspired to and valued, needed to happen one step at a time, and Dr. K perfectly understood and demonstrated this respect for the complexity of human life by listening to them.

TK-421 Jr. learns to ride his bike

Extraordinary physicians also need to be able to teach. A doctor encounters his patient for a ridiculously small fraction of their natural lives, and yet often aspires to fundamentally change the 99% of the lifetime that he will not witness. This would be impossible unless the patient can take away something more than a few prescription slips’ worth of information, particularly when the reason they are sick is a molecular defect that exerts its effects through multiple layers of biochemistry, physiology, and pathophysiology and requires their continual effort to be mitigated. Explaining all of this to them and translating it into actionable information they can apply—what to do if X, why do I sometimes feel Y—is no simple matter.

There are many doctors that I could commend for being amazing educators, both from before I came to Sinai and within this hospital. I want to recognize an awesome allergist-immunologist, Dr. L, whom I decided to shadow after seeing his remarkable lectures for my college’s physiology class figuring that he would likely be just as funny, kind, and insightful with his patients. He was all of the above, and had a particularly grueling body of knowledge to transfer to them, as many of them had complex immunological conditions that could not possibly be described in a few succinct sentences. The pen came out often, scribbling figures on papers that he would show them, diagramming what was wrong and how it could be fixed. I saw him draw pancreatic ducts and uric acid crystals, antibodies and lymphatic systems. The phrase “it’s too complicated to explain” never came out of his mouth, no matter whether the patient was six or sixty. Somebody with unpredictable bouts of debilitating pain comes to a doctor because they want answers as well as relief, and he took that duty incredibly seriously, offering his personal cell number to every patient in case they had lingering doubts or concerns.

Baby care lesson by Darth Vader

Sinai educators are almost too numerous to praise, although I will do my best. Firstly my small group leader Dr. A, who walks that beautiful line between tough love and caring approachability like no other teacher here, steering discussions back on topic, unafraid of giving us the no-nonsense raw deal on any topic using episodes from her personal experiences. How do you communicate the “soft skills” of being a doctor to a room full of bright-eyed medical students that have absolutely no experience and barely any practical medical knowledge? It’s not a task that I can even comprehend at this point, so major kudos to her for charging into it with such fearlessness. Dr. O, for putting a whole curriculum together on this very subject. Dr. I, who helped me educate a Spanish-speaking EHHOP patient on dietary recommendations despite my formidable level zero Spanish skills. Drs. Z and T, for always accommodating two medical students whenever my LCE patient came back for appointments, afterward offering the full debrief on what happened throughout the encounter.

One more Sinai doctor is memorable because he showed me another important quality of an extraordinary physician: humility in the face of uncertainty. Perhaps just as important in educating the patient from the endless textbooks and journals that support modern medicine is the admission that sometimes the answer is unknown. […patient history redacted…] Our preceptor for this course, Dr. G, has decades of experience with this disease and yet must admit to each and every patient diagnosed that the etiology is not clear and the treatments are supportive at best. It is a difficult topic to broach, one that requires a tricky combination of empathy and humbleness, but he does it without hesitation or diminishing the patient’s confidence in his plan of care. Nobody likes to say “I don’t know,” and therefore it is an onerous task (even though I am getting plenty of practice as a first year medical student). Being able to do it well, every day, with people that put their life in your hands, is extraordinary.

The fourth quality of an extraordinary physician is the willingness to make sacrifices for the good of your patient, and to do so gracefully under pressure. Sometimes you have to let them have your dignity, your patience, or both. On our hospital-based rounds, Dr. A had the unenviable task of introducing us to bedridden patients, some of whom made not-quite-appropriate comments about her appearance as a female doctor. The behavior, while clearly unacceptable, did not faze her in the slightest—she placed the rapport she was trying to develop between us students and the patient first, and the encounter turned out to be educational. You never know when you might need to treat a patient that riles you in every possible way, and the ability to control this is a hallmark of medical professionalism. The doctors at my clinical site, a private pediatrics office, have families that are less cooperative than others, but they receive the same attention and care regardless. I watched my preceptor, Dr. R, take a call from a family whose son was having severe psychological issues, even though they had moved out of state and hadn’t seen her in years. Their concern was genuine, and therefore so was hers. In her clinic the littlest patients often resisted most medical care, and therefore patience and specialized communication skills are vital. A triumvirate of disagreeable conditions—the screaming, hostile patient, the anxious and frazzled parents, the need to administer shots or other painful meds—never deterred her from her goal. She made it look routine, because it was.

Therefore, what makes an extraordinary physician is not just one talent, but a balance of orthogonal qualities: to listen, to educate, to be humble, and to gracefully make sacrifices. On a given day, a physician may be called upon to exercise any or all of these talents, and when they do it well, they become extraordinary. However, in order to maintain the trust that physicians are given by our society, doctors must above all be honest and worthy of that trust. That means being honest to the patient, of course, but honesty toward colleagues and toward themselves is just as critical. Medicine is not practiced in a vacuum; it requires the support of the people working in its institutions for one another, and the support of the community that it serves. On the day when the ethical dilemmas we discuss in class become real situations, we will all be tested. We will be tested when we see our colleagues fall short of their responsibilities, and we will be tested when our supervisors ask us to do things we find questionable. What is most extraordinary about medicine—besides its individual physicians—is that we are given the privilege to serve the people around us in such a valuable way with our lives, and it is this privilege that must be respected first before we even attempt to learn the talents that will make us extraordinary as individuals.

The Lego Doctor

What New York Does Right: Street Layout

31 March 2013

midtown nyc

I’ll admit it. Coming back to New York after a week on the coast of South Florida in March can be a little bit of a downer. According to my roommate’s trusty Game of Thrones weather app, on a three hour flight I went from Pentos to Winterfell. Sadly, beaches here will never compare with Miami’s South Beach, and neither palm trees nor mangroves will ever bloom in Central Park.

But with my feet firmly on New York concrete and a $2 slice of excellent pizza in my hand, the regret faded away. I’ve driven from New York to Seattle and back, and been all over the east and west coast (thanks to med school interviews). Despite what I’ve seen, there is a reason so many Americans, particularly young people, want to live in the city. New York gets so many things right, and these things are so entwined with the culture of this place that even if I leave for a bit, I know that they will be here for me when I get back, and that is the best comfort a home can offer. Here lies what New Yorkers can be proud to call their own, from birth to death. I’ll tackle one topic per post. Let’s start with what binds the whole city together: the streets.

History

By the miraculous circumstance of three historical forces: the Commissioner’s Plan of 1811, the design of Central Park by Frederick Law Olmstead, and the battle between Robert Moses and Jane Jacobs over the adaptation of the city to the automobile, Manhattan has evolved into one of the most fantastic urban landscapes to be traversed by cars, pedestrians, and bikers alike. Nowhere else in America will you find two-thousand-odd blocks of dense cityscape unbroken by the scars of massive highways.

Taylor map, Galt and Hoy, 1879 The Taylor map, 1879.1 The street layout seen here is practically identical to what we have today in 2013. The even “grid” starting at Houston is noticeable.

The result is astonishing in scope and is, at once, delightfully rational and historical. Above Houston St. (the effective extent of the city before the Commissioner’s plan), a mostly regular grid of narrow streets and wide avenues reigns, where one can think of a location—say, “Fifty-first and eighth”2—and immediately grasp the distance to it as well as the most likely routes. Below Houston St., a cobweb of competing street orientations must be learned, but therein lies sparkling neighborhoods and cultures that can be discovered and rediscovered over a lifetime. Because the grid expanded north as the city grew3 (to 21st St by 1834, and 57th St by 1844) and the central business district shifted from Lower Manhattan to Midtown as the main industries of the city evolved, the centrality of the street grid also coincides with the area of greatest commercial density, unlike cities with less room to grow (e.g., Boston).

Comparison

You might protest: “Oh, but many other great American cities use a grid layout.” None get it quite as perfect, however. Let’s use the following comparison, taken with all maps at the same scale, to start the discussion:

comparison of street grids Imagery from Google Maps.

Two questions as should arise as you compare these: Which is the easiest for the motorist and pedestrian to navigate? Which creates the most pleasing city environment? A number of things can disrupt a street grid plan:

  1. Too many irregularities. Philadelphia was one of the first cities in North America to use a grid system, but it did not enforce it as rigorously as New York; therefore, every block encountered is a potential surprise, and the order of the east-west street names must be memorized. This applies double to Toronto, because none of the streets are named predictably, and exceptions abound.

  2. No contrast between streets and avenues. DC was more rigorous in sticking to the grid, but the blocks are too squarish. The result is there is no rhythm between commercial “edges” facing the avenues and residential “meat” on the interior of each block, as there is in New York, allowing better mixed use of every block—even in high density zones, many blocks can accomodate smaller townhouses and walk-up buildings within their interior4. Having uniformly wide roads also fails to concentrate traffic along one axis. Because avenues are wider and farther apart in the Commissioner’s Plan, north-south travel is more busy but also more efficient (this also happens to be the long axis of Manhattan, and thus the one more traversed). This avenue layout creates traffic, visibility, and natural congregation points within each neighborhood for street-level commerce. Such traffic can be tolerated only because there aren’t…

  3. Too many two-way roads. Toronto is a terrible offender here. While motorists might think that two-way roads simplify route calculations, the reality is that they completely snarl traffic during high demand periods. Turning is the most expensive operation for traffic at an intersection, as cars have to cross through either other cars (with left turns) and possibly pedestrians in the crosswalk to clear the box. Until they can do that, cars in the lane behind them cannot move. At an intersection of two-way roads where all turns and pedestrian crossings are allowed, four lanes per road are going to be jammed for every green light without some special accomodation: separate left turn/right turn lanes and signals, which cost space and time per signal change, or randomly disallowing left or right turns, which destroys any claimed advantages in navigability.

    In New York, the average intersection is two unidirectional streets, causing no crossing of car paths and only one possible turning direction. Combined with the Barnes Dance to clear crowded intersections of pedestrians, and synchronized green lights (called a “green wave”) down major arteries, one can typically drive 50 blocks north or south on a large artery without hitting a red light. There is also a nearly regular alternation of north/south and east/west travel, with changes in directionality made very rarely throughout the grid. The result: a graceful compromise between pedestrian and automobile friendliness.

    third avenue Get on 3rd Avenue in the center lane and you will practically fly uptown. Taken from Google Street View.

  4. Streetcars and elevated trains. Driving among streetcars that occupy center lanes shared with cars is supremely frustrating. Not only is there a mad dash to slip around them between intersections, but every time the streetcars stop, all other lanes must stop as well to avoid harming entering and discharged passengers. This happens to be precisely the system that Toronto employs in its central business district. Elevated trains remove the traffic issues, but pollute thunderous noise, block out sunlight at the street level, and create an intimidating row of pillars for motorists to dodge. Sorry Chicago, your ‘L’ train may hold some quaint charm and makes for a cool scene in The Dark Knight, but Manhattan made life so much better by sinking most trains underground.

There are two more things that really kill street grids. They are probably best explained pictorially, since they would have been too unfair to show in the tiny representative samples I chose above. Here’s the first offender, seen across the street from where I lived in Toronto.

Toronto rail line Hey, it’s the CN tower over there! Should be an enjoyable five-block walk from here. Oh, wait. Taken from Google Street View.

This is not an angle of the CN tower often advertised in Toronto tourism material. Railway lines through a downtown core? Unsightly, loud, and difficult to cross, they are an impenetrable wall for pedestrians and bikers, which is a shame, because those brand new lakefront condos are aching to be connected to the rest of the city. Once railways are buried, as in the Park Avenue tunnel or the East River tunnels under 32nd and 33rd St. (both operational and electrified by 1910!), the street level can be pleasant again5. Here’s another neighborhood destroyer, in Philadelphia:

Vine St. expressway Hey, it’s downtown Philadelphia over there! Should be an enjoyable five-block walk from here. Oh, wait. Taken from Google Street View.

Expressways can likewise destroy the walkability and street atmosphere of a neighborhood. Take for instance the South Bronx, fully wrapped and skewered by interstates. Fortunately, most of Manhattan avoided a similar fate thanks to Jane Jacobs and other militant conservationists, who successfully fought off a Lower Manhattan Expressway that would have destroyed Soho and Little Italy (two of New York’s most vibrant neighborhoods today) and a Mid-Manhattan Expressway which, even more insanely, would have leveled some of the most expensive commercial real estate in the world and driven highways through the middle of skyscrapers6.

The division and congestion created by a highway is expensive and difficult to remedy. Just look at Boston, which opened a Central Artery straight through downtown in the 1950’s only to see it clog up with cars and decrease access to historical and cultural neighborhoods like the North End from downtown. The problem was recognized by 1982, with construction on the infamous “Big Dig” to bury the expressway underground taking sixteen years and costing an estimated $22 billion, several investigations, and one death7. The result is obviously much more pedestrian friendly, and the surrounding neighborhoods may eventually heal together with enough creative landscaping, but it came at an enormous price.

Big dig This used to be a expressway in Boston. Moving it underground was the most expensive US highway project to date. Taken from Google Street View.

This is not to say that highways are completely unimportant; Robert Moses dutifully ensured that Manhattan had limited-access north/south arteries along the East and Hudson rivers. And they can be used, for instance, to zip from City Hall to the Apollo in 20 minutes during off-peak hours, a trip spanning most of the island. But their disruption to pedestrians is constrained to the margins, leaving the city inbetween magnificently unblemished. From Avenue D to West End Ave, a pedestrian will be hard-pressed to find more than six lanes of traffic between him and the other end of a crosswalk.

New york grid Blocks on blocks on blocks. Imagery from Google Earth.

Continuity of neighborhoods

Here’s a maxim that could probably be proven using New York as a test case: the natural limit of neighborhood continuity across pavement is somewhere around six lanes. It’s a distance where you can still recognize your friend in front of a store on the other side of the avenue and jog over to her when traffic slows. Unlike, say, the twelve-lane death trap that some call Queens Boulevard8, you could get comfortable with your grandmother crossing a Manhattan avenue by herself. Anything just over six, e.g., by replacing two lanes with streetcar tracks on Spadina Ave in Toronto, and you start feeling noticeably less happy about crossing it every day. Let’s recall that six lanes are more than sufficient for fast travel in one direction—refer to that picture of 3rd Avenue above, which has five, giving you one center lane, a passing lane on either side, and outer lanes, which can accommodate the cars turning, parking, and performing general shenanigans without slowing down thru traffic.

On the other hand, there is also a reasonable minimum for street width. New York’s is generally a central quasi-double lane with parking on both sides of the street. Observe the following comparison between typical streets in New York and Philadelphia.

Street width woes E 70th in NYC vs. S 12th in Philly. Both images courtesy of Google Street View.

In New York, residents can park on both sides of most streets, and traffic flows down the large center lane. If somebody double-parks to unload a passenger or perform some shenanigans, as this minivan has done in the picture (it is a fact of American city traffic that vehicles will selfishly stop to occupy any point along a streetside), no matter: there is still enough room for cars to squeeze past, as the yellow taxi is doing. In Philadelphia, one whole side of the street is typically wasted. Why? It’s the awkward width of the street: if there were parked cars on both sides, somebody stopping to unload or perform shenanigans, as these construction guys are doing, would be impeding all movement. The center lane of this street has insufficient “wiggle room” and thus a whole edge must be lost to bare curbside that becomes the de facto “shenanigans” lane. Philly should try to repurpose some of this awkward space to make more bike lanes. (Also, what’s the deal with the abandoned trolley tracks in the center of the street, perfectly positioned to provoke anxiety in motorists and tip over passing cyclists?)

There is no way the Commissioner’s Plan could have predicted that 60 feet would be the perfect width for three cars with squeeze room for one more and two sidewalks on either side, but it works so well that it makes you wonder if New York streets didn’t unintentionally influence the standard 12 foot lane width for automobile traffic or the width of cars in general. Other cities that contend with comparable urban density in America can only wish they had evolved with such a fortuitous configuration9. As the outer boroughs of New York largely perpetuated the spread of the rectangular block style and street widths of Manhattan, the city has a remarkable uniformity to its street texture, from Coney Island Ave all the way to E 233rd St in the Bronx, with each block theoretically capable of the same eventual mixed residential and commercial density as a block in Midtown.

Street grid uniformity From left to right, the Bronx, Queens, Manhattan, and Brooklyn. Each tile is at the same scale. Imagery from Google Maps.

Streets aren’t just for cars

So far I’ve argued mostly on the premise that streets should be built for pedestrians as well as for cars. To anyone in New York, where just as many walk and use public transit to get to work as drive, this seems blatantly obvious. Many studies indicate, as should common sense, that supporting only a car-commuter culture will negatively affect public health. Having trudged around the dismal central business districts of Worcester, New Haven, Hartford, Springfield, Sioux Falls, Detroit and so many other American cities that seem to have given up on attracting pedestrians, it’s clear that any street life that would inspire young workers and businesses to come and explore has been lost. All of these places seem to have the same vacant storefronts surrounded by massive buildings and parking structures. How free can one feel in a city where it seems safer to stay in the car? What’s the psychological effect of being chronically tethered to a two-ton hunk of aluminum and steel?

Let’s not forget the bicyclists, either. Say what you will about Mayor Bloomberg, but watching this map grow and exploring its fringes over the past decade has transformed how I feel about and move around the city. You can now bike from Westchester County to Jacob Riis Park mostly along reserved bike lanes, some of them separated from auto traffic10. I’m planning some posts about some of the nice longer trips that can be made into the Palisades, the South County Trail, and others. The bike lane effort has been a stunning accomplishment for the DOT and I only hope that they’re here to stay after the mayor leaves.

Coming next

Alright, I’ve said my piece and that wraps this first part of a (possibly never-ending) series on things New York gets right. Stay tuned for articles planned on, but not limited to: pizza, subways, taxis, skylines, street food, bagels and lox… need I go on?

This post used the word “shenanigans” a total of five times.

Did I get something wrong? You can flame me on Reddit.

  1. Codex 99, a bottomless blog on the history of the visual arts and graphic design, has a fantastic review on the evolution of three-dimensional cartography depicting New York. 

  2. This is the correct way to say a Manhattan address, with street name before avenue. If you’re cabbing to the East Village, don’t mess this up. It’s in the rules. Technically only two blocks exist where this could be ambiguous, but still. If you clicked that link, you may have noticed Google gets it wrong. Google is not a New Yorker. 

  3. The Times hosts an interactive map showing the order in which Manhattan streets opened. 

  4. In crowded commercial districts like Midtown, you might find this situation inverted—sometimes with goofy results—but the streetfront is usually so continuous that it can be hard to notice! 

  5. Boston had a rail line through the South End and Back Bay sunk into a tunnel in preparation for the construction of a new highway, but resistance was so fierce following the Central artery disaster that it was abandoned. Decades later, it was repurposed into one of a neat linear park that connects the surrounding neighborhoods. 

  6. The idea was to facilitate movement between Long Island and Jersey, although why this had to occur through the most crowded and expensive part of Manhattan was an objection that thankfully was never overcome. The only expressway of this category, the Trans-Manhattan Expressway, has become a continually-clogged pinchpoint for interstate trucking and travellers, with less than ideal consequences for local residents. The Mid-Manhattan Expressway project did inspire some impressive “futurism”-laden design illustrations, though. 

  7. 12 tons of concrete ceiling panels in a tunnel collapsed, crushing a car and killing one of its passengers. This resulted in criminal charges and a review of Boston’s entire highway system

  8. I’m not kidding about it being dangerous. In the 90’s an average of 10 people were killed per year trying to cross Queens Boulevard, earning it the name The Boulevard of Death and the installation of cheerful signs proclaiming A pedestrian was killed crossing here

  9. A rather fun counterpoint to this article is a critique of the Commissioner’s Plan apparently written in the 1930’s, which marveled that a city of 85,000 could have laid itself out in preparation for millions of residents, but proposes that more avenues and diagonal arteries should have been included, and less cross streets. From the standpoint of 2013, I’m not sure I agree. Broadway creates confusing and crowded 6-way intersections wherever it crosses an avenue, and its controversial replacement with a pedestrian plaza in Times Square has actually improved safety and traffic in that area. 

  10. They aren’t always respected: as this short video tracing the bike lane on W 106th St. shows, many cars and UPS men will use the lane as an excuse for more shenanigans. But the occasional intrusions aside, is this not a glorious way to move about a city? 

Hacker News Sidebar: An Extension for Google Chrome

tldr: I’ve fixed up the Hacker News Sidebar Chrome extension. It shows comment threads from HN in a handy tab next to any other pages that you visit. Browse the source code on GitHub, and install the extension from the Chrome Store.

screenshot

Like many other web developers, I use Google Chrome as my primary web browser because it is incredibly fast, the UI gets out of the way, and the web development tools are superlative. That might change some day—Firefox is making a comeback with regard to all of these things, and I like using it as well, just not quite as much to make me switch back yet.

One surprisingly nice thing about Chrome is that it is really easy to develop extensions, particularly if you are a web developer. Generally all that’s required are a few JavaScript files and optionally some HTML and CSS and you could do anything from bookmark syncing, page syncing across devices, or elimination of those pesky tracking scripts. Admittedly there are limitations: you can only modify the browser UI in a few specific ways, like adding buttons or icons next to the Omnibox that pop out content, or items to the context menu.

Chrome extensions function much like an offline HTML5 app, running in the background as a hidden tab. Besides all the typical features of the DOM, like the XHR object for asynchronously communicating with webservers, they can access special Javascript APIs (under the chrome global object) for opening and manipulating tabs, history, windows, and bookmarks. They can also selectively insert Javascript into webpages, like the Greasemonkey scripts or Userscripts that are popular for enhancing the functionality of certain websites. To securely communicate between code running in the user’s browser tabs and your extension’s hidden background tab, you use a messaging API. All the special permissions your extension needs must be specified in a manifest.json file.

As you may have guessed by the links at the end of each post on this blog, I read HN quite a bit—enough so that I’d rather just direct any discussion to that forum, rather than try to police my own comments database. Meta-discussions on link-aggregating websites (e.g. HN, reddit) are often more interesting than the original content. Once upon a time I used this Chrome extension to reveal these HN threads automatically: it would pop out a side tab on any page that had been posted to HN. Whenever the orange tab happily emerged, a quick scan of the sidebar would show what HN thought and how many upvotes it had, sometimes significantly changing my impression of the original page.

Unfortunately, that extension has fallen into disrepair. HN has switched to HTTPS and now disallows framing of their site via the X-Frame-Options: DENY header 1, causing the tab to be blank. Additionally, the Chrome extension packaging scheme has changed to tighten security policies, so barring some serious updates, this extension will stop working in September 2013. In order to get this extension running again, I had to rewrite much of it. Let’s break down the source code.

All Chrome extensions begin with a manifest.json file. The first half of this file says that I need the xhr_handler.js script to run in my background tab, and I need to insert hn.css, jquery.js, and script.js into all pages (as specified by matches). Note that through some scoping magic that Chrome does, these inserted scripts will not interfere with any global variables or functions created by their host pages, but they will still be able to affect the DOM. Finally, there is some metadata for the Chrome Store in the second half, along with permissions, which enumerates the domains that I want my background script to be able to access.

{
   "manifest_version": 2,
   "background": {"scripts": ["xhr_handler.js"]},
   "content_scripts": [ {
      "css": [ "hn.css" ],
      "js": [ "jquery.js", "script.js" ],
      "matches": [ "http://*/*", "https://*/*" ]
   } ],
   "icons": { "48": "icon-48.png",
             "128": "icon-128.png" },
   "description": "Hacker News integration for Chrome",
   "name": "Hacker News Sidebar",
   "permissions": [ "http://api.thriftdb.com/api.hnsearch.com/items/*", "https://news.ycombinator.com/*" ],
   "version": "1.0.8"
}

Let’s look at the script.js file next, which will be inserted into every page I visit. For the most part, this looks like a run of the mill jQuery script. The first atypical tidbit is this:

var port = chrome.extension.connect({}),
   callbacks = [];
// ...[snip]...

port.onMessage.addListener(function(msg) {
  callbacks[msg.id](msg.text);
  delete callbacks[msg.id];
});

function doXHR(params, callback) {
  params.id = callbacks.push(callback) - 1;
  port.postMessage(params);
}

This appears to be a wrapper for performing XHR (also called AJAX requests), but why all the bother when I would typically just use $.ajax? Well, despite their special scope, content scripts still execute with all the cross domain restrictions that the host page has; they can’t perform AJAX requests outside the host, port, and protocol the host page was served rom. That’s a problem because we need to pull outside content from both the HNSearch API and HN itself. So, to work around this, the code packages the parameters for the XHR into a message that is sent via the Message Passing API under the special chrome global object to the background script, xhr_handler.js, where it can executed with the correct permissions.

What happens there? Let’s see:

function xhrCall(url, port, id) {
  var xhr = new XMLHttpRequest();
  xhr.open("GET", url, true);
  xhr.onreadystatechange = function() {
    if (xhr.readyState == 4) {
      port.postMessage({id: id, text: xhr.responseText});
    }
  };
  xhr.send();
}

chrome.extension.onConnect.addListener(function(port){
  port.onMessage.addListener(function(request) {
    xhrCall(request.url, port, request.id);
  });
});

Pretty much all it does is actually perform the XHR request and send the responseText (the raw text content of the response) right back. Note the use on both ends of the port.postMessage method to send data and the addListener methods to create event handlers that receive data. This is very reminiscent of the HTML5 Web Workers API.

So now that we can perform XHR to the necessary outside sites, back in script.js, we query the HNSearch API filtering by the URL of the current page:

var exclude = /\.(xml|txt|jpg|png|avi|mp3|pdf|mpg)$/;
// ...[snip]...
var curPath = window.location.href;
if (exclude.test(curPath)) { return; }

var queryURL = "https://api.thriftdb.com/api.hnsearch.com/items/_search?filter[fields][url][]=" \
  + encodeURIComponent(curPath);
doXHR({'action': 'get', 'url': queryURL}, function(response) {
  // JSON.parse will not evaluate any malicious JavaScript embedded into JSON
  var data = JSON.parse(response);

  // No results, maybe it's too new
  if (data.results.length < 1) {
    doXHR({'action':'get','url': HN_BASE + "newest"}, function(response) {
      searchNewestHN(response);
    });
    return;
  }

  // If there is a result, create the orange tab and panel
  var foundItem = data.results[0].item;
  createPanel(HN_BASE + 'item?id=' + foundItem.id);
});

and if we find results, we kick off creation of the orange tab with createPanel; if not, we give the newest HN stories a try with searchNewestHN, in case this is a story that just made it to the front page. searchNewestHN works by parsing the HTML of the front page and finding any anchors that link to this page.

function searchNewestHN(html) {
  var titleAnchor = $('a[href=\'' + window.location.href.replace(/'/g, "\\'") + '\']', html),
    linkAnchor = titleAnchor.parent().parent().next().find('a').get(1);
  if (linkAnchor) {
    createPanel(HN_BASE + $(linkAnchor).attr('href'));
  }
}

If it does find it, it too calls createPanel. The createPanel function is pretty dry with a lot of dry DOM construction, but the fun part is when we actually need to put the HN comment thread into the DOM:

function createPanel(HNurl) {
  // ...[snip]...
  var HNembed = $("<div />").attr({'id' : 'HNembed'});
  var HNsite = $("<iframe />").attr({'id' : 'HNsite', 'src' : 'about: blank'});
  // ...[snip]...
  HNembed.append(HNsite);
  HNembed.hide();

  $('body').append(HNtab);
  $('body').append(HNembed);

  doXHR({'action': 'get', 'url': HNurl}, function(response) {
    var doc = HNsite.get(0).contentDocument;
    response = response.replace(/<head>/, '<head><base target="_blank" href="'+HN_BASE+'"/>');
    doc.open();
    doc.write(response);
    doc.close();
  });
}

This is not your typical use of an <iframe>: instead of setting the src attribute to the comment thread URL, which we can’t do because HN forbids framing, we have to leave it as about:blank, fetch the contents of the comment thread page via XHR, and then write to the <iframe>’s document object extracted with the contentDocument property. We can write the same HTML that HN serves, with one modification: by adding a <base> element to the <head> of this document, we can ensure that all the links within are followed relative to HN, and not the URL of the host site. The .open() and .write() calls here are indeed those yucky methods you have been taught to never use since DOM scripting became practical, and in general you really shouldn’t, but here it is fair game because the document we are writing to is completely empty.

So, after all of that lovely bit-pushing, we get a collapsible sidebar with an HN comment thread in it! To try it out yourself, install the extension from the Chrome Store, or clone the source code and load the directory into Chrome as an unpacked extension.

screenshot

One note on a privacy weakness of this extension, which you may have noticed if you were following the code above: the extension sends all the URLs that you visit to https://api.thriftdb.com/. Obviously, that isn’t going to be acceptable for some users, but there’s no way to look up Hacker News threads for all the URLs you visit without sending those URLs somewhere. The Hacker News’d extension, which does something similar but doesn’t add the comment thread to the browser window, makes a strange attempt to mitigate privacy concerns by MD5ing URLs, but then it requires a special server-side component that is constantly scraping the front page and shoving the MD5s into a MongoDB on Heroku, and besides that MongoDB being necessarily more incomplete than the official API, it doesn’t really solve the privacy problem anyway. So, if you want to use this on your primary browser you basically have to trust the ThriftDB folks with your browsing history (hey, at least it’s being sent over SSL).

  1. Presumably, HN uses the relatively new X-Frame-Options header to prevent people from embedding HN comment threads directly into their own site, either because they don’t want them surrounded by other people’s content or because they want to block clickjacking schemes for tricking users into upvoting items. Unfortunately this also precludes the simplest method of accomplishing what the extension used to do—loading the comment thread into an <iframe> by setting its src attribute. I suppose I could have also tried to get around this restriction by modifying HTTP headers as they are received, but the way I did it seemed easier.