<?xml version="1.0" encoding="UTF-8"?>
<feed xml:lang="en-US" xmlns="http://www.w3.org/2005/Atom">
  <title>Articles — mutelight.org</title>
  <id>tag:mutelight.org,2009:/articles</id>
  <updated>2013-06-05T07:18:13-07:00</updated>
  <link rel="self" type="application/atom+xml" href="https://mutelight.org/articles.atom"></link>
  <link rel="alternate" type="text/html" href="https://mutelight.org"></link>
  <entry>
    <title>Discriminating Input</title>
    <content type="html"><![CDATA[<p>While designing our <a href="https://devcenter.heroku.com/articles/platform-api-reference">V3 platform API</a>, we made the decision to make the formatting in our requests and responses as symmetric as possible. Although common for an API to return JSON, it&rsquo;s not quite as common to take it as input, but is our recommended usage for all incoming <code>PATCH</code>/<code>POST</code>/<code>PUT</code> requests.</p>

<p>Largely reasons largely for developer convenience, we decided to allow fall back to form-encoded parameters as well (for the time being at least), so we put together a helper method that allows us to handle these in a generic fashion. It looks something like this:</p>

<pre><code class="language-ruby">class API &lt; Sinatra::Base
  post &quot;/resources&quot; do
    params = parse_params
    Resource.create(name: params[:name])
    201
  end

  private

  def parse_params
    if request.content_type == &quot;application/json&quot;
      indifferent_params(MultiJson.decode(request.body.read))
      request.body.rewind
    else
      params
    end
  end
end
</code></pre>

<p>By specifying <code>Content-Type: application/json</code>, JSON-encoded data can be sent to and read by the API:</p>

<pre><code>curl -X POST https://api.example.com/resources \
    -d '{&quot;name&quot;:&quot;my-resource&quot;}' -H &quot;Content-Type: application/json&quot;
</code></pre>

<p>The more traditional method for encoding POSTs is to use the <code>application/x-www-form-urlencoded</code> MIME type which looks like <code>company=heroku&amp;num_founders=3</code> and is sent in directly as part of the request body. Rack will decode form-encoded bodies by default and add them to the <code>params</code> hash, so our API easily falls back to this:</p>

<pre><code>curl -X POST https://api.example.com/resources -d &quot;name=my-resource&quot;
</code></pre>

<p>(Note that Curl will send <code>Content-Type: application/x-www-form-urlencoded</code> by default.)</p>

<p>Good so far, but a side-effect that we hadn&rsquo;t intended is that our API will also read standard query parameters:</p>

<pre><code>curl -X POST https://api.example.com/resources?name=my-resource
</code></pre>

<p>On closer examination of the Rack source code, it&rsquo;s easy to see that Rack is trying to simplify its users lives by blending all incoming parameters into one giant input hash:</p>

<pre><code class="language-ruby">def params
  @params ||= self.GET.merge(self.POST)
rescue EOFError
  self.GET.dup
end
</code></pre>

<p>While not a problem per se, this does widen the available options for use of API to cases beyond what we considered to be reasonable. We cringed to think about seeing technically correct, but somewhat indiscriminate usage examples:</p>

<pre><code>curl -X POST https://api.heroku.com/apps?region=eu -d &quot;name=my-app&quot;
</code></pre>

<p>By re-implementing the helper above to ignore <code>params</code>, the catch-all set of parameters, and instead use <code>request.POST</code>, which contains only form-encoded input, we an exclude query input:</p>

<pre><code class="language-ruby">  def parse_params
    if request.content_type == &quot;application/json&quot;
      indifferent_params(MultiJson.decode(request.body.read))
      request.body.rewind
    elsif request.form_data?
      indifferent_params(request.POST)
    else
      {}
    end
  end
</code></pre>

<h2 id="section-0" class="link"><a href="#section-0">rack-test</a></h2>

<p>As an addendum, it&rsquo;s worth mentioning that <code>rack-test</code> also sends <code>application/x-www-form-urlencoded</code> by default (and always will unless you explicitly override <code>Content-Type</code> to a non-nil value), and that&rsquo;s what&rsquo;s going on when you do this:</p>

<pre><code class="language-ruby">it &quot;creates a resource&quot; do
  post &quot;/resources&quot;, name: &quot;my-resource&quot;
end
</code></pre>

<p>We found that it was worthwhile writing our tests to check the primary input path foremost, so most look closer to the following:</p>

<pre><code class="language-ruby">it &quot;creates a resource&quot; do
  header &quot;Content-Type&quot;, &quot;application/json&quot;
  post &quot;/resources&quot;, MultiJson.encode({ name: &quot;my-resource&quot; })
end
</code></pre>
]]></content>
    <published>2013-06-05T07:18:13-07:00</published>
    <updated>2013-06-05T07:18:13-07:00</updated>
    <link href="https://mutelight.orgparams"></link>
    <id>tag:mutelight.org,2013-06-05:/params</id>
    <author>
      <name>Brandur Leach</name>
      <uri>https://mutelight.org</uri>
    </author>
  </entry>
  <entry>
    <title>Testing Sinatra With Rack-test</title>
    <content type="html"><![CDATA[<p>A common problem when starting out with Sinatra and trying to exercise what you&rsquo;ve built with <code>rack-test</code> is that by default, Sinatra will swallow your errors and spit them out as a big HTML page in the response body. Trying to debug your tests by inspecting an HTML backtrace from <code>last_response.body</code> is a harrowing experience (take it from someone who&rsquo;s tried).</p>

<p>The solution is to tell Sinatra to raise errors back to you instead of burying them in HTML. Here&rsquo;s the proper combination of options to accomplish that:</p>

<pre><code class="language-ruby">set :raise_errors, true
set :show_exceptions, false
</code></pre>

<p>Here&rsquo;s a more complete example:</p>

<pre><code class="language-ruby"># app.rb
class App &lt; Sinatra::Base
  configure do
    set :raise_errors, true
    set :show_exceptions, false
  end

  get &quot;/&quot; do
    raise &quot;error!&quot;
  end
end
</code></pre>

<pre><code class="language-ruby"># app_test.rb
describe App do
  include Rack::Test::Methods

  it &quot;shows an error&quot; do
    get &quot;/&quot;
  end
end
</code></pre>
]]></content>
    <published>2012-12-29T11:32:30-08:00</published>
    <updated>2012-12-29T11:32:30-08:00</updated>
    <link href="https://mutelight.orgsinatra-rack-test"></link>
    <id>tag:mutelight.org,2012-12-29:/sinatra-rack-test</id>
    <author>
      <name>Brandur Leach</name>
      <uri>https://mutelight.org</uri>
    </author>
  </entry>
  <entry>
    <title>The Asset Pipeline in Sinatra</title>
    <content type="html"><![CDATA[<p>The spectacular asset pipeline that shipped with Rails 3.1 is an easy feature to get used to once you&rsquo;ve started using it, and when writing light Sinatra apps it&rsquo;s one feature that I miss enough to pull in. Initially, I&rsquo;d boot up a Sprockets module directly in my application&rsquo;s rackup file (that&rsquo;s <code>config.ru</code>) and map it to a particular URL path. This approach works well, but makes a mess in the rackup file and offers little in terms of fine-grained control. More recently I&rsquo;ve found that I can get better flexibility and clarity by running it from a custom Sinatra module.</p>

<p>Add Sprockets and Yahoo&rsquo;s YUI compressor to your <code>Gemfile</code>:</p>

<pre><code class="language-ruby">gem &quot;sprockets&quot;
gem &quot;yui-compressor&quot;

# I find it well worth to include CoffeeScript and SASS as well
gem &quot;coffee-script&quot;
gem &quot;sass&quot;
</code></pre>

<p>Your assets file structure should look something like this:</p>

<pre><code>+ assets
  + images
    - my-jpg.jpg
    - my-png.png
  + javascripts
    - app.js
    - my-scripts.coffee
  + stylesheets
    - app.css
    - my-styles.sass
</code></pre>

<p><code>app.js</code> should load all other JavaScript assets in its directory (in the example structure above this will pick up <code>my-scripts.coffee</code>):</p>

<pre><code class="language-javascript">//= require_tree
</code></pre>

<p><code>app.css</code> as well (includes <code>my-styles.sass</code>):</p>

<pre><code class="language-css">//= require_tree
</code></pre>

<p>The Sinatra module should look something like this:</p>

<pre><code class="language-ruby">class Assets &lt; Sinatra::Base
  configure do
    set :assets, (Sprockets::Environment.new { |env|
      env.append_path(settings.root + &quot;/assets/images&quot;)
      env.append_path(settings.root + &quot;/assets/javascripts&quot;)
      env.append_path(settings.root + &quot;/assets/stylesheets&quot;)

      # compress everything in production
      if ENV[&quot;RACK_ENV&quot;] == &quot;production&quot;
        env.js_compressor  = YUI::JavaScriptCompressor.new
        env.css_compressor = YUI::CssCompressor.new
      end
    })
  end

  get &quot;/assets/app.js&quot; do
    content_type(&quot;application/javascript&quot;)
    settings.assets[&quot;app.js&quot;]
  end

  get &quot;/assets/app.css&quot; do
    content_type(&quot;text/css&quot;)
    settings.assets[&quot;app.css&quot;]
  end

  %w{jpg png}.each do |format|
    get &quot;/assets/:image.#{format}&quot; do |image|
      content_type(&quot;image/#{format}&quot;)
      settings.assets[&quot;#{image}.#{format}&quot;]
    end
  end
end
</code></pre>

<p>Now use the assets module as middleware in  <code>config.ru</code>, and delegate everything else to your main app:</p>

<pre><code class="language-ruby">use Assets
run Sinatra::Application
</code></pre>
]]></content>
    <published>2012-11-10T14:59:56-08:00</published>
    <updated>2012-11-10T14:59:56-08:00</updated>
    <link href="https://mutelight.orgasset-pipeline"></link>
    <id>tag:mutelight.org,2012-11-10:/asset-pipeline</id>
    <author>
      <name>Brandur Leach</name>
      <uri>https://mutelight.org</uri>
    </author>
  </entry>
  <entry>
    <title>Have Unicorn Log to $stdout</title>
    <content type="html"><![CDATA[<p>A strange quirk of Unicorn is that by default it will write all its logging output to <code>$stderr</code>. Even a relatively harmless operation like a restart will result in noise written to your system error log:</p>

<pre><code class="language-sh">executing [&quot;/home/core/.bundle/gems/ruby/1.8/bin/unicorn_rails&quot;, &quot;-c&quot;, &quot;config/normalized_unicorn.rb&quot;] (in /home/core)
forked child re-executing...
I, [2012-10-17T09:00:35.029145 #12322]  INFO -- : inherited addr=/tmp/core.sock fd=4
I, [2012-10-17T09:00:35.029885 #12322]  INFO -- : Refreshing Gem list
reaped #&lt;Process::Status: pid=2784,exited(0)&gt; worker=1
reaped #&lt;Process::Status: pid=2785,exited(0)&gt; worker=2
reaped #&lt;Process::Status: pid=2783,exited(0)&gt; worker=0
master complete
master process ready
worker=1 ready
worker=2 ready
worker=0 ready
</code></pre>

<p>Simply redefining Unicorn&rsquo;s logger to one pointing to <code>$stdout</code> will fix the problem:</p>

<pre><code class="language-ruby"># by default, Unicorn will log to $stderr; go to $stdout instead
logger Logger.new($stdout)
</code></pre>
]]></content>
    <published>2012-10-21T20:12:24-07:00</published>
    <updated>2012-10-21T20:12:24-07:00</updated>
    <link href="https://mutelight.orgunicorn-stdout"></link>
    <id>tag:mutelight.org,2012-10-21:/unicorn-stdout</id>
    <author>
      <name>Brandur Leach</name>
      <uri>https://mutelight.org</uri>
    </author>
  </entry>
  <entry>
    <title>1 SSL Endpoint, N Apps</title>
    <content type="html"><![CDATA[<p>A few months ago, SSL was rolled out as an included feature for every app on the Heroku platform by enabling secure connection to each app&rsquo;s <strong>heroku/herokuapp.com</strong> domain, so even if developers prefer to use a custom domain, they&rsquo;ll at least have an SSL option for the components of their app where a secure connection is critical.</p>

<p>The platform&rsquo;s answer for developers requiring SSL on a custom domain is the use of the <a href="https://devcenter.heroku.com/articles/ssl-endpoint">SSL Endpoint</a> addon, priced at $20 a month (the dark days of $100/mo. ssl:ip are finally over!). After adding SSL Endpoint to an app, a developer uploads their cert and an endpoint is created with a name like <code>mie-6498.herokussl.com</code>. He or she then CNAMEs their domain to the endpoint and secure requests are routed through with no app changes necessary.</p>

<p>And just a final bit of background: any given request on the Heroku platform enters through the <a href="https://devcenter.heroku.com/articles/http-routing">routing mesh</a>. The tl;dr is that it finds an appropriate runtime where the an app is deployed and forwards its requests through.</p>

<h2>One Endpoint</h2>

<p>In case the $20/mo. per app for a custom domain seems a steep price to pay, you may be happy to find out that in many cases a single SSL Endpoint can be shared between many apps.</p>

<p>Requests coming through an SSL Endpoint follow the same rules as the rest of the platform&ndash;a request may enter through an endpoint but from there is routed through the mesh normally. Therefore, it&rsquo;s not an SSL Endpoint&rsquo;s associated app that decides where a request goes, but rather the incoming domain that&rsquo;s been CNAME&rsquo;d to the endpoint.</p>

<p>A savvy developer can take advantage of this behavior to allow a single SSL Endpoint to route to any number of Heroku apps. For the connection to stay secure, the cert uploaded to the endpoint needs to be signed for any domains that you intended for use for it, but even a <a href="http://www.startcom.org/">free cert from StartCom</a> allows two domains to be included without any special verification. A wildcard certificate (i.e. <code>*.mutelight.org</code>) will secure an entire stack of apps deployed into the Heroku cloud.</p>

<p>Below is a simple example demonstrating how a single endpoint is shared for both <a href="https://brandur.org">brandur.org</a> and <a href="https://facts.brandur.org">facts.brandur.org</a>:</p>

<pre><code class="language-bash">#
# the app brandur-org below has ssl:endpoint
# the app facts-web does not
#

$ heroku addons -a brandur-org
ssl:endpoint

$ heroku addons -a facts-web
No addons installed

#
# both www.brandur.org (entry point for the app brandur-org) and
# facts.brandur.org (app facts-web) are CNAME'd to mie-6498
#

$ host www.brandur.org
www.brandur.org is an alias for mie-6498.herokussl.com.

$ host facts.brandur.org
facts.brandur.org is an alias for mie-6498.herokussl.com.

#
# both apps get a secure connection because brandur-org's cert includes both
# domains
#

$ heroku certs -a brandur-org
Endpoint                Common Name(s)           Expires               Trusted
----------------------  -----------------------  --------------------  -------
mie-6498.herokussl.com  facts.brandur.org,       2013-07-21 03:31 UTC  True
                        www.brandur.org
</code></pre>
]]></content>
    <published>2012-09-03T00:19:42-07:00</published>
    <updated>2012-09-03T00:19:42-07:00</updated>
    <link href="https://mutelight.org1-endpoint-n-apps"></link>
    <id>tag:mutelight.org,2012-09-03:/1-endpoint-n-apps</id>
    <author>
      <name>Brandur Leach</name>
      <uri>https://mutelight.org</uri>
    </author>
  </entry>
  <entry>
    <title>Max Connections for a Postgres Service</title>
    <content type="html"><![CDATA[<p>Once in a while, it&rsquo;s useful to know how many connections your Postgres service can support. For example, at Heroku we use this information to help alert us when any of our production-critical databases are approaching their connection limit.</p>

<p>Inspecting a Postgres configuration file will reveal a setting that specifies the maximum number of connections that its associated service will allow:</p>

<pre><code>max_connections = 20
</code></pre>

<p>As with other settings, this can be checked by connecting to any running Postgres and executing the following query:</p>

<pre><code class="language-sql">select name, setting from pg_settings where name = 'max_connections';
</code></pre>

<p><strong>Protip:</strong> you&rsquo;ll notice that for all our Postgres services at Heroku, from Dev to Ronin, and all the way to Mecha, the response will be <code>500</code>.</p>
]]></content>
    <published>2012-09-02T23:18:31-07:00</published>
    <updated>2012-09-02T23:18:31-07:00</updated>
    <link href="https://mutelight.orgpg-max-connections"></link>
    <id>tag:mutelight.org,2012-09-02:/pg-max-connections</id>
    <author>
      <name>Brandur Leach</name>
      <uri>https://mutelight.org</uri>
    </author>
  </entry>
  <entry>
    <title>Prettifying JSON for Curl Development</title>
    <content type="html"><![CDATA[<p>In the same vein as my post from a few weeks ago on <a href="/netrc">developing an API with Curl and Netrc</a>, here&rsquo;s a handy trick for sending pre-prettified JSON output back to clients, but only those that are identifying as Curl. The reasoning being that prettified JSON isn&rsquo;t useful most of the time, but it&rsquo;s a nice touch while using developing or testing against an API with Curl.</p>

<p>Bundle MultiJson in your <code>Gemfile</code>:</p>

<pre><code class="language-ruby">gem &quot;multi_json&quot;
</code></pre>

<p>Now define a helper for identifying Curl clients, and use it wherever encoding JSON:</p>

<pre><code class="language-ruby"># sample Sinatra app

helpers do
  def curl?
    !!(request.user_agent =~ /curl/)
  end
end

get &quot;/articles&quot; do
  articles = Article.all
  [200, MultiJson.encode(articles, pretty: curl?)]
end
</code></pre>
]]></content>
    <published>2012-09-02T23:02:34-07:00</published>
    <updated>2012-09-02T23:02:34-07:00</updated>
    <link href="https://mutelight.orgpretty-json"></link>
    <id>tag:mutelight.org,2012-09-02:/pretty-json</id>
    <author>
      <name>Brandur Leach</name>
      <uri>https://mutelight.org</uri>
    </author>
  </entry>
  <entry>
    <title>HTTP Basic Authentication with Backbone</title>
    <content type="html"><![CDATA[<p>I recently ran across a situation where I needed to call down to an HTTP basic authenticated API in a Backbone app. Short of doing some hacking of the source, Backbone doesn&rsquo;t provide an easy way to accomplish this, so here&rsquo;s one simple option for your perusal.</p>

<p>Backbone relies on the inclusion of jQuery or Zepto in your project to provide the underlying infrastructure for making AJAX calls. If you&rsquo;re using jQuery, there&rsquo;s a function called <code>$.ajaxSetup</code> that will set options before every AJAX call. Use it to set the <code>Authorization</code> header (<em>warning: CoffeeScript</em>):</p>

<pre><code class="language-coffee">$.ajaxSetup
  headers:
    Authorization: &quot;Basic #{toBase64(&quot;:secret-api-password&quot;)}&quot;
</code></pre>

<p>Under HTTP basic, both the user and password need to be base64 encoded before being sent along to the server. JavaScript doesn&rsquo;t provide utilities to handle that out of the box, so the <code>toBase64</code> function above needs to be implemented to get this example running.</p>

<p>A nice option is <a href="http://code.google.com/p/crypto-js/">CryptoJS</a>. Download the package and include the following files in your project:</p>

<ul>
<li><code>core.js</code></li>
<li><code>enc-base64.js</code></li>
</ul>

<p>Now you&rsquo;re ready to implement <code>toBase64</code> and complete this example:</p>

<pre><code class="language-coffee">toBase64 = (str) -&gt;
  words = CryptoJS.enc.Latin1.parse(str)
  CryptoJS.enc.Base64.stringify(words)
</code></pre>
]]></content>
    <published>2012-09-02T20:18:29-07:00</published>
    <updated>2012-09-02T20:18:29-07:00</updated>
    <link href="https://mutelight.orgbackbone-http-basic-auth"></link>
    <id>tag:mutelight.org,2012-09-02:/backbone-http-basic-auth</id>
    <author>
      <name>Brandur Leach</name>
      <uri>https://mutelight.org</uri>
    </author>
  </entry>
  <entry>
    <title>Caps Lock + Tmux</title>
    <content type="html"><![CDATA[<p>Nothing. But beyond that, without rebinding, the location and function of caps lock on a modern keyboard is more than inconvenient &ndash; it&rsquo;s actively destructive. Enabling caps lock during a Vim session in normal code causes all hell to break loose, and what&rsquo;s going on isn&rsquo;t obvious unless you&rsquo;ve seen the symptoms before.</p>

<p>I was pleasantly surprised to find that in OSX you can now disable caps lock out of the box if you don&rsquo;t intend to rebind it. This is accomplished via <code>System Preferences --&gt; Keyboard --&gt; Modifier Keys --&gt; Caps Lock Key --&gt; No Action</code>, and provides a measurable improvement over the system default.</p>

<p>Then I started to wonder whether I could put caps lock to good use by solving another problem caused by Apple&rsquo;s keyboard design, and the answer turned out to be yes.</p>

<h2>Caps Lock as Tmux Prefix</h2>

<p>Tmux has moved beyond a terminal multiplexing tool and has become one of the most important tools in my kit by acting as the de facto window manager for all my important tools and sessions. As such, I hit my Tmux prefix shortcut <code>C-a</code> <em>a lot</em>, which is tremendously inconvenient because even in 2012 Apple is still jamming a <code>fn</code> key onto everything they make so that <code>ctrl</code> is harder to hit.</p>

<p>Switching to caps lock as a Tmux prefix solves this problem forever. Here&rsquo;s how to do it:</p>

<ol>
<li><p>Download <a href="http://pqrs.org/macosx/keyremap4macbook/pckeyboardhack.html.en">PCKeyboardHack</a>, install, and restart.</p></li>

<li><p>From the new System Preferences pane, change the keycode under the <code>Change Caps Lock</code> entry to <code>109</code> (that&rsquo;s <code>F10</code>), and check its box.</p></li>

<li><p>In your <code>.tmux.conf</code>, change applicable settings to use <code>F10</code>:</p>

<pre><code class="language-bash"># thanks to PCKeyboardHack, F10 is caps lock and caps lock is F10
set-option -g prefix F10

# go to last window by hitting caps lock two times in rapid succession
bind-key F10 last-window
</code></pre></li>
</ol>
]]></content>
    <published>2012-08-19T08:43:46-07:00</published>
    <updated>2012-08-19T08:43:46-07:00</updated>
    <link href="https://mutelight.orgcaps-lock"></link>
    <id>tag:mutelight.org,2012-08-19:/caps-lock</id>
    <author>
      <name>Brandur Leach</name>
      <uri>https://mutelight.org</uri>
    </author>
  </entry>
  <entry>
    <title>The Heroku CLI&#39;s API Workflow</title>
    <content type="html"><![CDATA[<p>Ever wondered how the Heroku command line client accomplishes its work? We don&rsquo;t talk about it much, but the Heroku CLI isn&rsquo;t a black box, it&rsquo;s a fairly thin consumer of our own RESTful API, and that means everything you do in your day to day workflow on Heroku is available to work with in a programmatic fashion. The CLI even uses our own implementation of the API library called <a href="https://github.com/heroku/heroku.rb">heroku.rb</a> (and by the way, heroku.rb is a great choice if you want to consume the API from Ruby).</p>

<p>A handy tool that we use here regularly is inspecting the CLI&rsquo;s workflow by telling Excon to send its output to standard out. Try it for yourself:</p>

<pre><code class="language-bash">EXCON_STANDARD_INSTRUMENTOR=true heroku list
</code></pre>

<p>Any calls that are implemented via heroku.rb make their requests using Excon, but a few of the older endpoints still use Restclient. If you run into one of these, you can do something very similar:</p>

<pre><code class="language-bash">RESTCLIENT_LOG=stdout heroku drains -a mutelight
</code></pre>
]]></content>
    <published>2012-07-09T17:58:52-07:00</published>
    <updated>2012-07-09T17:58:52-07:00</updated>
    <link href="https://mutelight.orgheroku-workflow"></link>
    <id>tag:mutelight.org,2012-07-09:/heroku-workflow</id>
    <author>
      <name>Brandur Leach</name>
      <uri>https://mutelight.org</uri>
    </author>
  </entry>
  <entry>
    <title>406, 415, and API Formatting</title>
    <content type="html"><![CDATA[<p>While working on getting our API restructured, I had to lookup how we want to respond to a client who has requested a format that we don&rsquo;t support. Is it a <code>406</code>? A <code>415</code>? Here are some plain English explanations:</p>

<ul>
<li><code>406 Not acceptable</code> &ndash; In the context of format, when the server can&rsquo;t (or won&rsquo;t) respond in the format that the client has requested. This requested format could come in via an <code>Accept</code> header or an extension in the path.</li>
<li><code>415 Unsupported media type</code> &ndash; when the client has sent content in a request body that the server doesn&rsquo;t support. This would occur during a <code>POST</code> or <code>PUT</code> and may be described by the <code>Content-Type</code> header.</li>
</ul>

<p>A user on Stack Overflow puts it as succinctly as possible: <a href="http://stackoverflow.com/a/3294567">&ldquo;406 when you can&rsquo;t send what they want, and 415 when they send what you don&rsquo;t want.&rdquo;</a>.</p>
]]></content>
    <published>2012-07-03T21:46:23-07:00</published>
    <updated>2012-07-03T21:46:23-07:00</updated>
    <link href="https://mutelight.org406-415"></link>
    <id>tag:mutelight.org,2012-07-03:/406-415</id>
    <author>
      <name>Brandur Leach</name>
      <uri>https://mutelight.org</uri>
    </author>
  </entry>
  <entry>
    <title>A Simple @todo Pattern</title>
    <content type="html"><![CDATA[<p>Modern task tracking apps are both mindblowing in their sophistication, and somewhat ostentatious in their flashiness and sheer volume. We have apps like Remember the Milk that offer multiplatform support with cloud synchronization, and apps like Clear that provide such a beautiful interface and compelling experience that they beg to be used. The earliest apps on both the iPhone and iPad platforms were built for todo lists, and even Apple entered the game years later by introducing their Reminders app in iOS 5.</p>

<p>Despite this attractive selection, I use none of the above. Today, I wanted to share a very simple todo pattern that I&rsquo;ve been using for months now with great results. Here it is in its entirety:</p>

<pre><code>@todo
=====

  * Pick up the milk
  * h/Submit TPS report

Finished
--------

  * h/Order stationery

Defunct
-------

  * Submit talk

# vi: ts=2 sw=2 foldmethod=indent foldlevel=20
</code></pre>

<p>It&rsquo;s <em>that</em> simple:</p>

<ol>
<li>Current tasks go at the top.</li>
<li>Finished go under <em>finished</em>.</li>
<li>Tasks that were missed or have lapsed go under <em>defunct</em>.</li>
</ol>

<p>The list stays open in Vim wrapped in Tmux pane at all times, and gets synced back to Dropbox. Finished items are transferred between lists using fast Vim bindings. If I think of something away from my computer, I add it to my phone, then transfer the task the next time I&rsquo;m back.</p>

<p>The Vim hints at the end provide some nice folding behavior, which is useful when your finished list has become very long. Open and close individual lists using <code>zo</code> and <code>zc</code> respectively (the <code>foldlevel</code> hint at the end ensures that all lists are expanded when the file is first opened).</p>
]]></content>
    <published>2012-07-02T15:43:29-07:00</published>
    <updated>2012-07-02T15:43:29-07:00</updated>
    <link href="https://mutelight.orgtodo"></link>
    <id>tag:mutelight.org,2012-07-02:/todo</id>
    <author>
      <name>Brandur Leach</name>
      <uri>https://mutelight.org</uri>
    </author>
  </entry>
  <entry>
    <title>Building an API with Netrc</title>
    <content type="html"><![CDATA[<p>If you track the progress of the <a href="https://github.com/heroku/heroku">Heroku client</a>, you may have noticed that in the last few months we&rsquo;ve <a href="https://github.com/heroku/heroku/blob/master/CHANGELOG#L278">switched the way that your credentials are stored</a> over from a custom format in <code>~/.heroku</code> to an older and more normalized storage standard, <code>.netrc</code>. This isn&rsquo;t an isolated event either, you may have noticed that GitHub has recently changed the recommended <a href="https://github.com/brandur/dummy">clone method on new repositories to https</a>, which has the side-effect of bypassing your standard access with <code>~/.ssh/id_rsa</code>. How do you get back to not being prompted for your credentials every time you push to the repository? Netrc.</p>

<p><code>.netrc</code> is an old standard that dates all the way back to the days of FTP, that romantic wild west era of the Internet where the concept of &ldquo;passive mode&rdquo; kind of made sense. Its job is to store a user&rsquo;s credentials for accessing remote machines in a simple and consistent format:</p>

<pre><code>machine brandur.org
  login brandur@mutelight.org
  password my-very-secure-personal-password
machine mutelight.org
  login brandur@mutelight.org
  password my-even-secure-password-with-a-number-on-the-end-7
</code></pre>

<p>Although originally intended for FTP, its use has since expanded to a other network clients including Git, Curl, and of course Heroku.</p>

<p>A common pattern that I&rsquo;ve run into while building API&rsquo;s over the last few months is to protect APIs with HTTP basic authentication. This isn&rsquo;t necessarily the best solution in the long term, passing tokens provisioned with OAuth2 may be better, but it&rsquo;s a mechanism that can be set up quickly and easily.</p>

<p>Take this Sinatra app as an example:</p>

<pre><code class="language-ruby"># run with:
#   gem install sinatra
#   ruby -rubygems api.rb

require &quot;sinatra&quot;

set :port, 5000

helpers do
  def auth
    @auth ||= Rack::Auth::Basic::Request.new(request.env)
  end

  def auth_credentials
    auth.provided? &amp;&amp; auth.basic? ? auth.credentials : nil
  end

  def authorized?
    auth_credentials == [ &quot;&quot;, &quot;my-secret-api-key&quot; ]
  end

  def authorized!
    halt 401, &quot;Forbidden&quot; unless authorized?
  end
end

put &quot;/private&quot; do
  authorized!
  200
end
</code></pre>

<p>After running it, we can test our new API with Curl:</p>

<pre><code>curl -i -u &quot;:my-secret-api-key&quot; -X PUT http://localhost:5000/private

HTTP/1.1 200 OK
X-Frame-Options: sameorigin
X-XSS-Protection: 1; mode=block
Content-Type: text/html;charset=utf-8
Content-Length: 0
Connection: keep-alive
Server: thin 1.3.1 codename Triple Espresso
</code></pre>

<p>Now here&rsquo;s the interesting part. Add the following lines to your <code>.netrc</code>:</p>

<pre><code>machine localhost
  password my-secret-api-key
</code></pre>

<p>Try the same Curl command again but using the <code>-n</code> (for <code>--netrc</code>) flag:</p>

<pre><code>curl -i -n -X PUT http://localhost:5000/private
</code></pre>

<p>Voilà! The speed of being able to run ad-hoc queries against an API you&rsquo;re building rather than drudging up your API key every time turns out to be a huge win practically, and it&rsquo;s a pattern that I now use regularly during development.</p>

<p>A limitation that&rsquo;s hinted at above is that you can only have a single entry for <code>localhost</code>. Generally, I find that this isn&rsquo;t a huge problem because most of the APIs I want to hit are deployed in a staging or production environment with a named URL.</p>

<h2>Heroku</h2>

<p>Now onto a nice real-world example. Are you a Heroku user? Have you updated your Gem since February 2012? If the answer to both these questions is <strong>yes!</strong>, try this from a console:</p>

<pre><code>curl -n https://api.heroku.com/apps
</code></pre>

<h2>Security</h2>

<p>A glaring problem with <code>.netrc</code> is that it keeps a large number of your extremely confidential credentials out in the open in plain text. Presumably, the file is <code>chmod</code>&lsquo;ed to <code>600</code> and you&rsquo;re using full-disk encryption, but that&rsquo;s still probably not enough (say someone happens to find your computer unlocked).</p>

<p>The <a href="https://rubygems.org/gems/netrc">netrc</a> gem used by the Heroku client will try to find a GnuPG encrypted file at <code>~/.netrc.gpg</code> before falling back to the plain text version. Although this convention is far from a standard, it&rsquo;s still recommended security practice.</p>
]]></content>
    <published>2012-06-19T19:40:09-07:00</published>
    <updated>2012-06-19T19:40:09-07:00</updated>
    <link href="https://mutelight.orgnetrc"></link>
    <id>tag:mutelight.org,2012-06-19:/netrc</id>
    <author>
      <name>Brandur Leach</name>
      <uri>https://mutelight.org</uri>
    </author>
  </entry>
  <entry>
    <title>Your Ruby App Should Have a `bin/console`</title>
    <content type="html"><![CDATA[<p>Those of us who have worked or are working on Rails are somewhat spoiled by the ability to boot <code>script/rails console</code> and immediately start running commands from inside our projects. What may not be very well known is that this console isn&rsquo;t a piece of Rails black magic, and makes a nice pattern that extends well to any other type of non-Rails Ruby project.</p>

<p>Here&rsquo;s the basic pattern:</p>

<pre><code class="language-ruby">#!/usr/bin/env ruby

require &quot;irb&quot;
require &quot;irb/completion&quot; # easy tab completion

# require your libraries + basic initialization

IRB.start
</code></pre>

<p>With the right initialization, this will immediately drop you into a console with all your project&rsquo;s models, classes, and utilities available, and even with tab completion! It also translates easily over to cloud platforms, being only one <code>heroku run bin/console</code> away, so to speak.</p>

<p>I picked up the idea somewhere at Heroku where public opinion generally sways against heavy Rails-esque frameworks and towards more custom solutions built from the right set of lightweight components.</p>

<p>Here&rsquo;s a real world example for the <a href="https://github.com/brandur/hekla/blob/master/bin/console">bin/console of Hekla</a>, which runs this technical journal:</p>

<pre><code class="language-ruby">#!/usr/bin/env ruby

require &quot;irb&quot;
require &quot;irb/completion&quot;
require &quot;bundler/setup&quot;
Bundler.require

$: &lt;&lt; &quot;./lib&quot;
require &quot;hekla&quot;

DB = Sequel.connect(Hekla::Config.database_url)

require_relative &quot;../models/article&quot;

# Sinatra actually has a hook on `at_exit` that activates whenever it's
# included. This setting will supress it.
set :run, false

IRB.start
</code></pre>
]]></content>
    <published>2012-06-10T15:21:51-06:00</published>
    <updated>2012-06-10T15:21:51-06:00</updated>
    <link href="https://mutelight.orgbin-console"></link>
    <id>tag:mutelight.org,2012-06-10:/bin-console</id>
    <author>
      <name>Brandur Leach</name>
      <uri>https://mutelight.org</uri>
    </author>
  </entry>
  <entry>
    <title>A Cross-version Debug Pattern for Ruby</title>
    <content type="html"><![CDATA[<p>As circumstances would have it, I work with a few projects that need to be compatible with both Ruby 1.8 and 1.9. I also like to use a debugger, and getting one to work seemlessly under both versions isn&rsquo;t exactly intuitive, so I wanted to share a nice pattern that I&rsquo;ve been using recently in my personal projects.</p>

<p>The traditional debugger <code>ruby-debug</code> has been known to be 1.9 incompatible for some time now, but more recently, its updated version <code>ruby-debug19</code> is no longer 1.9 compatible having been broken by 1.9.3 without a new release. Luckily, the awesome new <a href="https://github.com/cldwalker/debugger"><code>debugger</code></a> gem stepped in to fill the gap.</p>

<p>Include both debuggers in your <code>Gemfile</code> with platform conditionals:</p>

<pre><code class="language-ruby">group :development, :test do
  gem &quot;debugger&quot;,   &quot;~&gt; 1.1.3&quot;,  :platforms =&gt; [:ruby_19]
  gem &quot;ruby-debug&quot;, &quot;~&gt; 0.10.4&quot;, :platforms =&gt; [:ruby_18]
end
</code></pre>

<p>I debug pretty often, but don&rsquo;t like to type a lot, so I usually include a shortcut in my <code>test_helper.rb</code> to get a debugger invoked quickly regardless of the Ruby version that you&rsquo;re running:</p>

<pre><code class="language-ruby">def d
  begin
    require &quot;debugger&quot;
  rescue LoadError
    require &quot;ruby-debug&quot;
  end
  debugger
end
</code></pre>

<p>Now drop it into a file like so:</p>

<pre><code class="language-ruby">def requires_frequent_debugging
  risky_call rescue nil
  Singleton.manipulate_global_state
  d # the debugger will start on the next line
  Model.do_business_logic
  super
end
</code></pre>

<p>It might seem like the debugger would start in the <code>d</code> method rather than where you want to debug, forcing you to finish the stack frame before you could start debugging. Fortunately, that&rsquo;s not the case. The <code>d</code> method has returned by the time the debugger is invoked, leaving you exactly where you want to be.</p>

<p>In a classic case of open-source overkill, I&rsquo;ve extracted the pattern described above into a trivial gem called <a href="https://github.com/brandur/d2">d2</a>. Throw it in your Gemfile, make sure that your project is either using <code>Bundler.setup</code> or including <code>require 'd2'</code> somewhere, then use <code>d2</code> somewhere to trigger the debugger.</p>

<p><span class="addendum">Aside &ndash;</span> A slightly interesting Ruby tidbit related to the code above is that we use <code>rescue LoadError</code> because a generic <code>rescue</code> only catches <code>StandardError</code> exceptions. <code>LoadError</code> is derived from a different hierarchy headed by <code>ScriptError</code>.</p>
]]></content>
    <published>2012-06-09T14:33:13-06:00</published>
    <updated>2012-06-09T14:33:13-06:00</updated>
    <link href="https://mutelight.orgruby-debug"></link>
    <id>tag:mutelight.org,2012-06-09:/ruby-debug</id>
    <author>
      <name>Brandur Leach</name>
      <uri>https://mutelight.org</uri>
    </author>
  </entry>
  <entry>
    <title>Authbind with a Simple Test</title>
    <content type="html"><![CDATA[<p>Having recently deployed an <a href="http://aws.amazon.com/elasticloadbalancing/">ELB</a> in front of the production instances running the core Ruby application in our ecosystem, we started experimenting with the idea of removing Nginx from the box&rsquo;s HTTP stack. Why? Having been devised by people smarter than myself, I couldn&rsquo;t understand this idea initially, so let me explain a little further.</p>

<p>We use Unicorn because of its nice <a href="https://github.com/blog/517-unicorn">restarting trick</a> that enables deploys with minimal complexity, and with no dropped connections. A side effect of the mechanism Unicorn uses to provide this feature is that the Unicorn master process runs on a single port which accepts connections, then delegates to one of the worker processes running on the box. That single port bound to by the master process makes a very nice target for the ELB, removing the need for a reverse proxy local to the box. One less component in the HTTP stack is one less piece that can fail, and reduces the incumbent knowledge required to properly manage our stack.</p>

<p>The fact that Unicorn was designed with the expectation of being run behind Nginx to buffer incoming requests and handle slower connections (it&rsquo;s right there on <a href="http://unicorn.bogomips.org/PHILOSOPHY.html">the Unicorn philosophy page</a>) is another discussion, but we generally found that Unicorn runs pretty well on its own for our purposes. That is except when it&rsquo;s behind an ELB in HTTPS mode, but those findings deserve an article of their own.</p>

<h2>Authbind</h2>

<p>Assuming that you want to deploy Unicorn on port 80, the very first challenge you&rsquo;d run into is that on a typical Linux box, root privileges are required to bind to any ports below 1024. A great way to work around this is by using Authbind, start by installing it via your favorite package manager:</p>

<pre><code class="language-bash">aptitude install authbind
</code></pre>

<p>Authbind&rsquo;s permissions are managed with a special set of files in <code>/etc/authbind</code>. Create a file telling Authbind that binding to port 80 should be allowed:</p>

<pre><code class="language-bash">touch /etc/authbind/byport/80
</code></pre>

<p>Authbind determines that a user is allowed to bind an application to port 80 if they have access to execute this file. Change ownership of the file to the user your web server runs under (assumed to be <code>http</code> here) and make sure it has executable (<code>x</code>) permissions. Alternatively, we could accomplish the same thing using groups.</p>

<pre><code class="language-bash"># as root
chown http /etc/authbind/byport/80
chmod 500 /etc/authbind/byport/80
</code></pre>

<p>Test the setup using Python&rsquo;s built-in HTTP server:</p>

<pre><code class="language-bash"># as http user
authbind python -m SimpleHTTPServer 80 # Python 2.x
authbind python -m http.server 80      # Python 3.x
</code></pre>

<p>That&rsquo;s it! Notice that the web server command here should be prefixed by the <code>authbind</code> command for this to be allowed. Another Authbind invocation worth mentioning is <code>authbind --deep</code> which enables port binding permissions for the program being executed, as well as any other child programs spawned from it.</p>
]]></content>
    <published>2012-06-08T11:16:35-06:00</published>
    <updated>2012-06-08T11:16:35-06:00</updated>
    <link href="https://mutelight.orgauthbind"></link>
    <id>tag:mutelight.org,2012-06-08:/authbind</id>
    <author>
      <name>Brandur Leach</name>
      <uri>https://mutelight.org</uri>
    </author>
  </entry>
  <entry>
    <title>Upgrade to Heroku Postgres Dev</title>
    <content type="html"><![CDATA[<p>Our <a href="https://postgres.heroku.com/">Heroku Postgres Dev</a> plans recently went into public beta. They&rsquo;re very exciting work, providing the full power of a true Postgres database for development applications, and for free!</p>

<p>Some key features of the new dev plan is that databases under it are 9.1 (up from 8.3 which is what the shared databases ran under), support hstore, and can be managed remotely using <code>heroku pg:psql</code> or any other Postgres client.</p>

<p>However, since the dev plan adds a brand new database, the default is to end up with an empty store with none of your previous application data. If you&rsquo;re like me, and not too familiar with Heroku Postgres, it might not be immediately obvious how to seemlessly get your data migrated over. Lucky for you though, you&rsquo;re on Heroku! Using pgbackups, there&rsquo;s a very simple way to move your data between databases and produce a backup as a convenient byproduct.</p>

<p>Add the <code>pgbackups</code> addon and capture a backup of your current shared database:</p>

<pre><code>heroku addons:add pgbackups
heroku pgbackups:capture
</code></pre>

<p>The Heroku command will tell you that a backup was produced with a name like <code>b001</code>. Now add your new Postgres dev database:</p>

<pre><code>heroku addons:add heroku-postgresql:dev
</code></pre>

<p>The name of your new database will come back as a token like <code>HEROKU_POSTGRESQL_CYAN</code>. It&rsquo;s attached to your app, but not yet acting as its primary database.</p>

<p>Now all that&rsquo;s left to do is restore the backup you made, and make it your primary:</p>

<pre><code>heroku pgbackups:restore HEROKU_POSTGRESQL_CYAN b001
heroku pg:promote HEROKU_POSTGRESQL_CYAN
</code></pre>

<p>Open a psql session to the new Postgres dev instance and check that all your data is properly in place:</p>

<pre><code>heroku pg:psql
</code></pre>

<p>Optionally, you can destroy your old shared database:</p>

<pre><code>heroku addons:remove shared-database
</code></pre>
]]></content>
    <published>2012-06-07T14:44:00+02:00</published>
    <updated>2012-06-07T14:44:00+02:00</updated>
    <link href="https://mutelight.orgheroku-postgres-dev"></link>
    <id>tag:mutelight.org,2012-06-07:/heroku-postgres-dev</id>
    <author>
      <name>Brandur Leach</name>
      <uri>https://mutelight.org</uri>
    </author>
  </entry>
  <entry>
    <title>The Hitchhiker&#39;s Guide to the Galaxy</title>
    <content type="html"><![CDATA[<p class="subheading">Wikipedia can improve the way that you travel by putting a vast store of knowledge at your fingertips. Offline versions allow it to be carried across international lines without incurring massive data charges.</p>

<p>Many of us have fond memories of reading of Arthur Dent and Ford Prefect traveling the galaxy accompanied by their towels and depressed robot. Readers will remember the friendly book inscribed with the words <em>Don&rsquo;t Panic</em>, the story&rsquo;s namesake and described as &ldquo;the standard repository for all knowledge and wisdom&rdquo;. Back in the 90&rsquo;s the idea of a device with compiled information on nearly everything small enough to carry in your pocket was laughable. The closest thing at the time were encyclopedias spanning entire bookshelves (or CD-ROM&rsquo;s).</p>

<p>Today we have our own form of the Guide, and it&rsquo;s better than even Douglas Adams could have imagined: Wikipedia.</p>

<p>Wikipedia is the ultimate travel resource. On a trip across the country it lets you look up everything&ndash;from towns and landmarks you&rsquo;re visiting to fact checking your tour guide.</p>

<p>This would be a world with almost perfect flow of information if not for one thing. In every country on Earth, phone carriers actively encourage technological retrogression by keeping data roaming rates prohibitively expensive&ndash;and you&rsquo;re going to need data for a Wikipedia lookup.</p>

<p>As luck would have it though, there are a few solutions that will compensate for 3G deficiency on international trips. I personally download a 5 GB dump of Wikipedia with <strong>AllofWiki</strong>, and furthermore use it to browse offline in any country. While traveling Europe last month, I used it to look up the <a href="http://en.wikipedia.org/wiki/Berlin_wall">Berlin Wall</a>, <a href="http://en.wikipedia.org/wiki/Kunsthaus_Tacheles">Kunsthaus Tacheles</a>, the <a href="http://en.wikipedia.org/wiki/Lady_Moura">Lady Moura</a> (anchored in Monaco), <a href="http://en.wikipedia.org/wiki/Catacombs_of_paris">the Catacombs of Paris</a>, <a href="http://en.wikipedia.org/wiki/Tgv">the TGV</a>, and the <a href="http://en.wikipedia.org/wiki/English_language">English language</a> (good read!) amongst hundreds of other subjects. A few days into the trip, it became an absolutely indispensable resource.</p>

<p>3G Kindles are also a good options with international wireless Wikipedia access (for free). Also try <a href="http://offline-wiki.googlecode.com/git/app.html">Offline Wiki in HTML5</a> for a nice notebook solution.</p>

<p>Although we&rsquo;re not yet having Wiki updates pushed to us via the Sub-Etha, this feels like the future.</p>
]]></content>
    <published>2012-01-02T23:59:00-07:00</published>
    <updated>2012-01-02T23:59:00-07:00</updated>
    <link href="https://mutelight.orgthe-hitchhikers-guide-to-the-galaxy"></link>
    <id>tag:mutelight.org,2012-01-02:/the-hitchhikers-guide-to-the-galaxy</id>
    <author>
      <name>Brandur Leach</name>
      <uri>https://mutelight.org</uri>
    </author>
  </entry>
  <entry>
    <title>Learn to Speak Vim</title>
    <content type="html"><![CDATA[<p>I don&rsquo;t often post a blog article containing nothing but a link to content elsewhere, but I&rsquo;ll be doing it today because Yan Pritzker&rsquo;s <a href="http://yanpritzker.com/2011/12/16/learn-to-speak-vim-verbs-nouns-and-modifiers/">Learn to Speak Vim &ndash; Verbs, Nouns, and Modifiers!</a> is just that important.</p>

<p>Frustrated beginners will claim that Vim invovles nothing but rote memorization&ndash;and they&rsquo;re right, but only on the most basic level. Vim&rsquo;s far more important feature is enabling its users to manipulate code on a large scale by building actions from the editor&rsquo;s primitive building blocks. Thinking of these actions as phrases built from verbs, nouns, and modifiers is a very effective way of illustrating this concept.</p>
]]></content>
    <published>2011-12-28T22:03:00-07:00</published>
    <updated>2011-12-28T22:03:00-07:00</updated>
    <link href="https://mutelight.orglearn-to-speak-vim"></link>
    <id>tag:mutelight.org,2011-12-28:/learn-to-speak-vim</id>
    <author>
      <name>Brandur Leach</name>
      <uri>https://mutelight.org</uri>
    </author>
  </entry>
  <entry>
    <title>Subtleties of the X Clipboard</title>
    <content type="html"><![CDATA[<p>A few months back, I started running the lean Archlinux build that I&rsquo;ve been using through to today. I elected not to use a display manager, instead preferring booting <a href="http://awesome.naquadah.org/">Awesome</a> (an aptly named tiling window manager) via the <code>startx</code> script. After getting used to Awesome&rsquo;s key bindings, and throwing Luakit, Urxvt and Tmux into the mix, I got about as close to an optimized Linux build as I was likely to get.</p>

<p>Everything was perfect &hellip; except for one aspect: the clipboard. Its behavior was utterly perplexing: I could select text and middle-click (or <code>Shift-Insert</code>) it most places I wanted, but I could only copy <em>out</em> of Chromium; while pasting it seemed to only respect text that had been copied from itself. Vim was even worse, even with <code>set clipboard=unnamed</code> it didn&rsquo;t seem to play nice with anything else.</p>

<p>This was pretty frustrating&ndash;the clipboard&rsquo;s importance in the everyday workflow really can&rsquo;t be understated. So what was the problem? To understand, we have to know a little more about the X clipboard.</p>

<h2>The X Clipboard</h2>

<p>In X10, <strong>cut-buffers</strong> were introduced. The concept behind this now obsolete mechanism was that when text was selected in a window, the window owner would copy it to a property of the root window called <code>CUT_BUFFER1</code>, a universally owned bucket available to every application on the system. General consensus on cut-buffers was that they were the absolute salt of the Earth, so a new system was devised.</p>

<p>Thus <strong>selections</strong> came about. Rather than applications copying data to a global bucket, they request ownership of a selection. When paste is called from another application, it requests data from the client that currently owns the selection. Aside from being much more versatile and less volatile than cut-buffers, selections can also be faster because no data has to be sent on a copy (only on paste). This is especially advantageous when there&rsquo;s a slow connection to the X server, but this strength is also a weakness because data made available by an application disappears when it closes.</p>

<p>Three selections are defined in the  <acronym title="Inter-Client Communication Conventions Manual">ICCCM</acronym>: <code>CLIPBOARD</code>, <code>PRIMARY</code>, and <code>SECONDARY</code>, each of which behaves like a clipboard in its own right:</p>

<ul>
<li><code>CLIPBOARD</code>: traditionally used when text is copied and pasted from the edit menu, or via the <code>Ctrl+C</code> and <code>Ctrl+V</code> shortcuts in applications that support them.</li>
<li><code>PRIMARY</code>: traditionally used when a mouse selection is made, and pasted with middle-click or <code>Shift-Insert</code>.</li>
<li><code>SECONDARY</code>: ill-defined secondary selection. Most applications don&rsquo;t use it.</li>
</ul>

<p>The heart of the problem for me is that I expected the X clipboard to behave like the clipboard on Windows or Mac OS X, but in fact X&rsquo;s architecture is fundamentally different with two separate, yet equally important, clipboards in use.</p>

<h3 id="section-0" class="link"><a href="#section-0">Vim</a></h3>

<p>Naturally, I had to know how Vim interacts with the X clipboard and was pleased to discover that it has some really great documentation on the subject (see for yourself with <code>:help x11-selection</code>). When running a GUI or X11-aware version of Vim, it has two registers that interact with X:</p>

<ul>
<li><code>*</code> (as in <code>&quot;*yy</code>): is the <code>PRIMARY</code> selection. <code>:set clipboard=unnamed</code> aliases it to the unnamed register.</li>
<li><code>+</code>: is the <code>CLIPBOARD</code> selection. <code>:set clipboard=unnamedplus</code> aliases it to the unnamed register.</li>
</ul>

<p>Vim does not interact with the <code>SECONDARY</code> selection.</p>

<h2>In Practice</h2>

<p>I&rsquo;m a Linux person at heart, but for me the two equal and separate selections remain an unfortunate usability problem.  Luckily for anyone with the same disposition, <a href="http://www.nongnu.org/autocutsel/">Autocutsel</a> can help make X&rsquo;s behavior more logical and intuitive. It&rsquo;s a great little program that synchronizes the cut-buffer with <code>CLIPBOARD</code>, or both the cut-buffer and <code>CLIPBOARD</code> with <code>PRIMARY</code> as well.</p>

<p>Install Autocutsel (<code>pacman -S autocutsel</code> on Arch) and put the following two lines into your <code>.xinitrc</code> (or just run them from a terminal to immediately observe the effects):</p>

<pre><code>autocutsel -fork &amp;
autocutsel -selection PRIMARY -fork &amp;
</code></pre>

<p>Now, no matter where you copy and paste from, be it <code>Ctrl+C</code> in Chrome, <code>p</code> in Vim, or through text selection in X, your clipboard is consistent across the entire system.</p>
]]></content>
    <published>2011-08-03T07:50:00-06:00</published>
    <updated>2011-08-03T07:50:00-06:00</updated>
    <link href="https://mutelight.orgsubtleties-of-the-x-clipboard"></link>
    <id>tag:mutelight.org,2011-08-03:/subtleties-of-the-x-clipboard</id>
    <author>
      <name>Brandur Leach</name>
      <uri>https://mutelight.org</uri>
    </author>
  </entry>
</feed>