RJBS Hanukkah Calendar 2011http://hanukkah.rjbs.manxome.org/2011/2012-01-09T10:45:15-05:00Ricardo SignesXML::Atom::SimpleFeedMore Idiocy-Mitigationware from RJBShttp://hanukkah.rjbs.manxome.org/2011/2011-12-28.html<div class='pod'><h2 id="Lazy-lazy-lazy-lazy-lazy-oops-">Lazy lazy lazy lazy lazy, oops!</h2>
<p>Among my many personal failings, there are two that conspire to cause me more professional embarrassment than most others: I am lazy, and I am sloppy. I don't want to do a lot of inspection to ensure that I haven't made a mistake, and I do not have a supernatural talent for not making mistakes. I do my crosswords in pen, but by the time I'm done, there's very little room left to fill in the right answers, because of all the crossed-out wrong ones.</p>
<p>Some might suggest that the solution to these problems is to grow as a person, take more time on my work, check things more carefully, and be more diligent in the first place. Fortunately for me, those people are wrong. The solution is to be <i>even lazier</i>.</p>
<p>I used to forget to add all my prereqs to my dists' <i>Makefile.PL</i>s. I'd add a <code>use Foo::Bar</code> somewhere, I wouldn't list it as a prereq, and my tests would pass, but somebody else's would not, because they would not have already installed Foo::Bar, and wouldn't be told to, because I was lazy and screwed up. I tried using <a href="https://metacpan.org/module/Test::Without::Module">Test::Without::Module</a> fruitlessly for a while, but eventually I realized it was never going to work: TWM was for hiding things without which the dist should work. I wanted to solve a different problem. I wanted to make sure that everything I needed was listed somewhere.</p>
<p>Laziness helped. I did nothing (other than release a few more broken dists) and eventually Jérôme Quelin released the excellent <a href="https://metacpan.org/module/Dist::Zilla::Plugin::AutoPrereqs">Dist::Zilla::Plugin::AutoPrereqs</a>, which became entirely indispensable so quickly that it was brought into the core Dist-Zilla distribution.</p>
<h2 id="Lazy-lazy-lazy-lazy-lazy-again-oops-">Lazy lazy lazy lazy lazy again, oops!</h2>
<p>Now I had a new problem.</p>
<p>I went and deleted almost all the prereq declarations from my <i>dist.ini</i> files and let AutoPrereqs sort it out for me. It did much better than I did: it picked up modules I forgot, or that I mistakenly thought were core. It picked up the versions that I required, which led me to more often put version requirements in the code, where they belonged. It better sorted my library requirements between testing prereqs and runtime prereqs. It knew to look not just at files in <i>./lib</i> but also in <i>./t</i>.</p>
<p>It even knew how to detect most of my internal for-testing-only packages – but not all of them. Sometimes it would still add a prereq on t::lib::SomeTest, or some internal package I was setting up strangely. Worse, sometimes it would (quite reasonably) pick up a <code>require Foo::Bar</code> statement from a test, well after a line saying to skip the whole test unless Foo was available – quite clear to a human, and totally reasonable in practice, but not good enough for a stupid static analyzer.</p>
<p>These dists would get released with a bunch of prereqs that didn't make sense and couldn't be installed. <code>make test</code> would still work, but anyone trying to install would be stymied by the bogus requirements. I thought about just waiting this one out, to see if Jérôme Quelin would be my savior again, but then I realized that Tatsuhiko Miyagawa had already done a bunch of the work: I could just check every prereq against his simple <a href="http://cpanmetadb.plackperl.org/">CPAN Meta DB</a> service. If the package didn't appear in the CPAN index, a release would be a bad idea.</p>
<h2 id="Hooray-for-lazy-">Hooray for lazy!</h2>
<p>Implementing the whole thing took only about <a href="https://github.com/rjbs/dist-zilla-plugin-checkprereqsindexed/blob/master/lib/Dist/Zilla/Plugin/CheckPrereqsIndexed.pm">a hundred lines of code</a>. Adding the plugin to every one of my dists' configurations took about <a href="https://github.com/rjbs/dist-zilla-pluginbundle-rjbs/commit/6d3f7f3c780d2e86b60d6151fa5b431e8be7baf0">one line of code</a>. The plugin has already saved me <i>dozens</i> of broken releases, and will surely save me more as time goes on. It's not my favorite thing that I wrote in 2011, but it has probably most often saved me from looking like an idiot.</p>
<h2 id="See-Also">See Also</h2>
<ul>
<li><p><a href="https://metacpan.org/module/Dist::Zilla::Plugin::CheckPrereqsIndexed">Dist::Zilla::Plugin::CheckPrereqsIndexed</a></p>
</li>
<li><p><a href="https://metacpan.org/module/Test::Without::Module">Test::Without::Module</a></p>
</li>
</ul>
</div>2011-12-28T00:00:00-05:00Ricardo SignesInfinite Diversity in Infinite MooseXhttp://hanukkah.rjbs.manxome.org/2011/2011-12-27.html<div class='pod'><h2 id="Mutability-Laziness-and-Requiredness">Mutability, Laziness, and Requiredness</h2>
<p>Moose attributes can be mutable ("rw") or immutable ("ro"). (This is a simplification, but basically true.) If an object has a mutable attribute, its value can be changed over and over again during the lifetime of the object. If it's immutable, it can never be changed. It will get a value during initialization and that value will never change. (I am a big fan of immutability, and have <a href="http://rjbs.manxome.org/rubric/entry/1929">written about how mutability can bite you</a> in the past.)</p>
<p>Moose attributes can be required or not. If they're required, then either they must have a value supplied to the constructor or there must be a default for the attribute. (You can mark an attribute both required <i>and</i> give it a default, but with a default, the "required" is redundant.)</p>
<p>Moose attributes can be lazy or not. If they're lazy, and no value was provided in the constructor, then the default won't be sorted out until the attribute's reader is called – in other words, if at all possible, the default <i>won't</i> be sorted out.</p>
<p>These are all useful knobs to be able to twist, but they can't quite all be twisted at the same time. Or, maybe more accurately, there are some combinations of these behaviors that you can't get out of the box.</p>
<p>For example, you might want a way to say that an attribute is both required <i>and</i> lazy. That is, you want a lack of any provided value to be fatal, but only when someone tries to read the attribute that's lacking its required value. You can't really get this with stock Moose attributes, because <code>lazy</code> means that you must provide a default, and if you have a default, <code>required</code> is redundant. When Moose is missing this kind of feature, it's often added in a MooseX module, and that's just what happened here. <a href="https://metacpan.org/module/MooseX::LazyRequire">MooseX::LazyRequire</a> provides a trait that lets you mark attributes as having a "lazy requiredness." (Okay, so this isn't <i>quite</i> lazy and required, but you get the gist, right?)</p>
<h2 id="A-Can-of-WORMs">A Can of WORMs</h2>
<p>I had been using MooseX::LazyRequire for quite a while when I realized that there was another set of knob positions that I wanted but couldn't tune in myself: I wanted attributes that were not required or lazy, but immutable. That is, I wanted to be able to set the value myself, once, but never change it.</p>
<p>In other words, I didn't want a "ro" or "rw" attribute, but a "<a href="http://en.wikipedia.org/wiki/Write_Once_Read_Many">worm</a>" attribute. So, I wrote MooseX::WORM, and immediately everyone under the age of 30 said "What the heck is WORM?" and I finally succumbed and renamed it to <a href="https://metacpan.org/module/MooseX::SetOnce">MooseX::SetOnce</a>.</p>
<p>It works like this:</p>
<table class='code-listing'><tr><td class='line-numbers'><br /><code>1: <br />2: <br />3: <br />4: <br />5: <br />6: <br />7: <br />8: <br />9: <br />10: <br />11: <br />12: <br />13: <br />14: <br />15: <br />16: <br />17: <br />18: <br />19: <br />20: <br />21: <br />22: <br />23: <br />24: <br />25: </code><br /> </td><td class='code'><br /><code><span class="structure">{</span><br /> <span class="keyword">package</span> <span class="word">Person</span><span class="structure">;</span><br /> <span class="keyword">use</span> <span class="word">Moose</span><span class="structure">;</span><br /> <span class="keyword">use</span> <span class="word">MooseX::SetOnce</span><span class="structure">;</span><br /><br /> <span class="word">has</span> <span class="word">first_kiss</span> <span class="operator">=></span> <span class="structure">(</span><br /> <span class="word">is</span> <span class="operator">=></span> <span class="single">'rw'</span><span class="operator">,</span><br /> <span class="word">isa</span> <span class="operator">=></span> <span class="single">'Person'</span><span class="operator">,</span><br /> <span class="word">traits</span> <span class="operator">=></span> <span class="structure">[</span> <span class="single">'SetOnce'</span> <span class="structure">]</span><span class="operator">,</span><br /> <span class="word">predicate</span> <span class="operator">=></span> <span class="single">'ever_been_kissed'</span><span class="operator">,</span><br /> <span class="structure">);</span><br /><span class="structure">}</span><br /><br /><span class="keyword">my</span> <span class="symbol">$alice</span> <span class="operator">=</span> <span class="word">Person</span><span class="operator">-></span><span class="word">new</span><span class="structure">;</span><br /><span class="keyword">my</span> <span class="symbol">$bob</span> <span class="operator">=</span> <span class="word">Person</span><span class="operator">-></span><span class="word">new</span><span class="structure">;</span><br /><br /><span class="keyword">if</span> <span class="structure">(</span><span class="symbol">$alice</span><span class="operator">-></span><span class="word">ever_been_kissed</span><span class="structure">)</span> <span class="structure">{</span><br /><span class="comment"> # never reached<br /></span><span class="structure">}</span><br /><br /><span class="symbol">$alice</span><span class="operator">-></span><span class="word">first_kiss</span><span class="structure">(</span><span class="symbol">$bob</span><span class="structure">);</span><br /><br /><span class="keyword">if</span> <span class="structure">(</span><span class="symbol">$alice</span><span class="operator">-></span><span class="word">ever_been_kissed</span><span class="structure">)</span> <span class="structure">{</span><br /><span class="comment"> # reached!<br /></span><span class="structure">}</span></code><br /> </td></table>
<p>The use cases are relatively few and far between, but when you need it, you need it, and it's easy to use.</p>
<h2 id="and-thanks-Dist::Zilla-Moose-hackers-">...and thanks, Dist::Zilla Moose hackers!</h2>
<p>My initial implementation of MooseX::SetOnce was pretty simple, and worked. As Moose changed, though, it broke, and it looked like learning to fix it was going to be more work than I was excited to do. Fortunately for me, I had used it in Dist::Zilla, and a few of the core Moose team were using Dist::Zilla and, to keep it working, have supplied me with numerous patches to keep it working just right over time. Frankly, I'm not sure how much of the code left in there is mine anymore. <code>git blame</code> suggests about 70%, but I'm guessing most of that is Pod, blank lines, and bugs.</p>
<p>Thanks, everybody!</p>
<h2 id="See-Also">See Also</h2>
<ul>
<li><p><a href="https://metacpan.org/module/MooseX::SetOnce">MooseX::SetOnce</a></p>
</li>
<li><p><a href="https://metacpan.org/module/MooseX::LazyRequire">MooseX::LazyRequire</a></p>
</li>
</ul>
</div>2011-12-27T00:00:00-05:00Ricardo SignesSometimes, You've Gotta Re-invent That Wheelhttp://hanukkah.rjbs.manxome.org/2011/2011-12-26.html<div class='pod'><h2 id="Everybody-Routes">Everybody Routes</h2>
<p>This year, I spent a couple days trying to settle on an HTTP request dispatcher. That is, I wanted to make it easy to say "if somebody requests a path that looks like this, dispatch to this template or subroutine." Everybody who writes anything for the web deals with this problem. It has been solved. It's been solved at least a dozen times, actually.</p>
<p>I just couldn't stand any of the solutions. They were all pretty good-looking tools, but they didn't fit exactly into the hole I'd left in my design. If I was going to avoid a resdesign of my system, I was going to need a to build a bid adapter around these things. I was also going to need a big glass of beer to cry in, because I was going to loathe the work.</p>
<p>So, I just wrote my own. Sure, it was only going to be designed for the needs of one man, but at least that man was me!</p>
<p>Its implementation was inspired by my desire to keep using <a href="http://masonhq.com/">Mason</a> for templates. I really like Mason, at least as a templating system. I like the way its templates can be broken down and can easily use each other. I like the way that many parts of it can be replaced as components. I haven't seen another system that I like as much. None of this is to say, though, that Mason is without its own massive glaring problems.</p>
<p>One of them is that once you remove it from the context of Apache or a similar full-featured HTTP daemon, you realize how many features it's missing. For example, there's no way to trivially tell Mason which templates are okay to serve to HTTP requests and which are only for internal use. There's no magic filename like <i>index.html</i>, unless you want to use a dhandler -- and then you'd have two problems. What I really wanted was a router that would sit between Mason and the web request, doing just that one job that Apache used to do. I wanted it to auto-populate a list of templates that were known to be directly reachable by the user, but I wanted to be able to add routes with placeholders, too. I wanted to be able to simulate Mason's dhandler.</p>
<p>For all this, I only needed a few simple kinds of routes:</p>
<table class='code-listing'><tr><td class='line-numbers'><br /><code>1: <br />2: <br />3: </code><br /> </td><td class='code'><br /><code>/about/careers # simple, static path<br />/user/:uid/profile # a path with named placeholders<br />/posts/:list/query/* # a path with a slurpy star -- "the rest of the URI"</code><br /> </td></table>
<p>These are all easy to route, and cover all of Mason's standard routing and more. The placeholders could be typed. The slurpy star works for dhandlers. A few existing routers handled this, but not all with this exact set of features, or with the ability to easily add routes after construction, and so on.</p>
<p>I ended up with a router like this:</p>
<table class='code-listing'><tr><td class='line-numbers'><br /><code>1: <br />2: <br />3: <br />4: <br />5: <br />6: <br />7: <br />8: <br />9: <br />10: <br />11: <br />12: <br />13: <br />14: <br />15: <br />16: </code><br /> </td><td class='code'><br /><code><span class="keyword">my</span> <span class="symbol">$router</span> <span class="operator">=</span> <span class="word">Router::Dumb</span><span class="operator">-></span><span class="word">new</span><span class="structure">;</span><br /><br /><span class="symbol">$router</span><span class="operator">-></span><span class="word">add_route</span><span class="structure">(</span> <span class="word">Router::Dumb::Route</span><span class="operator">-></span><span class="word">new</span><span class="structure">({</span><br /> <span class="word">parts</span> <span class="operator">=></span> <span class="structure">[</span> <span class="words">qw(about careers)</span> <span class="structure">]</span><span class="operator">,</span><br /> <span class="word">target</span> <span class="operator">=></span> <span class="single">'endpoints/about/careers'</span><span class="operator">,</span><br /><span class="structure">});</span><br /><br /><span class="symbol">$router</span><span class="operator">-></span><span class="word">add_route</span><span class="structure">(</span> <span class="word">Router::Dumb::Route</span><span class="operator">-></span><span class="word">new</span><span class="structure">({</span><br /> <span class="word">parts</span> <span class="operator">=></span> <span class="structure">[</span> <span class="words">qw(user :uid profile)</span> <span class="structure">]</span><span class="operator">,</span><br /> <span class="word">target</span> <span class="operator">=></span> <span class="single">'view/user/profile'</span><span class="operator">,</span><br /><span class="structure">});</span><br /><br /><span class="symbol">$router</span><span class="operator">-></span><span class="word">add_route</span><span class="structure">(</span> <span class="word">Router::Dumb::Route</span><span class="operator">-></span><span class="word">new</span><span class="structure">({</span><br /> <span class="word">parts</span> <span class="operator">=></span> <span class="structure">[</span> <span class="words">qw(posts :list query *)</span> <span class="structure">]</span><span class="operator">,</span><br /> <span class="word">target</span> <span class="operator">=></span> <span class="single">'view/list/query'</span><span class="operator">,</span><br /><span class="structure">});</span></code><br /> </td></table>
<p>This is boring to type, but that's okay, because I knew I wouldn't have to. I was going to organize Mason's <i>comp_root</i> like this:</p>
<table class='code-listing'><tr><td class='line-numbers'><br /><code>1: <br />2: <br />3: <br />4: <br />5: <br />6: <br />7: </code><br /> </td><td class='code'><br /><code>./comp_root<br />./comp_root/endpoints/INDEX<br />./comp_root/endpoints/about/INDEX<br />./comp_root/endpoints/about/careers<br />./comp_root/view/user/profile<br />./comp_root/view/list/profile<br />./comp_root/widget/userbox</code><br /> </td></table>
<p>Anything in <i>endpoints</i> was automatically routable. The <i>INDEX</i> files would handle requests for their containing directory -- otherwise, directories would not be routable. Templates outside of the endpoints directory would only be reachable by explicit routes (like routes to the view templates), or from within existing templates (like common widgets in the widget directory).</p>
<p>I wrote a helper class that would take a router and a directory and map files inside it to routes. That covered the endpoints directory. Next up, I needed to map to the view templates, so I made another helper that would read routes from a simple (read: dumb) text file that looks like this:</p>
<table class='code-listing'><tr><td class='line-numbers'><br /><code>1: <br />2: <br />3: <br />4: </code><br /> </td><td class='code'><br /><code>/user/:uid/profile => /view/user/profile<br /> uid isa Int<br /><br />/posts/:list/query/* => /view/list/query</code><br /> </td></table>
<p>This would set up the routes to the target, and add type constraints if requested. It's a dumb file format, but I can replace it whenever, because it's not part of the router. It's just a file for a tiny helper that converts the file's contents into the lower-level work of calling the router's <i>add_route</i> method.</p>
<p>This had another nice benefit: with our Catalyst applications, our web designer would often add a page to the site, only to find that it wasn't reachable. We needed an action for it. We had a few hacks to work around this, but they were grotty and unsatisfactory. Now, he can either put it in the endpoint directory or edit an extremely simple and straightforward text file to make the new page immediately available -- no need to screw around with the controller classes.</p>
<p>All told, this took a couple hours of work. I put <a href="https://github.com/rjbs/Router-Dumb">the source for Router-Dumb</a> on GitHub, and eventually ended up giving in and putting it on the CPAN. I wanted to use it in more than one place, at which point it was easier to make it yet another CPAN dep than a weird thing that had to be installed from git.</p>
<h2 id="See-Also">See Also</h2>
<ul>
<li><p><a href="https://metacpan.org/module/Router::Dumb">Router::Dumb</a></p>
</li>
<li><p><a href="http://rjbs.manxome.org/rubric/entry/1900">an earlier version of this article, in my blog</a></p>
</li>
<li><p><a href="https://metacpan.org/module/Path::Router">Path::Router</a></p>
</li>
<li><p><a href="https://metacpan.org/module/Path::Dispatcher">Path::Dispatcher</a></p>
</li>
<li><p><a href="https://metacpan.org/module/Router::Simple">Router::Simple</a></p>
</li>
<li><p><a href="https://metacpan.org/module/Forward::Routes">Forward::Routes</a></p>
</li>
</ul>
</div>2011-12-26T00:00:00-05:00Ricardo SignesYo Dawg, I Heard You Liked Perl…http://hanukkah.rjbs.manxome.org/2011/2011-12-25.html<div class='pod'><h2 id="If-youre-going-to-eval-that-string...">If you're going to eval that string...</h2>
<p>If you're going to eval a string, you might as well at least <i>sort of</i> try to get it right. On one hand, I try very hard to never have to use <code>eval STRING</code>, but on the other hand, quite a lot of the code that I release to the CPAN has been generated by a computer program, which means I still have the problem of trying to build valid Perl in a program and then run it without giving it a close inspection. (Why is it computer-generated? It's because <a href="http://advent.rjbs.manxome.org/2009/2009-12-11.html">Dist:Zilla</a> is mucking with the source to add things like <code>$VERSION</code> to it.)</p>
<p>Because I was going to be screwing around with hunks of Perl, I wanted a way to try and limit the mistakes I could make to the particularly weird, and not to the kind of stupid mistakes I make eighty-six times already every day. For example, did you notice that I forgot one of the colons in Dist::Zilla, above? I do that all the time, and never learn.</p>
<p>Or, well, maybe I do learn, because I ended up doing what I always do when I realize that I'm not going to get any better. I wrote more code to keep me in check. <i>Type</i> check. (Sorry.)</p>
<p>I use <a href="http://moose.perl.org/">Moose</a> all the time, and it has a set of interfaces for type constraints. Type constraints are, at the heart of it, just little data validators that look at a scalar and say "yes, it's what you asked for" or "no, it's no good." They're most often seen constraining the values that can be stored in an instance attribute, but you can use them all over the place. I use them for validating user input and subroutine parameters, for example, and I usually use the ones whose names start with MooseX::Types.</p>
<p>For example, I could do this:</p>
<table class='code-listing'><tr><td class='line-numbers'><br /><code>1: <br />2: <br />3: <br />4: <br />5: <br />6: <br />7: <br />8: <br />9: <br />10: </code><br /> </td><td class='code'><br /><code><span class="keyword">use</span> <span class="word">MooseX::Types::Moose</span> <span class="words">qw(Int)</span><span class="structure">;</span><br /><br /><span class="keyword">sub</span> <span class="word">add_two_numbers</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="structure">(</span><span class="symbol">$x</span><span class="operator">,</span> <span class="symbol">$y</span><span class="structure">)</span> <span class="operator">=</span> <span class="magic">@_</span><span class="structure">;</span><br /><br /> <span class="word">die</span> <span class="double">"first argument to add_two_numbers wasn't an int"</span> <span class="word">if</span> <span class="operator">!</span> <span class="word">Int</span><span class="operator">-></span><span class="word">check</span><span class="structure">(</span><span class="symbol">$x</span><span class="structure">);</span><br /> <span class="word">die</span> <span class="double">"second argument to add_two_numbers wasn't an int"</span> <span class="word">if</span> <span class="operator">!</span> <span class="word">Int</span><span class="operator">-></span><span class="word">check</span><span class="structure">(</span><span class="symbol">$y</span><span class="structure">);</span><br /><br /> <span class="keyword">return</span> <span class="symbol">$x</span> <span class="operator">+</span> <span class="symbol">$y</span><span class="structure">;</span><br /><span class="structure">}</span></code><br /> </td></table>
<p>Yeah, it's contrived, but you get the picture. You can use these things everywhere – and since they're the <i>only</i> things you can use for attribute validation, get used to using them. The more type constraints you have in your arsenal, the more effectively you can validate your data and catch errors before they manifest in mysterious ways, much later.</p>
<h2 id="MooseX::Types::Perl">MooseX::Types::Perl</h2>
<p>To help prevent my constant stupid errors in package names (and some other things) I wrote MooseX::Types::Perl. It's just a set of type constraints that require datas fall into fairly strict interpretations of common Perl langauge tokens.</p>
<p>For example, Dist::Zilla is going to build a dist, it makes sure that its name is a <code>DistName</code> and that all the modules in it are <code>ModuleName</code>s. These are pretty similar – the latter expects dashes and the former double-colons. It's worth noting, too, that neither of them allows the use of apostrophes for package separators. In fact, there are a lot of things that Perl will accept that these types will not. Here's a quote from a <a href="https://rt.perl.org/rt3//Public/Bug/Display.html?id=105922">recent bug report</a> to Perl.</p>
<pre><code> This is apparently a valid package name:
_#@*$!@*^(
But this is not:
#@*$!@*^(_</code></pre>
<p>Yeah, I'm just not going to support that. Neither, too, are packages like <code>Rÿche::Queen</code> supported, with that pesky umlaut. These work, in many places, but they're not really reliable, and you end up getting into lots of twitchy differences with things like normalization and filesystem encoding, and … well, it's just not going to make your life a lot of fun, unless you're into that sort of thing. Me? I just want my code to work, so I make my types very, very strict.</p>
<p>This means that MooseX::Types::Perl's Identifier type limits you to the kind of variable names that will not get you glowered at by your co-workers: alpha-or-underscore followed by zero or more alnum-or-underscore – and that's all ASCII, people!</p>
<p>There are also types for version strings, and I am a big fan of requiring things conform to StrictVersionStr. Keep your weird version strings to yourself! It is only because I want to avoid funny looks that I don't just make all my versions conform to <code>Int</code>!</p>
<p>MooseX::Types::Perl is really simple in concept, and also <a href="https://github.com/rjbs/moosex-types-perl/blob/master/lib/MooseX/Types/Perl.pm">in its implementation</a>. It shows how easy it is to build your own type libraries for dealing with all <i>your</i> favored value domains and catching your screw-ups early with a nice, perfectly clear 47-line stack trace.</p>
<h2 id="See-Also">See Also</h2>
<ul>
<li><p><a href="https://metacpan.org/module/MooseX::Types::Perl">MooseX::Types::Perl</a></p>
</li>
<li><p><a href="https://metacpan.org/module/MooseX::Types">MooseX::Types</a></p>
</li>
</ul>
</div>2011-12-25T00:00:00-05:00Ricardo SignesYes. I wrote a source filter.http://hanukkah.rjbs.manxome.org/2011/2011-12-24.html<div class='pod'><h2 id="Everybody-Has-Queries">Everybody Has Queries</h2>
<p>About a lifetime ago, I worked at a place that <a href="http://en.wikipedia.org/wiki/Epitaxy">grew semiconductors</a>. We gathered a lot of data about every wafer that we produced, and there were lots of variables to consider. What affected what? What could be changed to make everything even better. Frankly, I <i>never had any idea.</i></p>
<p>I was just the programmer! Sometimes, people pretended I was a data analyst, but I made it pretty clear that I had no idea how to analyze data for trends or correlation or… anything. We had lots of people on staff who did know that stuff. The problem was that they weren't about to learn <a href="http://en.wikipedia.org/wiki/T-SQL">T-SQL</a> or Visual Basic just to get custom, automated queries dumped into Excel where they could muck around, build graphs, or copy the data into <a href="http://en.wikipedia.org/wiki/SPSS">SPSS</a>.</p>
<p>These were smart guys. There were lots of Ph.D.s spread around, and learning a little programming was easy. It would've been a waste of my time and theirs to try to teach them <i>programming</i>, though. What it really meant was that I could write a tool for them that didn't need nice buttons or a GUI or even much error handling. I could write them a programming tool, but it couldn't be designed the way a programmer would want.</p>
<p>I'm still not sure that it was great idea, but it <i>was</i> a big success within the organization, so it couldn't have been all bad.</p>
<p>What I wrote was <a href="https://metacpan.org/module/Querylet">Querylet</a>, which provided a simplified programming system that took a series of commands to query and munge data, then output it for the user to read.</p>
<p>A very simple Querylet program might look like this:</p>
<table class='code-listing'><tr><td class='line-numbers'><br /><code>1: <br />2: <br />3: <br />4: <br />5: <br />6: <br />7: <br />8: <br />9: <br />10: <br />11: <br />12: <br />13: <br />14: <br />15: <br />16: <br />17: <br />18: <br />19: <br />20: <br />21: <br />22: <br />23: <br />24: </code><br /> </td><td class='code'><br /><code>use Querylet;<br /><br />database: dbi:SQLite:dbname=wafers.db<br /><br />query:<br /> SELECT wafer_id, material, diameter, failurecode<br /> FROM grown_wafers<br /> WHERE reactor_id = 105<br /> AND product_type <> 'Calibration'<br /><br />add column surface_area:<br /> $value = $row->{diameter} * 3.14;<br /><br />add column cost:<br /> $value = $row->{surface_area} * 100 if $row->{material} eq 'GaAs';<br /> $value = $row->{surface_area} * 200 if $row->{material} eq 'InP';<br /><br />munge column failurecode:<br /> $value = 10 if $value == 3; # 3's have been reclassified<br /><br />munge all values:<br /> $value = '(null)' unless defined $value;<br /><br />output format: html</code><br /> </td></table>
<p>It should be pretty obvious what this does! It connects to a database, executes a query, adds virtual columns, munges existing columns, and then outputs the whole resultset as an HTML table. We used a subclass of Querylet that built in the database connection, so nobody ever had to think about <code>dsn</code> strings.</p>
<p>So, what <i>did</i> they have to learn? Well, they had to learn SQL. That was easy, even when they had to do non-trivial joins. I wrote a short tutorial on how the relational model worked, and MS SQL Server had very good facilities for things like subqueries, making lots of SQL easy to write and execute efficiently.</p>
<p>They had to learn a couple Perl primitives, like the mathematical operators, statement conditions, and <code>eq</code> versus <code>==</code>. They didn't have to learn anything about hashes. They just had to learn that <code>$value</code> meant a single value under consideration and that <code>$row->{field}</code> meant one field in a row.</p>
<p>Then there was a short list of directives like "add column" or "munge all values" to learn. This took about one page of documentation.</p>
<h2 id="Output-Plugins-">Output Plugins!</h2>
<p>Of course, outputting the data to an HTML table was dumb. They wanted the data to use in complex programs, and let me tell you, they were not going to write those in JavaScript running on MSIE 5. We needed to get this stuff into Excel!</p>
<p>We had this line, above:</p>
<table class='code-listing'><tr><td class='line-numbers'><br /><code>1: </code><br /> </td><td class='code'><br /><code><span class="word">output</span> <span class="label">format:</span> <span class="word">html</span></code><br /> </td></table>
<p>There was also a <a href="https://metacpan.org/module/Querylet::Output::Text">text output plugin</a> that used <a href="https://metacpan.org/module/Text::Table">Text::Table</a> to make a nice little dump of data in your console. I've used that one myself, I'm not ashamed to say. It wasn't very useful for the engineers, though. The useful thing was having a simple way to plug in more output formats. It made it possible to write <a href="https://metacpan.org/module/Querylet::Output::Excel::OLE">Querylet::Output::Excel::OLE</a>.</p>
<p>This was a very simple plugin with a very, very important effect. If you loaded it and requested the output format <code>excel</code>, then Querylet would launch Excel, make a new workbook, and put the query's results into a worksheet in it. You'd just run your program and <i>poof!</i> Excel would have your data and you could get to work. (It used <a href="https://metacpan.org/module/Win32::OLE">Win32::OLE</a>, which was probably one of my most beloved modules at that job.)</p>
<h3 id="Input-Plugins-">Input Plugins!</h3>
<p>Of course, none of these querylets stayed the same forever – at the very least, little parts of the query would have to change. They needed parameters, and so I added them.</p>
<p>You could write the program:</p>
<table class='code-listing'><tr><td class='line-numbers'><br /><code>1: <br />2: <br />3: <br />4: <br />5: <br />6: <br />7: <br />8: </code><br /> </td><td class='code'><br /><code>use Querylet::Input::Term<br />use Querylet;<br /><br />database: dbi:SQLite:dbname=wafers.db<br /><br />query: SELECT * FROM users WHERE userid = ?<br /><br />input: userid</code><br /> </td></table>
<p>...and when you ran the program, you'd be prompted for the <code>userid</code> before it ran. Each <code>input</code> directive prompted for the next <code>?</code> in the query. You'd have to implement your own input plugin. The example looks like this:</p>
<table class='code-listing'><tr><td class='line-numbers'><br /><code>1: <br />2: <br />3: <br />4: <br />5: <br />6: <br />7: <br />8: <br />9: <br />10: <br />11: <br />12: <br />13: <br />14: </code><br /> </td><td class='code'><br /><code><span class="keyword">package</span> <span class="word">Querylet::Input::Term</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="pragma">base</span> <span class="words">qw(Querylet::Input)</span><span class="structure">;</span><br /><br /><span class="keyword">sub</span> <span class="word">default_type</span> <span class="structure">{</span> <span class="single">'term'</span> <span class="structure">}</span><br /><span class="keyword">sub</span> <span class="word">handler</span> <span class="structure">{</span> <span class="cast">\</span><span class="symbol">&from_term</span> <span class="structure">}</span> <br /><br /><span class="keyword">sub</span> <span class="word">from_term</span> <span class="structure">{</span><br /> <span class="keyword">my</span> <span class="structure">(</span><span class="symbol">$query</span><span class="operator">,</span> <span class="symbol">$parameter</span><span class="structure">)</span> <span class="operator">=</span> <span class="magic">@_</span><span class="structure">;</span><br /><br /> <span class="word">print</span> <span class="double">"$parameter: "</span><span class="structure">;</span><br /> <span class="keyword">my</span> <span class="symbol">$input</span> <span class="operator">=</span> <span class="readline"><STDIN></span><span class="structure">;</span><br /> <span class="word">chomp</span> <span class="symbol">$input</span><span class="structure">;</span><br /> <span class="symbol">$query</span><span class="operator">-></span><span class="structure">{</span><span class="word">input</span><span class="structure">}</span><span class="operator">-></span><span class="structure">{</span><span class="symbol">$parameter</span><span class="structure">}</span> <span class="operator">=</span> <span class="symbol">$input</span><span class="structure">;</span><br /><span class="structure">}</span></code><br /> </td></table>
<p>...but it's not what we used. Half the time, we'd pop up a <a href="https://metacpan.org/module/Win32::GUI">Win32::GUI</a> dialog box to get the input. The other time... we did <i>something else</i>.</p>
<h2 id="Something-Else">Something Else</h2>
<p>So, we've got all these querylet programs now, and people want to share, or have them work on machines that haven't been given a Perl install. (By the time I left, I still hadn't gotten the ActivePerl MSI into our standard workstation deployment.) Some of these querylets, we wanted to be easy for any line operator to run. No problem!</p>
<p>We gave the engineers a shared directory on a sort of sandbox server. Then we told them to use <a href="https://metacpan.org/module/Querylet::CGI">Querylet::CGI</a>. With that loaded, all the <code>input</code> parameters would come from the CGI environment. If there was no input, the query would be turned into an HTML form with one text area per <code>input</code> directive. Posting the form would run the query and return the results as HTML or a downloadable <i>XLS</i> file built with another output plugin, <a href="https://metacpan.org/module/Querylet::Output::Excel::XLS">Querylet::Output::Excel::XLS</a>.</p>
<p>Actually, though, we told them to use <a href="https://metacpan.org/module/Querylet::CGI::Auto">Querylet::CGI::Auto</a>. It was just like Querylet::CGI, but if you ran the querylet from your command prompt, it would switch back to GUI input and OLE output.</p>
<h2 id="In-retrospect...">In retrospect...</h2>
<p>In retrospect, I'm not sure what I think about Querylet. I think it was a very, very efficient use of my time, at that job. Instead of writing dozens or hundreds of custom programs, I gave the engineers a programming tool targeting their strengths and interests. Its implementation as a source filter, though, was really self-indulgent and gross. It could probably be entirely re-implemented as a text format with hunks that get embedded in blocks and string-evaluated. Still, when I think about doing that, I don't see what there is to gain. It still wouldn't be safe or highly maintainable. It just might have better error messages sometimes.</p>
<p>I guess I <i>am</i> happy with this horrible hack, after all.</p>
<h2 id="See-Also">See Also</h2>
<ul>
<li><p><a href="https://metacpan.org/module/Querylet">Querylet</a></p>
</li>
<li><p><a href="https://metacpan.org/module/Querylet::Output::Excel::OLE">Querylet::Output::Excel::OLE</a></p>
</li>
<li><p><a href="https://metacpan.org/module/Querylet::Output::Excel::XLS">Querylet::Output::Excel::XLS</a></p>
</li>
<li><p><a href="https://metacpan.org/module/Querylet::Output::Text">Querylet::Output::Text</a></p>
</li>
<li><p><a href="https://metacpan.org/module/Querylet::CGI">Querylet::CGI</a></p>
</li>
<li><p><a href="https://metacpan.org/module/Querylet::CGI::Auto">Querylet::CGI::Auto</a></p>
</li>
</ul>
</div>2011-12-24T00:00:00-05:00Ricardo SignesFactory Factory Factory Factoryhttp://hanukkah.rjbs.manxome.org/2011/2011-12-23.html<div class='pod'><h2 id="I-like-building-classes-">I like building classes!</h2>
<p>I've really been happy with doing more and more stuff with <a href="http://en.wikipedia.org/wiki/Trait_%28computer_programming%29">roles</a> instead of <a href="http://en.wikipedia.org/wiki/Subclass_%28computer_science%29#Subclasses_and_superclasses">subclasses</a>. More and more, I've ended up writing a bunch of roles all related to some kind of <i>thing</i> that I want to make, and then I pick the roles I want, when I make the thing. Is that clear as mud? Let me elaborate:</p>
<p>Stevan Little and I wrote <a href="https://metacpan.org/module/HTTP::Throwable">HTTP::Throwable</a>, a system for throwing exceptions that will be transformed into HTTP responses. There are a bunch of roles that might show up in these: <code>Redirect</code>, for example, means that the exception is going to need a <code>Location</code> header. <code>BoringText</code> can be brought in to let any of the methods get a very simple, boring <code>text_body</code> method.</p>
<p>If you wanted exceptions to have more data, for communicating detailed error messages, you might write a <code>JSONRPC</code> role that gave them a few more attributes and a <code>json_body</code> method. You might add a <code>ForUser</code> role that tacked on even more data, but only if the error came after authentication. It should be easy to imagine many more such roles.</p>
<p>Now, imagine that you've figured out all the roles you might need for your application's full gamut of exception behavior, and that you might need any possible intersection of those. Just <i>naming</i> each and every combination would be a pain. Then you'd have to remember those names, or at least the rule you used to name them. Then you'd have to generate all that code and put it into files. Then you'd have to make sure all the classes were loaded. <i>What a drag!</i></p>
<p>Moose is meant to make a lot of this unneeded, because it gives you tools to generate classes easily at runtime without doing a lot of nasty futzing about with globs and stashes – or worse, building up strings and using <code>eval</code>. Moose lets you generate classes with a nice, simple method-based API. Even that is kind of a drag to use, though, and I wanted something better.</p>
<p>After writing a few one-off somethings kinda-better, I brought some of the designs to the table at work, where my colleague <a href="http://blog.plover.com/">Mark Jason Dominus</a> and I hammered together <a href="https://metacpan.org/module/MooseX::ClassCompositor">MooseX::ClassCompositor</a>.</p>
<h2 id="Class-Compositor-">Class Compositor?</h2>
<p>A MooseX::ClassCompositor is a class <a href="http://en.wikipedia.org/wiki/Factory_%28software_concept%29">factory</a>: it's a thing whose job is to churn out classes. I didn't want to call it MooseX::ClassFactory, though, because there's a pretty strong knee-jerk reaction among many Perl programmers that "factory" means "overcomplicated." In fact, I bet I've lost a few readers in the last few lines alone by admitting that this article is about a class factory.</p>
<p>Really, a class factory is a simple thing, and it's used to make your code simpler. You tell it what kind of class you want, and it builds it for you. First, though, you have to set the factory up with some settings to explain what kind of classes you plan to build. In our case, we want to make HTTP::Throwble-ish classes.</p>
<table class='code-listing'><tr><td class='line-numbers'><br /><code>1: <br />2: <br />3: <br />4: <br />5: <br />6: <br />7: <br />8: <br />9: <br />10: <br />11: <br />12: </code><br /> </td><td class='code'><br /><code><span class="keyword">my</span> <span class="symbol">$compositor</span> <span class="operator">=</span> <span class="word">MooseX::ClassCompositor</span><span class="operator">-></span><span class="word">new</span><span class="structure">({</span><br /> <span class="word">class_basename</span> <span class="operator">=></span> <span class="single">'MyApp::HTTP::Throwable'</span><span class="operator">,</span><br /> <span class="word">fixed_roles</span> <span class="operator">=></span> <span class="structure">[</span> <span class="single">'=HTTP::Throwable'</span> <span class="structure">]</span><span class="operator">,</span><br /> <span class="word">role_prefixes</span> <span class="operator">=></span> <span class="structure">{</span><br /> <span class="single">''</span> <span class="operator">=></span> <span class="single">'HTTP::Throwable::Role::'</span><span class="operator">,</span><br /> <span class="single">'='</span> <span class="operator">=></span> <span class="single">''</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /><span class="structure">});</span><br /><br /><span class="keyword">my</span> <span class="symbol">$class</span> <span class="operator">=</span> <span class="symbol">$compositor</span><span class="operator">-></span><span class="word">class_for</span><span class="structure">(</span> <span class="words">qw( Redirect JSONRPC ForUser )</span><span class="structure">);</span><br /><br /><span class="symbol">$class</span><span class="operator">-></span><span class="word">throw</span><span class="structure">(</span> <span class="operator">...</span> <span class="structure">);</span></code><br /> </td></table>
<p>We've configured our compositor to <i>always</i> compose the HTTP::Throwable role. If you pass arguments to its <code>class_for</code> method, it will expand them according to the <code>role_prefixes</code> you gave it (using <a href="http://advent.rjbs.manxome.org/2009/2009-12-16.html">String::RewritePrefix</a>) and then also compose those roles. These generated classes get automatically-generated names, too, but we've specified a starting namespace for them with the <code>class_basename</code> parameter, so we can identify the objects more or less, as needed.</p>
<p>Calls to <code>class_for</code> are memoized, too. That means that if you keep asking for the same class over and over, it will quickly give you the one it built before, so it doesn't need to go through all the work of role summation, glob muckery, and so on.</p>
<h2 id="More-Complicated-Composition">More Complicated Composition</h2>
<p>There are a few more useful things you can do with your compositor. For one, you can compose class metaroles. If you know what this is, you may already be glad to hear you can do it. If you <i>don't</i> know what this is, the use case 99% of the time will be adding the following <code>class_metaroles</code> argument:</p>
<table class='code-listing'><tr><td class='line-numbers'><br /><code>1: <br />2: <br />3: <br />4: <br />5: <br />6: <br />7: <br />8: <br />9: <br />10: <br />11: </code><br /> </td><td class='code'><br /><code><span class="keyword">my</span> <span class="symbol">$compositor</span> <span class="operator">=</span> <span class="word">MooseX::ClassCompositor</span><span class="operator">-></span><span class="word">new</span><span class="structure">({</span><br /> <span class="word">class_basename</span> <span class="operator">=></span> <span class="single">'MyApp::HTTP::Throwable'</span><span class="operator">,</span><br /> <span class="word">fixed_roles</span> <span class="operator">=></span> <span class="structure">[</span> <span class="single">'=HTTP::Throwable'</span> <span class="structure">]</span><span class="operator">,</span><br /> <span class="word">class_metaroles</span> <span class="operator">=></span> <span class="structure">{</span><br /> <span class="word">class</span> <span class="operator">=></span> <span class="structure">[</span> <span class="single">'MooseX::StrictConstructor::Trait::Class'</span> <span class="structure">]</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /> <span class="word">role_prefixes</span> <span class="operator">=></span> <span class="structure">{</span><br /> <span class="single">''</span> <span class="operator">=></span> <span class="single">'HTTP::Throwable::Role::'</span><span class="operator">,</span><br /> <span class="single">'='</span> <span class="operator">=></span> <span class="single">''</span><span class="operator">,</span><br /> <span class="structure">}</span><span class="operator">,</span><br /><span class="structure">});</span></code><br /> </td></table>
<p>...and now all your constructed classes have <a href="https://metacpan.org/module/MooseX::StrictConstructor">strict constructors</a>. If you get nothing else out of this article, click that link and start using strict constructors <i>everywhere</i>.</p>
<p>You can also use <a href="https://metacpan.org/module/MooseX::Role::Parameterized">parameterized roles</a> with your compositor:</p>
<table class='code-listing'><tr><td class='line-numbers'><br /><code>1: <br />2: <br />3: <br />4: <br />5: </code><br /> </td><td class='code'><br /><code><span class="keyword">my</span> <span class="symbol">$class</span> <span class="operator">=</span> <span class="symbol">$compositor</span><span class="operator">-></span><span class="word">class_for</span><span class="structure">(</span><br /> <span class="single">'Redirect'</span><span class="operator">,</span><br /> <span class="structure">[</span> <span class="word">RPC</span> <span class="operator">=></span> <span class="word">JSONRPC</span> <span class="operator">=></span> <span class="structure">{</span> <span class="word">our_ident</span> <span class="operator">=></span> <span class="single">'myApp'</span><span class="operator">,</span> <span class="word">default_port</span> <span class="operator">=></span> <span class="number">8080</span> <span class="structure">}</span> <span class="structure">]</span><span class="operator">,</span><br /> <span class="single">'ForUser'</span><span class="operator">,</span><br /><span class="structure">);</span></code><br /> </td></table>
<p>Array references in the roles list use the first element as the nonce name for the parameterized role (for memoization) and the other two parameters as the role's short name and parameters.</p>
<h2 id="Sugar-">Sugar!</h2>
<p>The way we actually use this is actually with something a bit more like this:</p>
<table class='code-listing'><tr><td class='line-numbers'><br /><code>1: <br />2: <br />3: <br />4: <br />5: <br />6: <br />7: <br />8: <br />9: <br />10: <br />11: <br />12: <br />13: </code><br /> </td><td class='code'><br /><code><span class="keyword">use</span> <span class="word">Sub::Exporter</span> <span class="word">-setup</span> <span class="operator">=></span> <span class="structure">[</span> <span class="words">qw(error)</span> <span class="structure">];</span><br /><br /><span class="keyword">my</span> <span class="symbol">$COMPOSITOR</span> <span class="operator">=</span> <span class="word">MooseX::ClassCompositor</span><span class="operator">-></span><span class="word">new</span><span class="structure">({</span><br /> <span class="word">class_basename</span> <span class="operator">=></span> <span class="single">'MyApp::HTTP::Throwable'</span><span class="operator">,</span><br /> <span class="word">fixed_roles</span> <span class="operator">=></span> <span class="structure">[</span> <span class="single">'=HTTP::Throwable'</span><span class="operator">,</span> <span class="operator">...</span> <span class="structure">]</span><span class="operator">,</span><br /> <span class="word">class_metaroles</span> <span class="operator">=></span> <span class="structure">{</span> <span class="word">class</span> <span class="operator">=></span> <span class="structure">[</span> <span class="single">'MooseX::StrictConstructor::Trait::Class'</span> <span class="structure">]</span> <span class="structure">}</span><span class="operator">,</span><br /> <span class="word">role_prefixes</span> <span class="operator">=></span> <span class="structure">{</span><br /> <span class="single">''</span> <span class="operator">=></span> <span class="single">'HTTP::Throwable::Role::'</span><span class="operator">,</span><br /> <span class="single">'='</span> <span class="operator">=></span> <span class="single">''</span><span class="operator">,</span><br /> <span class="structure">}</span><br /><span class="structure">});</span><br /><br /><span class="keyword">sub</span> <span class="word">error</span> <span class="structure">{</span> <span class="symbol">$COMPOSITOR</span><span class="operator">-></span><span class="word">class_for</span><span class="structure">(</span><span class="magic">@_</span><span class="structure">);</span> <span class="structure">}</span></code><br /> </td></table>
<p>...then, anywhere in our code base where we've used that package and imported <code>error</code>, we can write:</p>
<table class='code-listing'><tr><td class='line-numbers'><br /><code>1: </code><br /> </td><td class='code'><br /><code><span class="word">error</span><span class="structure">(</span><span class="words">qw(ForUser Plumbing Temporary)</span><span class="structure">)</span><span class="operator">-></span><span class="word">throw</span><span class="structure">(</span><span class="double">"We'll be right back!"</span><span class="structure">);</span></code><br /> </td></table>
<p>Because of MooseX::ClassCompositor, we've ended up with much, much less of our program's logic stored in prebuilt classes. Instead, we have been able to break things into reusable roles that we compose as needed, efficiently, just in time for use. It's been a very successful and enjoyable experiment.</p>
<h2 id="See-Also">See Also</h2>
<ul>
<li><p><a href="https://metacpan.org/module/MooseX::ClassCompositor">MooseX::ClassCompositor</a></p>
</li>
<li><p><a href="https://metacpan.org/module/MooseX::StrictConstructor">MooseX::StrictConstructor</a></p>
</li>
</ul>
</div>2011-12-23T00:00:00-05:00Ricardo SignesIt's Only a Model!http://hanukkah.rjbs.manxome.org/2011/2011-12-22.html<div class='pod'><h2 id="Testing-the-CPAN">Testing the CPAN</h2>
<p>Ages ago, <a href="http://www.stonehenge.com/merlyn/">Randal Schwarz</a> wrote the <code>minicpan</code> script to produce a minimal, local mirror of the CPAN. With his gracious permission, I converted it to a <a href="https://metacpan.org/module/CPAN::Mini">CPAN distribution</a> so I could install it as easily as anything else. The problem I had was that I really liked writing tests, and writing tests for <code>minicpan</code> was going to be a big pain.</p>
<p>I threw together bunch of pretend CPAN distribution files, all empty tarballs, and wrote an <i>02packages</i> file by hand, and then realized I'd need to put the thing onto a web server, because <code>minicpan</code> works over HTTP. I'd have to update the tarballs by hand, as well as the index, if I needed to test for new things. The whole thing just started to seem like a huge pain, so testing for the CPAN-Mini distribution devolved to the two lamest (but better than nothing) test methods often used: I ran it by hand before releasing and I shipped a test to make sure that the modules all compiled. <i>Ugh.</i></p>
<p>Let's take a quick moment to explain how the CPAN works, on a very basic level, and greatly simplified. There are two crucial parts to a CPAN archive (and many more parts that I will happily ignore, here). First, there is a big pile of files, mostly gzipped tar archives, organized into directories by uploader. Almost to a one, these files are "distributions" with a <i>Makefile.PL</i> or <i>Build.PL</i> and a bunch of code and tests. Secondly, there is <i>02packages.details.txt.gz</i>, which is a very simple list of what Perl packages have been indexed as official parts of the CPAN, and in what archive they can be found. There's an indexer that takes care of updating this file when new distributions are uploaded, and a permissions system, and a whois system, but only the repository of tarballs and the index of packages are relevant for things like the CPAN shell, cpanminus, or <code>minicpan</code>.</p>
<p>Part of the problem with testing against the CPAN is that tests usually want a fixed set of data so they can have a fixed set of expectations. The CPAN is changing all the time. Taking a snapshot is easy, but snapshots are huge! My heavily pruned snapshot is about 1.7 gigs. So, you could prune things down much, much more, but there still might be indexing glitches, affecting what you thought the results should be. Worse, you might care about the contents of distributions, and you might craft those contents yourself, which might lead to things like <a href="https://metacpan.org/module/CPAN::Test::Dummy::Perl5::Build">CPAN::Test::Dummy::Perl5::Build</a>, one of many modules that exist only to serve as test data for live CPAN testing.</p>
<p>These problems, and more, all arose when we decided, at <a href="http://pobox.com/">work</a>, to build a CPAN-like system for managing our deployment of Perl code. We needed to test a lot of its tools against simple distributions, but we didn't want to have to upload them, or even make them. We wanted to make a list of distribution names that we wanted, and then have them show up in directories, and then have an <i>02package</i> file built for us.</p>
<p>I worked on it then, and at several of the annual Perl QA Hackathons, and this spring I delivered the first pass at the final product: <a href="http://fakecpan.org/">Fake CPAN</a>. Before I explain what <i>that</i> is, I'll explain the tools that are under the hood.</p>
<h2 id="Module::Faker">Module::Faker</h2>
<p>The first thing I wanted was a way to quickly build a really boring CPAN distribution from as little input as possible. With that, I could test things like extracting it, running its tests, installing it, and even scanning it to find its author, provided packages, and so on. I sat down and banged out Module::Faker – although it was called ExtUtils::FakeFaker until Schwern begged me not to put it under ExtUtils.</p>
<p>Module::Faker would take a file that looks like a <i>META.yml</i> or <i>META.json</i> file and build the dist that could contain such a file. For example:</p>
<table class='code-listing'><tr><td class='line-numbers'><br /><code>1: <br />2: <br />3: <br />4: </code><br /> </td><td class='code'><br /><code><span class="synIdentifier">name</span><span class="synSpecial">:</span> Email-Infinite<br /><span class="synIdentifier">abstract</span><span class="synSpecial">:</span> please make it stop<br /><span class="synIdentifier">X_Module_Faker</span><span class="synSpecial">:</span><br /> <span class="synIdentifier">cpan_author</span><span class="synSpecial">:</span> RJBS</code><br /> </td></table>
<p>Given this input YAML, Module::Faker will build a distribution with <i>lib/Email/Infinite.pm</i>, with the right Pod in it, and a package declaration, and a version number. It will add a real <i>META.yml</i> and a <i>Makefile.PL</i> and a bunch of other stuff. The <code>X_Module_Faker</code> key provides some extra information, like who should be listed as the author. There are a bunch of other options that I can specify, too, like extra modules to build, extra files to add, version numbers, and so on.</p>
<p>Module::Faker is, in a way, the <a href="http://advent.rjbs.manxome.org/2009/2009-12-11.html">Dist::Zilla</a> of bogus code. (There are some who would suggest that Dist::Zilla is the Dist::Zilla of bogus code.)</p>
<p>Later, I went back and added an ever terser way to specify distributions to fake up. These days, you can just create a file with the right sort of name, like: <i>RJBS_Another-Dist-1.24.tar.gz.dist</i></p>
<p>It will create Another::Dist with version 1.24 and all the required cruft to go along with it, attribute it to me (RJBS), and package it up into a tarball.</p>
<p>Module::Faker was already really useful for testing things like the PAUSE indexer or simple analytical tools. What I wanted, though, was a way to test something that operated on a bunch of distributions in aggregate, with an index file.</p>
<h2 id="CPAN::Faker">CPAN::Faker</h2>
<p>CPAN::Faker builds a bunch of distributions, with an index file. (Who saw that coming?) It uses Module::Faker to build a bunch of fake distributions and then builds an index for them. It's pretty darn simple to use, to. First you drop a bunch of Module::Faker source files (like the YAML or <i>.dist</i> files above) into a directory. Then you run:</p>
<table class='code-listing'><tr><td class='line-numbers'><br /><code>1: </code><br /> </td><td class='code'><br /><code>$ cpanfaker --source ./source-dir --dest ./dest-dir</code><br /> </td></table>
<p>It will build every dist, put it under the hashed author dir like <i>authors/id/R/RJ/RJBS</i>, and then build an index, listing the highest-versioned, first-indexed source for each package. It also creates a few of the other files things need, like <i>CHECKSUMS</i> and so on. The contents of <i>dest-dir</i>, after <code>cpanfaker</code> runs, will be suitable for use by a CPAN client like <code>cpan</code> or <code>minicpan</code> or <code>cpanm</code>¹.</p>
<h2 id="Fake-CPAN">Fake CPAN</h2>
<p>With CPAN::Faker working, I was finally able to do a whole lot of testing. I could easily produce a tiny but CPAN-like hierarchy. For testing CPAN-Mini, what I needed next was to put the whole thing on a web server.</p>
<p>It seemed like a real waste to just put it on some personal server, though, for use only by me when testing my stuff. I wanted to make it a commodity dataset that anybody could use. Once I started to think about it that way, it seemed like I might want a lot of different kinds of these archives. <code>minicpan</code>, after all, would never really open the tarballs up. Other tools might, though, like <code>cpan</code> or <a href="http://pause.perl.org">PAUSE</a>. Not only would having <i>some</i> content matter, but having <i>interesting</i> content would matter. After all, CPAN::Test::Dummy::Perl5::Build isn't just there for fun: it has characteristics that make it useful as a test.</p>
<p>Some of these might get pretty big, so having a bunch of fake CPANs seemed like a good way to segment things into different groups. Then, once these became popular, people's tests might be broken if the exact contents of the fake changed. For example, imagine that a fake CPAN was built populated entirely by dists with edge-case installer behaviors. CPAN client authors could use this fake to test that their code handled things correctly. When a new weird behavior was identified, it could be added to the fake for everyone to test.</p>
<p>The problem is that anyone using the data for testing would suddenly start seeing failures. Rather than require that users bundle the whole fake, we can provide every version of every fake. <code>minicpan</code> has a fake called "minicpan." The latest version is 1.002, and you can find its contents at <code>http://fakecpan.org/fake/minicpan/1.002/cpan/</code></p>
<p>When new dists are added to it, or when its package index file is changed, or whenever it has to change in any way, that can go into <code>http://fakecpan.org/fake/minicpan/1.00<b>3</b>/cpan/</code>. Tests sent out into the wild can always run against the 1.002 URL, and tests run by the developer can always target the latest version by using a URL like <code>http://fakecpan.org/fake/minicpan/<b>latest</b>/cpan/</code></p>
<p>Fake CPAN is currently being used for testing CPAN-Mini, PAUSE, MetaCPAN, CPAN Grep, and other tools.</p>
<h2 id="Footnotes">Footnotes</h2>
<ol>
<li><p>To use a fake CPAN with <code>minicpan</code>, you'll need to use the <code>--mirror</code> and <code>--mirror-only</code> switches.</p>
</li>
</ol>
<h2 id="See-Also">See Also</h2>
<ul>
<li><p><a href="https://metacpan.org/module/CPAN::Mini">CPAN::Mini</a></p>
</li>
<li><p><a href="https://metacpan.org/module/Module::Faker">Module::Faker</a></p>
</li>
<li><p><a href="https://metacpan.org/module/CPAN::Faker">CPAN::Faker</a></p>
</li>
<li><p><a href="http://fakecpan.org/">Fake CPAN</a></p>
</li>
</ul>
</div>2011-12-22T00:00:00-05:00Ricardo SignesIt's Just a Palette Swaphttp://hanukkah.rjbs.manxome.org/2011/2011-12-21.html<div class='pod'><h2 id="Because-I-am-Lazy-">Because I am Lazy!</h2>
<p>If you followed my <a href="http://advent.rjbs.manxome.org/2009/">2009</a> or <a href="http://advent.rjbs.manxome.org/2010/">2010</a> RJBS Advent calendars, or the <a href="http://perladvent.org/2011/">2011</a> Perl Advent calendar, the look of this calendar is probably pretty familiar. I admit it: I'm lazy. I'm no good at visual design (obviously) and I couldn't work up the motivation to do a lot of redesign work. Heck, I couldn't bring myself to do much <i>initial</i> visual design work. The HTML and CSS for <a href="https://metacpan.org/module/WWW::AdventCalendar">WWW::AdventCalendar</a>'s default templates – which I use – were largely lifted from the <a href="http://www.catalystframework.org/calendar">Catalyst Advent Calendar</a>. (By the way, thanks, Catalyst Advent Calendar authors!)</p>
<p>I didn't want to use exactly the same look every year, though. I wanted it to be clear, at least to me, which year I had found myself looking at. To make this possible without having to do much design work, I did a bunch of coding work instead. (This is a common theme for me: it's usually easier to write more code than to learn how to solve the underlying problem.) I made it easy to implement the strategy used by lazy (or, if you prefer, "efficient") graphics programmers since time immemorial: the palette swap.</p>
<p>If you don't know what a palette swap is, think back the early Mario Bros. games. Remember how Luigi and Mario were totally indistinguishable, save for their coloring? Palette swap!</p>
<h2 id="Show-me-the-code-">Show me the code!</h2>
<p>First, let me show you how it got used. First, I went through my CSS file and make a bunch of changes like this:</p>
<table class='code-listing'><tr><td class='line-numbers'><br /><code>1: <br />2: <br />3: <br />4: <br />5: <br />6: <br />7: <br />8: <br />9: <br />10: <br />11: <br />12: <br />13: <br />14: </code><br /> </td><td class='code'><br /><code><span class="synType">diff --git a/share/templates/style.css b/share/templates/style.css</span><br />index b7b603b..27d551e 100644<br /><span class="synType">--- a/share/templates/style.css</span><br /><span class="synType">+++ b/share/templates/style.css</span><br /><span class="synStatement">@@ -3,8 +3,8 @@</span><br /> %color<br /> </%args><br /> body {<br /><span class="synSpecial">- color: #eaf;</span><br /><span class="synSpecial">- background: #61375c;</span><br /><span class="synIdentifier">+ color: <% $color{bodyFG} %>;</span><br /><span class="synIdentifier">+ background: <% $color{bodyBG} %>;</span><br /> font-family: verdana, tahoma, sans-serif;<br /> }</code><br /> </td></table>
<p>In other words, I turned my CSS file into a template. Instead of literal colors, I told it to go get the color from a hash. Almost every use of color got a name. Code sample backgrounds, foregrounds for each type of syntax element, section header foregrounds and backgrounds, the big background blotter, and so on. You can probably imagine what the rest of <a href="https://github.com/rjbs/WWW-AdventCalendar/commit/86bf8ff8d7809a7cc7502e4061cebcf1d5a9c2b8#diff-2">this diff</a> looked like: more of the same.</p>
<p>That's fine, but the hash had to come from somewhere. I could have just built a hash mapping names to colors, but I didn't want to end up with this:</p>
<table class='code-listing'><tr><td class='line-numbers'><br /><code>1: <br />2: <br />3: <br />4: <br />5: <br />6: <br />7: <br />8: <br />9: <br />10: </code><br /> </td><td class='code'><br /><code><span class="keyword">my</span> <span class="symbol">%color</span> <span class="operator">=</span> <span class="structure">(</span><br /> <span class="word">bodyFG</span> <span class="operator">=></span> <span class="single">'#eaf'</span><span class="operator">,</span><br /> <span class="word">bodyBG</span> <span class="operator">=></span> <span class="single">'#61375c'</span><span class="operator">,</span><br /> <span class="word">codeFG</span> <span class="operator">=></span> <span class="single">'#eaf'</span><span class="operator">,</span><br /> <span class="word">codeBG</span> <span class="operator">=></span> <span class="single">'#61375c'</span><span class="operator">,</span><br /> <span class="word">perlFG</span> <span class="operator">=></span> <span class="single">'#eaf'</span><span class="operator">,</span><br /> <span class="word">perlBG</span> <span class="operator">=></span> <span class="single">'#61375c'</span><span class="operator">,</span><br /> <span class="operator">...</span><br /> <span class="operator">...</span><br /><span class="structure">);</span></code><br /> </td></table>
<p>I wanted to be able to reskin things <i>quickly</i> but <i>without</i> resorting to doing a <code>s///</code> substituiton on the code. I wanted named indirection:</p>
<table class='code-listing'><tr><td class='line-numbers'><br /><code>1: <br />2: <br />3: <br />4: <br />5: <br />6: <br />7: <br />8: <br />9: <br />10: <br />11: <br />12: <br />13: </code><br /> </td><td class='code'><br /><code><span class="keyword">my</span> <span class="symbol">%color</span> <span class="operator">=</span> <span class="structure">(</span><br /> <span class="word">faintPink</span> <span class="operator">=></span> <span class="single">'#eaf'</span><span class="operator">,</span><br /> <span class="word">deepLilac</span> <span class="operator">=></span> <span class="single">'#61375c'</span><span class="operator">,</span><br /><br /> <span class="word">bodyFG</span> <span class="operator">=></span> <span class="single">'faintPink'</span><span class="operator">,</span><br /> <span class="word">bodyBG</span> <span class="operator">=></span> <span class="single">'deepLilac'</span><span class="operator">,</span><br /> <span class="word">codeFG</span> <span class="operator">=></span> <span class="single">'faintPink'</span><span class="operator">,</span><br /> <span class="word">codeBG</span> <span class="operator">=></span> <span class="single">'deepLilac'</span><span class="operator">,</span><br /> <span class="word">perlFG</span> <span class="operator">=></span> <span class="single">'faintPink'</span><span class="operator">,</span><br /> <span class="word">perlBG</span> <span class="operator">=></span> <span class="single">'deepLilac'</span><span class="operator">,</span><br /> <span class="operator">...</span><br /> <span class="operator">...</span><br /><span class="structure">);</span></code><br /> </td></table>
<p>...but this is a problem, too. If you want everything that used to be a deep lilac to become burgundy, you'd have to do another big <code>s///</code> again. Instead, you can have <i>two</i> levels of indirection:</p>
<table class='code-listing'><tr><td class='line-numbers'><br /><code>1: <br />2: <br />3: <br />4: <br />5: <br />6: <br />7: <br />8: <br />9: <br />10: <br />11: <br />12: <br />13: <br />14: <br />15: <br />16: </code><br /> </td><td class='code'><br /><code><span class="keyword">my</span> <span class="symbol">%color</span> <span class="operator">=</span> <span class="structure">(</span><br /> <span class="word">faintPink</span> <span class="operator">=></span> <span class="single">'#eaf'</span><span class="operator">,</span><br /> <span class="word">deepLilac</span> <span class="operator">=></span> <span class="single">'#61375c'</span><span class="operator">,</span><br /><br /> <span class="word">generic0</span> <span class="operator">=></span> <span class="single">'faintPink'</span><span class="operator">,</span><br /> <span class="word">generic1</span> <span class="operator">=></span> <span class="single">'deepLilac'</span><span class="operator">,</span><br /><br /> <span class="word">bodyFG</span> <span class="operator">=></span> <span class="single">'generic0'</span><span class="operator">,</span><br /> <span class="word">bodyBG</span> <span class="operator">=></span> <span class="single">'generic1'</span><span class="operator">,</span><br /> <span class="word">codeFG</span> <span class="operator">=></span> <span class="single">'generic0'</span><span class="operator">,</span><br /> <span class="word">codeBG</span> <span class="operator">=></span> <span class="single">'generic1'</span><span class="operator">,</span><br /> <span class="word">perlFG</span> <span class="operator">=></span> <span class="single">'generic0'</span><span class="operator">,</span><br /> <span class="word">perlBG</span> <span class="operator">=></span> <span class="single">'generic1'</span><span class="operator">,</span><br /> <span class="operator">...</span><br /> <span class="operator">...</span><br /><span class="structure">);</span></code><br /> </td></table>
<p>Now, if you want to make that change to all things lilac, you just change the value for <code>generic1</code> to point to something else – pesumably a new entry for <code>burgundy</code>.</p>
<p>This is exactly the problem that <a href="https://metacpan.org/module/Color::Palette">Color::Palette</a> was meant to solve. It makes it easy to change one color everywhere, so that you never accidentally push the big change to use a new color scheme without upgrading that one block somewhere, accidentally rendering your Terms & Conditions in black on black. Oops!</p>
<h2 id="Using-Color::Palette">Using Color::Palette</h2>
<p>Using Color::Palette is easy.</p>
<table class='code-listing'><tr><td class='line-numbers'><br /><code>1: <br />2: <br />3: <br />4: <br />5: <br />6: <br />7: <br />8: <br />9: <br />10: <br />11: <br />12: <br />13: <br />14: <br />15: <br />16: <br />17: <br />18: <br />19: </code><br /> </td><td class='code'><br /><code><span class="keyword">use</span> <span class="word">Color::Palette</span><span class="structure">;</span><br /><span class="keyword">my</span> <span class="symbol">$palette</span> <span class="operator">=</span> <span class="word">Color::Palette</span><span class="operator">-></span><span class="word">new</span><span class="structure">({</span><br /> <span class="word">colors</span> <span class="operator">=></span> <span class="structure">{</span><br /> <span class="word">faintPink</span> <span class="operator">=></span> <span class="single">'#eaf'</span><span class="operator">,</span><br /> <span class="word">deepLilac</span> <span class="operator">=></span> <span class="single">'#61375c'</span><span class="operator">,</span><br /><br /> <span class="word">generic0</span> <span class="operator">=></span> <span class="single">'faintPink'</span><span class="operator">,</span><br /> <span class="word">generic1</span> <span class="operator">=></span> <span class="single">'deepLilac'</span><span class="operator">,</span><br /><br /> <span class="word">bodyFG</span> <span class="operator">=></span> <span class="single">'generic0'</span><span class="operator">,</span><br /> <span class="word">bodyBG</span> <span class="operator">=></span> <span class="single">'generic1'</span><span class="operator">,</span><br /> <span class="operator">...</span><br /> <span class="operator">...</span><br /> <span class="structure">}</span><span class="operator">,</span><br /><span class="structure">});</span><br /><br /><span class="keyword">my</span> <span class="symbol">$color</span> <span class="operator">=</span> <span class="symbol">$palette</span><span class="operator">-></span><span class="word">color</span><span class="structure">(</span><span class="single">'bodyFG'</span><span class="structure">);</span> <span class="comment"># returns a Graphics::Color object</span><br /><br /><span class="keyword">my</span> <span class="symbol">$css_hash</span> <span class="operator">=</span> <span class="symbol">$palette</span><span class="operator">-></span><span class="word">as_css_hash</span><span class="structure">;</span></code><br /> </td></table>
<p>That <code>as_css_hash</code> method returns a hash where the keys are color names and the values are CSS-style RGB color specifiers, like <code>#eaba03</code>. That's just where we get the <code>%color</code> used in the CSS changes we started with.</p>
<p>That's how WWW::AdventColor does things with Color::Palette, at least as of 1.105, but it's not necessariy the <i>best</i> way, because you can do this:</p>
<table class='code-listing'><tr><td class='line-numbers'><br /><code>1: <br />2: <br />3: <br />4: <br />5: <br />6: <br />7: <br />8: <br />9: <br />10: <br />11: <br />12: <br />13: <br />14: </code><br /> </td><td class='code'><br /><code><span class="keyword">use</span> <span class="word">Color::Palette</span><span class="structure">;</span><br /><span class="keyword">use</span> <span class="word">Color::Palette::Schema</span><span class="structure">;</span><br /><br /><span class="keyword">my</span> <span class="symbol">$color_schema</span> <span class="operator">=</span> <span class="word">Color::Palette::Schema</span><span class="operator">-></span><span class="word">new</span><span class="structure">({</span><br /> <span class="word">required_colors</span> <span class="operator">=></span> <span class="structure">[</span> <span class="words">qw( codeFG codeBG bodyFG bodyBG ... )</span> <span class="structure">]</span><span class="operator">,</span><br /><span class="structure">});</span><br /><br /><span class="keyword">my</span> <span class="symbol">$palette</span> <span class="operator">=</span> <span class="word">Color::Palette</span><span class="operator">-></span><span class="word">new</span><span class="structure">({</span><br /><span class="comment"> # exactly what we saw in the example above<br /></span><span class="structure">});</span><br /><br /><span class="keyword">my</span> <span class="symbol">$optimized</span> <span class="operator">=</span> <span class="symbol">$palette</span><span class="operator">-></span><span class="word">optimized_for</span><span class="structure">(</span><span class="symbol">$color_schema</span><span class="structure">);</span><br /><br /><span class="keyword">my</span> <span class="symbol">$css_hash</span> <span class="operator">=</span> <span class="symbol">$optimized</span><span class="operator">-></span><span class="word">as_strict_css_hash</span><span class="structure">;</span></code><br /> </td></table>
<p>A schema object, here, is about as simple as it looks. It's just represents a list of color names that must exist, and it has some behavior that lets you check an existing palette to see whether it can be used for something requiring that schema.</p>
<p>The <code>optimized_for</code> method on a palette takes a schema and returns a new palette with <i>only</i> those colors. Then, the <code>as_<b>strict</b>_css_hash</code> method acts just like <code>as_css_hash</code>, but the returned hashref will throw an error if you try to fetch the value for a key that doesn't exist. This means that if some naughty person has put <code>$color{deepLilac}</code> into the CSS template, and you've optimized down to just the semantic names (like "bodyFG") then your templates won't render. Later, when you replace lilac with burgundy and delete the defition of lilac, you won't start issuing CSS with an empty string as the color value for anything.</p>
<p>You could also get this behavior without the strict hash by using the <code>color</code> method on the palette, which dies on unknown names, but then you'll have to call another method on the result to get the hex string, and it just becomes a lot of typing.</p>
<p>No matter how you get at the colors afterward, though, Color::Palette makes it a lot easier to palette swap your application later.</p>
<h2 id="See-Also">See Also</h2>
<ul>
<li><p><a href="https://metacpan.org/module/Color::Palette">Color::Palette</a></p>
</li>
<li><p><a href="https://metacpan.org/module/Graphics::Color">Graphics::Color</a></p>
</li>
<li><p><a href="https://metacpan.org/module/Color::Library">Color::Library</a></p>
</li>
<li><p><a href="https://metacpan.org/module/WWW::AdventCalendar">WWW::AdventCalendar</a></p>
</li>
<li><p><a href="http://en.wikipedia.org/wiki/Ermac">Ermac</a></p>
</li>
</ul>
</div>2011-12-21T00:00:00-05:00Ricardo Signes