Eric Carmichael's Nerdery/2016-06-26T15:10:00+02:00My Diablo 2 Botting Phase2016-06-26T15:10:00+02:002016-06-26T15:10:00+02:00Eric Carmichaeltag:None,2016-06-26:/my-diablo-2-botting-phase.html<p>My favorite project (ever) was a Diablo II botting system I made in 2008. The summer before sophomore year of college.
If you have never played the game Diablo II, it's your typical
nerdy Dungeons and Dragons kind of collect gear + level up + beat up
monsters type of game. </p>
<p style="text-align: center;" class="image-wrapper">
<a href="http://us.blizzard.com/en-us/games/d2/"><img src="images/d2/ingame.jpg" alt="Diablo 2" style="max-width: 500px;"></a><br>
<span class="centered-label">Diablo …</span></p><p>My favorite project (ever) was a Diablo II botting system I made in 2008. The summer before sophomore year of college.
If you have never played the game Diablo II, it's your typical
nerdy Dungeons and Dragons kind of collect gear + level up + beat up
monsters type of game. </p>
<p style="text-align: center;" class="image-wrapper">
<a href="http://us.blizzard.com/en-us/games/d2/"><img src="images/d2/ingame.jpg" alt="Diablo 2" style="max-width: 500px;"></a><br>
<span class="centered-label">Diablo 2 - Act 2 - Town</span>
</p>
<h2>Diablo II Automation</h2>
<ul>
<li>started at 10-12 years old with <a href="http://autoitscript.com">AutoIt</a> bots that moved based
on the pixels on the screen</li>
<li>then at ~15 got more complicated with an AutoIt OCR (which I'll cover in another article later) that helped
me pickup nice items</li>
<li>around ~17 moved onto C# bots that no longer used pixels, instead read/wrote memory and injected packets</li>
<li>around ~19 I made a clientless bot (no game required, entire state tracked in the bot)</li>
</ul>
<iframe width="640" height="360" class="youtube" src="https://www.youtube-nocookie.com/embed/xqemOAJQBOU?rel=0" frameborder="0" allowfullscreen></iframe>
<p class="centered-label">
packet injecting bot
</p>
<p>A clientless bot tracked the game state, replied to every packet appropriately, and did this
all without a real game client running. Without the client running I could start thousands of bots on a regular
computer, instead of at most 2-4 bots that soaked up all of the memory/CPU to display graphics.</p>
<p>Normally this wouldn't be possible, because most bots required the game to be
running to see the map and path to the monsters. One of the secret ingredients to my bot was the ability
to generate the game map based on the seed received on game join.
I wrapped an API around this map generator and that's what made my bots extra special! </p>
<p>I couldn't have done any of this without the amazing reverse engineers who shared their work
in the Diablo II hacking community. There were entire public wikis
dedicated to definitions for each packet and memory structure. Pretty
cool stuff to be noodling on when you're still in highschool!</p>
<h2>Undetectable maphack</h2>
<p>Here's the neat little maphack that tested out the map generation API:</p>
<iframe width="640" height="360" class="youtube" src="https://www.youtube-nocookie.com/embed/pL7K58Cdo5Y?rel=0" frameborder="0" allowfullscreen></iframe>
<p class="centered-label">
Maphack
</p>
<p>A maphack is a tool that reveals unexplored areas in a game. StarCraft,
Counter-Strike, WarCraft, etc. all have similar tools that give
some players an advantage over others. It's pretty lame to do in Player vs
Player games like StarCraft, but in Diablo 2 it makes finding items less
tedious -- a little bit less cheaty :)</p>
<p>This maphack was novel in that it didn't do anything inside the game
that changed memory or hooked into anything in a strange way. Using the
map seed I was able to generate all the maps in the game, then
stitch each area together. Not to mention with the API we could run the maphack on
a separate computer!</p>
<p>Also, the maphack could do some not-so-undetectable things like hook
into the game and inject "teleport to X, Y" packets until you reach
the destination.</p>
<h2>Finally, clientless bot</h2>
<iframe width="640" height="360" class="youtube" src="https://www.youtube-nocookie.com/embed/9epvPv-mD1Y?rel=0" frameborder="0" allowfullscreen></iframe>
<p class="centered-label">
Bot pathing around the map
</p>
<p>Not only could the bot run without the client, that was pretty cool, but
it also required no configuration. Normally with other bots, you'd have to edit some
.ini or something similar outlining your character, where to put items,
what skills to use, some kind of script to do attacks in a smart way
(i.e. for ranged attacks position yourself far away).</p>
<p>What my bot did, instead of reading some .ini file, was look directly at
your character and infer a good build! This was like <a href="http://heroku.com">Heroku</a> for Diablo
II bots. You just pointed my bot at your character and it took over. If
you had, for example, the "lighting bolt" spell maxed out, the bot would
assume the "RangedAttack" pattern and stay at a decent distance while
staying in line of sight. If you had no items or skills, the bot would
smartly be able to at least punch the monsters!</p>
<p>One of the other cool pieces of this bot was the task queue based module
system. Every action in the game was fired off by some module, and
executed by being pulled off the task queue. For example, I had modules
Mover, Killer, Item Pickup, and Chicken. I could write a whole blog
on Mover Module, but to summarize it I used the non-client based map
generation to stitch together all of the required maps to get from
point A to point B. Meaning, you could ask the Mover module to go to the
last place in the game from the first point in the game, and it could
stitch every map together giving you all of the waypoints + quests
required to get to that location.</p>
<iframe width="640" height="360" class="youtube" src="https://www.youtube-nocookie.com/embed/kGcE9SBIJ68?rel=0" frameborder="0" allowfullscreen></iframe>
<p class="centered-label">
Bot picking stuff up
</p>
<p>The task based queue was especially useful. Consider if you were moving
from Point A to Point B and some monster smacks you to half health, how
will the bot react? The Chicken module will add a "very high" priority
task to get the hell out of that area! </p>
<p>Other cool parts of this bot were: CD Key rotator for running many bots
at once sharing a pool of keys, entire website payment gateway + API,
and some pretty impressive API performance using fancy caching techniques
with IO pooling.</p>
<p>The whole project was built in about 2 months with C#, PHP and JS!</p>Streaming Django Responses on Heroku2016-05-17T15:10:00+02:002016-05-17T15:10:00+02:00Eric Carmichaeltag:None,2016-05-17:/streaming-django-responses-on-heroku.html<p>Django comes with a nice utility class for returning a stream of responses
back to the user. This is useful for returning subprocess stdout or keeping
up with the progress during processing of some file.</p>
<p>Not enough people use this helper! </p>
<h2>Streaming Django Responses</h2>
<p>Here's a small example "Valley Girl …</p><p>Django comes with a nice utility class for returning a stream of responses
back to the user. This is useful for returning subprocess stdout or keeping
up with the progress during processing of some file.</p>
<p>Not enough people use this helper! </p>
<h2>Streaming Django Responses</h2>
<p>Here's a small example "Valley Girl" stream</p>
<p style="text-align: center;" class="image-wrapper">
<img src="images/valley_girl.jpg" class="img-responsive" alt="Valley Girl">
</p>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">time</span>
<span class="kn">from</span> <span class="nn">django.http</span> <span class="kn">import</span> <span class="n">HttpResponse</span><span class="p">,</span> <span class="n">StreamingHttpResponse</span>
<span class="k">def</span> <span class="nf">_valley_girl_stream</span><span class="p">():</span>
<span class="c1"># Get ready to be streamed 50 seconds of this nonsense</span>
<span class="k">for</span> <span class="n">_</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">50</span><span class="p">):</span>
<span class="k">yield</span> <span class="s2">"like, whatever</span><span class="se">\n</span><span class="s2">"</span>
<span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">some_endpoint</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
<span class="k">return</span> <span class="n">StreamingHttpResponse</span><span class="p">(</span><span class="n">_valley_girl_stream</span><span class="p">())</span>
</code></pre></div>
<p>You pass <code>StreamingHttpResponse</code> a generator and it does all of the hard
work for you. So, so handy!</p>
<h2>Gotch'ya!</h2>
<p>Watch out for problems with your WSGI servers and buffering data.</p>
<p style="text-align: center;" class="image-wrapper">
<img src="images/buffering.jpg" class="img-responsive" alt="Buffering problems">
<i><small>Buffering problems</small></i>
</p>
<p>For example, with <a href="http://docs.pylonsproject.org/projects/waitress/en/latest/">Waitress</a> and this code:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">_watch_process_import</span><span class="p">(</span><span class="n">xml_data</span><span class="p">):</span>
<span class="k">yield</span> <span class="s2">"Starting..."</span>
<span class="k">for</span> <span class="n">something</span> <span class="ow">in</span> <span class="n">whatever_dad</span><span class="p">:</span>
<span class="k">yield</span> <span class="n">something</span>
<span class="k">yield</span> <span class="s2">"Done!"</span>
<span class="k">def</span> <span class="nf">do_import</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
<span class="k">if</span> <span class="n">request</span><span class="o">.</span><span class="n">method</span> <span class="o">==</span> <span class="s1">'POST'</span><span class="p">:</span>
<span class="n">xml_data</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">FILES</span><span class="p">[</span><span class="s1">'file'</span><span class="p">]</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
<span class="k">return</span> <span class="n">StreamingHttpResponse</span><span class="p">(</span><span class="n">_watch_process_import</span><span class="p">(</span><span class="n">xml_data</span><span class="p">))</span>
</code></pre></div>
<p>If you run this with a normal Waitress config, it will not send until the connection is closed and flushed.
To make the messages actually stream realtime, I had to modify the first function:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">_watch_process_import</span><span class="p">(</span><span class="n">xml_data</span><span class="p">):</span>
<span class="c1"># this fills the buffer so the messages start streaming</span>
<span class="k">yield</span> <span class="s2">"@"</span> <span class="o">*</span> <span class="mi">50</span> <span class="o">*</span> <span class="mi">1024</span>
<span class="c1"># and now we get back to business...</span>
<span class="k">yield</span> <span class="s2">"Starting..."</span>
<span class="o">...</span>
</code></pre></div>
<p>Obviously, that's not very pretty and makes us feel dirty for having to do it.</p>
<p>So, to fix that, add the arg <code>--send-bytes=1</code> for Waitress to your <code>Procfile</code>, like this:</p>
<div class="highlight"><pre><span></span><code><span class="n">web</span><span class="o">:</span><span class="w"> </span><span class="n">waitress</span><span class="o">-</span><span class="n">serve</span><span class="w"> </span><span class="o">--</span><span class="n">port</span><span class="o">=</span><span class="n">$PORT</span><span class="w"> </span><span class="o">--</span><span class="n">send</span><span class="o">-</span><span class="n">bytes</span><span class="o">=</span><span class="mi">1</span><span class="w"> </span><span class="n">wsgi</span><span class="o">:</span><span class="n">application</span>
</code></pre></div>
<p>That makes waitress flush the buffer as soon as it contains >= 1 byte,
aka all the time with no delay!</p>My favorite pytest plugins2016-04-19T15:10:00+02:002016-04-19T15:10:00+02:00Eric Carmichaeltag:None,2016-04-19:/my-favorite-pytest-plugins.html<h3>What is pytest?</h3>
<blockquote>
<p>pytest is a mature full-featured Python testing tool that helps you write better programs.</p>
</blockquote>
<p>I use pytest on every project because of the ease of use and ability to customize/hook into
certain aspects of the testing processes. For example, if I want to hook in before …</p><h3>What is pytest?</h3>
<blockquote>
<p>pytest is a mature full-featured Python testing tool that helps you write better programs.</p>
</blockquote>
<p>I use pytest on every project because of the ease of use and ability to customize/hook into
certain aspects of the testing processes. For example, if I want to hook in before each test
and process some images (for some reason)... I can do that!</p>
<p>Another great thing about pytest is how easy it is to install plugins. You <code>pip install</code> the
plugin--that's it! No configuration required.</p>
<h2>My faves</h2>
<h3><a href="https://github.com/pytest-dev/pytest-xdist">xdist</a></h3>
<p>With <code>xdist</code> you can run your tests in parallel by providing the <code>-n</code> flag.</p>
<p><code>py.test -n 4</code> will run 4 tests at a time.</p>
<div class="highlight"><pre><span></span><code>$<span class="w"> </span>py.test<span class="w"> </span>-n<span class="w"> </span><span class="nv">4</span>
<span class="o">============================</span><span class="w"> </span><span class="nb">test</span><span class="w"> </span>session<span class="w"> </span><span class="nv">starts</span><span class="w"> </span><span class="o">============================</span>
platform<span class="w"> </span>darwin<span class="w"> </span>--<span class="w"> </span>Python<span class="w"> </span><span class="m">3</span>.5.0,<span class="w"> </span>pytest-2.9.1,<span class="w"> </span>py-1.4.31,<span class="w"> </span>pluggy-0.3.1
django<span class="w"> </span>settings:<span class="w"> </span>settings.test<span class="w"> </span><span class="o">(</span>from<span class="w"> </span>ini<span class="w"> </span>file<span class="o">)</span>
rootdir:<span class="w"> </span>/Users/eric/src/,<span class="w"> </span>inifile:<span class="w"> </span>pytest.ini
plugins:<span class="w"> </span>django-2.9.1,<span class="w"> </span>pythonpath-0.7,<span class="w"> </span>spec-1.0.1,<span class="w"> </span>xdist-1.14
gw0<span class="w"> </span><span class="o">[</span><span class="m">54</span><span class="o">]</span><span class="w"> </span>/<span class="w"> </span>gw1<span class="w"> </span><span class="o">[</span><span class="m">54</span><span class="o">]</span><span class="w"> </span>/<span class="w"> </span>gw2<span class="w"> </span><span class="o">[</span><span class="m">54</span><span class="o">]</span><span class="w"> </span>/<span class="w"> </span>gw3<span class="w"> </span><span class="o">[</span><span class="m">54</span><span class="o">]</span>
scheduling<span class="w"> </span>tests<span class="w"> </span>via<span class="w"> </span>LoadScheduling
.....................................
</code></pre></div>
<h3><a href="https://pypi.python.org/pypi/pytest-spec">spec</a></h3>
<p>Changes our test output from this:</p>
<p style="text-align: center;" class="image-wrapper">
<img src="images/pytest-no-spec.png" alt="Pytest without spec output"><br>
</p>
<p>To this:</p>
<p style="text-align: center;" class="image-wrapper">
<img src="images/pytest-spec.png" alt="Pytest Spec output"><br>
</p>
<p>To use this module add the <code>--spec</code> flag when running your test, like so:</p>
<div class="highlight"><pre><span></span><code>py.test --spec
</code></pre></div>
<h3><a href="https://github.com/bigsassy/pytest-pythonpath">pythonpath</a></h3>
<p>Not as flashy/useful as the others, but a nice tool to simplify adding
proper python paths to run your tests.</p>
<p>Add <code>python_paths</code> to your <code>pytest.ini</code> configuration:</p>
<div class="highlight"><pre><span></span><code><span class="k">[pytest]</span><span class="w"> </span>
<span class="na">python_paths</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s">your/path/apps your/path/libs</span>
</code></pre></div>
<h3><a href="https://github.com/jpvanhal/pytest-instafail">instafail</a></h3>
<p>This plugin instantly shows failed tests instead of waiting until all tests
finish to display the output. Works nicely with <code>spec</code> plugin as well.</p>
<p>To use it just add the <code>--instafail</code> flag when running your tests, like so:</p>
<div class="highlight"><pre><span></span><code>py.test --instafail
</code></pre></div>
<h2>Thanks for reading</h2>
<p>I know this is kind of a "who cares" blog post, but I'm trying to do at least
one blog a month and this one popped into my head so I went for it!</p>
<p>Hope it was worth something to somebody, there's already a great list of
plugins <a href="https://pytest.org/latest/plugins.html">here</a> but I didn't see any harm in outlining my faves.</p>
<p>In any case, happy holidays tomorrow everyone! :)</p>How I'm using RiotJS and Django REST Framework2016-03-23T15:10:00+01:002016-03-23T15:10:00+01:00Eric Carmichaeltag:None,2016-03-23:/how-im-using-riotjs-and-django-rest-framework.html<p>Yo man, what lib you using these days, Angular? React? Vue? Skrible? SomeOtherMadeUpNameJS? What about another one: RiotJS!? <p style="text-align: center;" class="image-wrapper"><img src="images/riot.png" class="img-responsive" alt="RiotJS"></p> We'll take a high level look at the tools I'm using on my current projects for making frontend <-> backend work fast and elegantly, in my opinion ;)</p><p>Yo man, what lib you using these days, Angular? React? Vue? SquanchJS? Some­MadeUp­NameJS?</p>
<p>What about another one: RiotJS!?</p>
<p style="text-align: center;" class="image-wrapper">
<a href="http://riotjs.com/"><img src="images/riot.png" class="img-responsive" alt="RiotJS"></a>
</p>
<p>We'll take a high level look at the tools I'm using on my current projects for making frontend <-> backend work fast and elegantly, in my opinion ;)</p>
<h3>What is RiotJS?</h3>
<p><a href="http://riotjs.com/">RiotJS</a> is "a user interface micro-library for web components."</p>
<p>In RiotJS you make everything as web components. Web components are just a custom tag (like div, a, span, p, etc.) defined so that it has self contained javascript and CSS, like this:</p>
<div class="highlight"><pre><span></span><code><span class="p"><</span><span class="nt">a-riot-tag</span><span class="p">></span>
<span class="cm"><!-- HTML on top --></span>
<span class="p"><</span><span class="nt">div</span> <span class="na">onclick</span><span class="o">=</span><span class="s">"{ clicked }"</span><span class="p">></span>I'm a dumb tag, click me!<span class="p"></</span><span class="nt">div</span><span class="p">></span>
<span class="cm"><!-- JS in the middle (<script> tag is not required) --></span>
<span class="p"><</span><span class="nt">script</span><span class="p">></span>
<span class="w"> </span><span class="c1">// Keep a reference to the tag</span>
<span class="w"> </span><span class="kd">var</span><span class="w"> </span><span class="nx">self</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="k">this</span>
<span class="w"> </span><span class="nx">self</span><span class="p">.</span><span class="nx">clicked</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kd">function</span><span class="p">()</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="nx">alert</span><span class="p">(</span><span class="s1">'Weee, ya clicked it!'</span><span class="p">)</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="p"></</span><span class="nt">script</span><span class="p">></span>
<span class="cm"><!-- CSS --></span>
<span class="p"><</span><span class="nt">style</span><span class="p">></span>
<span class="w"> </span><span class="nt">a-riot-tag</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">background-color</span><span class="p">:</span><span class="w"> </span><span class="kc">red</span><span class="p">;</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="p"></</span><span class="nt">style</span><span class="p">></span>
<span class="p"></</span><span class="nt">a-riot-tag</span><span class="p">></span>
</code></pre></div>
<p>This structure is actually incredibly beautiful, to me at least. When components feel like they're getting spaghetti, you break functionality down into smaller pieces. For a start, say you want to make an address CRUD (Create Retrieve Update Delete) form, you'd probably have components like:</p>
<ul>
<li><code><address-list></code> to list out each address</li>
<li><code><address-form></code> to display details for an address and edit them</li>
<li><code><address></code> to display details about the address</li>
</ul>
<p>You may further break down address into many components, like maybe a <code><phone-number></code> or <code><full-name></code> component with special formatting, properties, options, actions, whatever.</p>
<p style="text-align: center;" class="image-wrapper">
<img src="images/neapolitan.jpg" alt="HTML/JS/CSS Neapolitan"><br>
</p>
<p>Riot components are just custom html tags with some html on top, some javascript and then your styles on the bottom.</p>
<p>This web dev neapolitan seems weird at first... smashing template into logic while stylesheets sit in the corner wondering
how they ended up in this place. However, it actually feels quite nice. Worry shifts from organizing spaghetti
code to solving logic nightmares, because as soon as something "feels weird" you break it off into a new component.
Smaller/single purpose components are generally much easier to wrap your head around.</p>
<p>When I first started doing this component style, I broke things out into components for no reason. That made things feel
clunky and the usefulness of components wasn't making sense. When the logic flow through a form with a ton of methods
started feeling funny: I broke the form into components with separate ideas. The components can talk
to each other through <code>riot.observables</code> with </p>
<div class="highlight"><pre><span></span><code><span class="nx">observable</span><span class="p">.</span><span class="nx">trigger</span><span class="p">(</span><span class="s1">'some_event'</span><span class="p">,</span><span class="w"> </span><span class="p">{</span><span class="nx">some</span><span class="o">:</span><span class="w"> </span><span class="s1">'data'</span><span class="p">})</span>
</code></pre></div>
<p>and </p>
<div class="highlight"><pre><span></span><code><span class="nx">observable</span><span class="p">.</span><span class="nx">on</span><span class="p">(</span><span class="s1">'some_event'</span><span class="p">,</span><span class="w"> </span><span class="kd">function</span><span class="p">(</span><span class="nx">some_data</span><span class="p">){})</span>
</code></pre></div>
<p>For example, I recently built a <code><table></code> of users where you could click a row and it would appear in a <code><form></code>, then
when saved it would update the <code><table></code>. As I was building this all out it started to feel cumbersome trying to keep
everything working in the proper order while maintaining some kind of clarity in the code. Then I had a great "A-ha!"
moment and split the <code><table></code> and the <code><form></code> into separate components! </p>
<p>I didn't also split out the pagination, I didn't split out the <code><table></code> rows... I simply split it into two big pieces.
When you split <em>everything</em> into a component you get a mess. I find things will naturally split themselves up and
anything else is "early optimization" and will probably make your code more fragile/hard to follow.</p>
<h3>What is Django REST Framework (DRF)?</h3>
<p><a href="http://www.django-rest-framework.org/">Django REST Framework (DRF)</a> "is a powerful and flexible toolkit for building Web APIs."</p>
<p>I won't go into much detail about DRF, since it has a lot to do with <a href="https://www.djangoproject.com/">Django</a> models
and all of the batteries included. Instead, I'll let this small example speak for itself.</p>
<p>Here's an API for CRUD (Create Retrieve Update Delete) of some Addresses for a project I'm working on:</p>
<p>Let's make our views (receives a request, processes it, returns a response)</p>
<div class="highlight"><pre><span></span><code><span class="c1"># views.py</span>
<span class="k">class</span> <span class="nc">AddressViewSet</span><span class="p">(</span><span class="n">mixins</span><span class="o">.</span><span class="n">ListModelMixin</span><span class="p">,</span>
<span class="n">mixins</span><span class="o">.</span><span class="n">CreateModelMixin</span><span class="p">,</span>
<span class="n">mixins</span><span class="o">.</span><span class="n">UpdateModelMixin</span><span class="p">,</span>
<span class="n">mixins</span><span class="o">.</span><span class="n">DestroyModelMixin</span><span class="p">,</span>
<span class="n">viewsets</span><span class="o">.</span><span class="n">GenericViewSet</span><span class="p">):</span>
<span class="n">serializer_class</span> <span class="o">=</span> <span class="n">serializers</span><span class="o">.</span><span class="n">AddressSerializer</span>
<span class="k">def</span> <span class="nf">get_queryset</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">if</span> <span class="bp">self</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">user</span><span class="o">.</span><span class="n">is_superuser</span> <span class="ow">or</span> <span class="bp">self</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">user</span><span class="o">.</span><span class="n">is_staff</span><span class="p">:</span>
<span class="n">query</span> <span class="o">=</span> <span class="n">Address</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">all</span><span class="p">()</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">query</span> <span class="o">=</span> <span class="n">Address</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">filter</span><span class="p">(</span><span class="n">company</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">request</span><span class="o">.</span><span class="n">user</span><span class="o">.</span><span class="n">company</span><span class="p">)</span>
<span class="k">return</span> <span class="n">query</span><span class="o">.</span><span class="n">order_by</span><span class="p">(</span><span class="s2">"name"</span><span class="p">)</span>
</code></pre></div>
<p>The above code checks anyone poking at the endpoints and limits so they can only see certain things.
Super users can see all Addresses, while regular users can only see Addresses for their Company.
All results are ordered by name. Awesome sauce! Just the logic we need, not much
boiler plate coding. I love Python + DRF!</p>
<p>Now let's serialize our data (receive data -> validate its contents -> save it OR retrieve data -> output its contents) </p>
<div class="highlight"><pre><span></span><code><span class="c1"># serializers.py</span>
<span class="k">class</span> <span class="nc">AddressSerializer</span><span class="p">(</span><span class="n">serializers</span><span class="o">.</span><span class="n">ModelSerializer</span><span class="p">):</span>
<span class="k">class</span> <span class="nc">Meta</span><span class="p">:</span>
<span class="n">model</span> <span class="o">=</span> <span class="n">Address</span>
<span class="n">fields</span> <span class="o">=</span> <span class="p">(</span>
<span class="s1">'id'</span><span class="p">,</span>
<span class="s1">'company'</span><span class="p">,</span>
<span class="s1">'name'</span><span class="p">,</span>
<span class="s1">'address_1'</span><span class="p">,</span>
<span class="s1">'address_2'</span><span class="p">,</span>
<span class="s1">'city'</span><span class="p">,</span>
<span class="s1">'state'</span><span class="p">,</span>
<span class="s1">'zip_code'</span><span class="p">,</span>
<span class="s1">'is_bill_to'</span><span class="p">,</span>
<span class="s1">'is_ship_to'</span><span class="p">,</span>
<span class="p">)</span>
<span class="k">def</span> <span class="nf">validate</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">attrs</span><span class="p">):</span>
<span class="k">if</span> <span class="s1">'is_bill_to'</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">attrs</span> <span class="ow">and</span> <span class="s1">'is_ship_to'</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">attrs</span><span class="p">:</span>
<span class="k">raise</span> <span class="n">ValidationError</span><span class="p">(</span>
<span class="s2">"Must select at least one option: bill to, ship to, or both"</span>
<span class="p">)</span>
<span class="k">if</span> <span class="n">Address</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">filter</span><span class="p">(</span><span class="n">company</span><span class="o">=</span><span class="n">attrs</span><span class="p">[</span><span class="s1">'company'</span><span class="p">],</span> <span class="n">name</span><span class="o">=</span><span class="n">attrs</span><span class="p">[</span><span class="s1">'name'</span><span class="p">]):</span>
<span class="k">raise</span> <span class="n">ValidationError</span><span class="p">(</span>
<span class="s2">"That name is already in use! Please select a different name."</span>
<span class="p">)</span>
<span class="k">return</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">validate</span><span class="p">(</span><span class="n">attrs</span><span class="p">)</span>
</code></pre></div>
<p>Above we're defining which fields we'll pull from the model. Whether the fields are required, what is valid data, etc.
can be determined from the model fields. For example, you could have the <code>is_bill_to</code> and <code>is_ship_to</code> fields marked
with <code>default=False</code> and DRF will automatically mark them as not required, as they have a default. We also check to
make sure at least one of the previously mentioned options is picked, as well as making sure names are unique to
companies. The unique relation of name->business was be automatically inferred from the Model (awesome!) but the error
message relayed to the user I didn't like, so I wrote this custom instead.</p>
<p>Then attach our view to the router</p>
<div class="highlight"><pre><span></span><code><span class="c1"># urls.py</span>
<span class="n">router</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="sa">r</span><span class="s1">'addresses'</span><span class="p">,</span> <span class="n">views</span><span class="o">.</span><span class="n">AddressViewSet</span><span class="p">,</span> <span class="s1">'addresses'</span><span class="p">)</span>
</code></pre></div>
<p>Now I have all of these endpoints available to modify/view Addresses, WITH documentation!</p>
<p style="text-align: center;" class="image-wrapper">
<img src="images/api_docs.png" alt="API Docs"><br>
</p>
<h3>Tying them together?</h3>
<p>I'll briefly go over how to tie these together API-wise, but I'll hand wave away everything to do with building your front end
and serving static content. Pick your poison, there are a million ways to solve those problems! :)</p>
<p>I've recently learned a neat little practice to help with frontend dev is to make a wrapper for your API calls, this
puts all your endpoint URLs in one spot and makes things feel a bit tidier. Previously, I'd define the URLs in one spot,
grab those and make my call. Now I just return a promise instead.</p>
<div class="highlight"><pre><span></span><code><span class="c1">// Our clientside api wrapper</span>
<span class="kd">var</span><span class="w"> </span><span class="nx">API</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="nx">get_addresses</span><span class="o">:</span><span class="w"> </span><span class="kd">function</span><span class="p">(</span><span class="nx">filters</span><span class="p">)</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">return</span><span class="w"> </span><span class="nx">ajax_library</span><span class="p">(</span><span class="s1">'GET'</span><span class="p">,</span><span class="w"> </span><span class="nx">base_url</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="s2">"api/addresses/"</span><span class="p">,</span><span class="w"> </span><span class="nx">filters</span><span class="p">)</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</code></pre></div>
<p>In our riot tag we'll display some addresses:</p>
<div class="highlight"><pre><span></span><code><span class="p"><</span><span class="nt">addresses</span><span class="p">></span>
<span class="p"><</span><span class="nt">ul</span><span class="p">></span>
<span class="p"><</span><span class="nt">li</span> <span class="na">each</span><span class="o">=</span><span class="s">{</span> <span class="na">addresses</span> <span class="err">}</span><span class="p">></span>
{ name }
{ address_1 }
{ address_2 }
{ city }
{ state }
{ zip_code }
<span class="p"></</span><span class="nt">li</span><span class="p">></span>
<span class="p"></</span><span class="nt">ul</span><span class="p">></span>
var self = this
self.on('mount', function() {
API.get_addresses()
.success(function(data) {
self.update({addresses: data})
})
.error(function() {
alert("Error getting list of addresses!")
})
})
<span class="p"></</span><span class="nt">addresses</span><span class="p">></span>
</code></pre></div>
<p>And in our HTML we'll put the tag:</p>
<div class="highlight"><pre><span></span><code><span class="cm"><!-- somewhere in your template --></span>
<span class="nt"><addresses></addresses></span>
</code></pre></div>
<p>So. Easy. </p>
<p>We can easily extend the address tag to add/remove/modify addresses in beautiful ways, usually based on some events</p>
<div class="highlight"><pre><span></span><code><span class="c1">// somewhere global</span>
<span class="nb">window</span><span class="p">.</span><span class="nx">EVENTS</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nx">riot</span><span class="p">.</span><span class="nx">observable</span><span class="p">()</span>
<span class="c1">// in some tag somewhere else</span>
<span class="p">...</span><span class="w"> </span><span class="nx">make</span><span class="w"> </span><span class="nx">addition</span><span class="w"> </span><span class="nx">to</span><span class="w"> </span><span class="nx">address</span><span class="w"> </span><span class="nx">list</span><span class="w"> </span><span class="p">...</span>
<span class="nb">window</span><span class="p">.</span><span class="nx">EVENTS</span><span class="p">.</span><span class="nx">trigger</span><span class="p">(</span><span class="s1">'force_update_address_list'</span><span class="p">);</span>
<span class="c1">// in your address list</span>
<span class="nb">window</span><span class="p">.</span><span class="nx">EVENTS</span><span class="p">.</span><span class="nx">on</span><span class="p">(</span><span class="s1">'force_update_address_list'</span><span class="p">,</span><span class="w"> </span><span class="nx">do_update_address_list</span><span class="p">);</span>
</code></pre></div>
<p>From an even higher level the approach I've used on most CRUD pages goes like this:</p>
<ol>
<li>Get list of objects, display them in a table/list</li>
<li>Form for new objects or selecting object to populate form</li>
<li>Form creates a new object or updates an existing object</li>
<li>Force the list to update with new/updated object</li>
</ol>
<h3>Why not React?</h3>
<p>I don't like how React forces me to do so many new things, or I feel it does at least. That's not a great or objective
reason. It's similar to how I choose Django + Python over Ruby on Rails, I had a feeling and really enjoyed
one over the other. </p>
<p>When I use RiotJS I don't feel like I'm fighting anything, or that I'm investing hundreds of hours in a library that
the community will abandon in a few months (Angular1 vs Angular2, then leaving for React). It only took a little while to learn
RiotJS stuff and it was fun. I would run into problems, get lots of help from the Riot community and love how the problem
was solved.</p>
<p>The hardest part about finding the right frontend tool is how much information there is out there, and how fragmented the frontend
community seems. I don't have time any more to sort through dozens of tutorials and opinions to find the right path. I'm
looking for little communities centered around similar principles so I don't have to constantly try to find the latest
and greatest way to solve the same CRUD problems we face every day. By the time I learned AngularJS a lot of people
had already jumped ship.</p>
<p style="text-align: center;" class="image-wrapper">
<img src="images/programmer.jpg" class="img-responsive" alt="Smart Programmer">
</p>
<p>Jumping ship may even be worse for backend folks, but from my perspective it seems like backenders aren't switching
tools as often. Maybe backend folks switch between frameworks/paradigms once every 2-4 years, where frontend is about every
6 months. </p>
<p>It's probably just an outsider perspective, maybe it's not as bad as I think. However, I am still using the same Python
tools as when I started Django about 5 years ago. They've grown and changed slightly, but it seems I've only added
library after library instead of replacing one with the other. Maybe for every ReactJS guy there's still some
Backbone/mootool/jquery folks wondering why people left perfectly good tools behind?!</p>
<p>With all of that said: React may be the solution to all the problems. I can't say. I haven't taken the time to really
dive into it past a few toy things and I don't really want to. However, there are way smarter people in the community
than myself and they seem to find React the perfect solution. They're way better frontenders than I am!</p>
<p>React + Babel + some nice CSS libraries are probably the way to go if you're a pure frontender (or just love torture)
but, for a mostly backend guy like me the simplicity and "feel" of Riot is irresistible. Things mostly work how they should
and I don't have to learn a billion new things to do something I could have done using jQuery in 5 seconds.</p>Getting player names from Twitch streams or an exercise in futility2015-12-16T15:10:00+01:002015-12-16T15:10:00+01:00Eric Carmichaeltag:None,2015-12-16:/getting-player-names-from-twitch-streams-or-an-exercise-in-futility.html<p>I had an idea last month for a site based around identifying video game streamers while they are in game. If I can identify them
in real time then I could have real time fantasy league style betting based around the stats... it was a fun experiment but my techniques …</p><p>I had an idea last month for a site based around identifying video game streamers while they are in game. If I can identify them
in real time then I could have real time fantasy league style betting based around the stats... it was a fun experiment but my techniques
ended up not being very practical.</p>
<h2>The Idea</h2>
<p>In the game League of Legends I could try to find the "yellow health bar" that only your character has.
Once I have the coordinates for the yellow health bar I can look above it to see the character name! Boom,
with the character name I can look up stats and track all of their games automatically.</p>
<h2>The process</h2>
<p>For example, watching <a href="http://www.twitch.tv/skumbagkrepo">Krepo's</a> stream here's how I'd go about getting his
character name.</p>
<h3>Get the stream frame</h3>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">livestreamer</span> <span class="kn">import</span> <span class="n">Livestreamer</span>
<span class="k">while</span> <span class="kc">True</span><span class="p">:</span>
<span class="n">session</span> <span class="o">=</span> <span class="n">Livestreamer</span><span class="p">()</span>
<span class="n">streams</span> <span class="o">=</span> <span class="n">session</span><span class="o">.</span><span class="n">streams</span><span class="p">(</span><span class="s1">'http://www.twitch.tv/</span><span class="si">%s</span><span class="s1">'</span> <span class="o">%</span> <span class="n">streamer</span><span class="p">)</span>
<span class="k">if</span> <span class="n">streams</span><span class="p">:</span>
<span class="n">stream</span> <span class="o">=</span> <span class="n">streams</span><span class="p">[</span><span class="s1">'source'</span><span class="p">]</span>
<span class="n">container</span> <span class="o">=</span> <span class="n">av</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="n">stream</span><span class="o">.</span><span class="n">url</span><span class="p">)</span>
<span class="n">video_stream</span> <span class="o">=</span> <span class="nb">next</span><span class="p">(</span><span class="n">s</span> <span class="k">for</span> <span class="n">s</span> <span class="ow">in</span> <span class="n">container</span><span class="o">.</span><span class="n">streams</span> <span class="k">if</span> <span class="n">s</span><span class="o">.</span><span class="n">type</span> <span class="o">==</span> <span class="sa">b</span><span class="s1">'video'</span><span class="p">)</span>
<span class="n">image</span> <span class="o">=</span> <span class="kc">None</span>
<span class="k">for</span> <span class="n">packet</span> <span class="ow">in</span> <span class="n">container</span><span class="o">.</span><span class="n">demux</span><span class="p">(</span><span class="n">video_stream</span><span class="p">):</span>
<span class="k">for</span> <span class="n">frame</span> <span class="ow">in</span> <span class="n">packet</span><span class="o">.</span><span class="n">decode</span><span class="p">():</span>
<span class="n">image</span> <span class="o">=</span> <span class="n">frame</span><span class="o">.</span><span class="n">to_image</span><span class="p">()</span>
<span class="n">features</span> <span class="o">=</span> <span class="n">process_image</span><span class="p">(</span><span class="n">image</span><span class="p">)</span>
</code></pre></div>
<p style="text-align: center;" class="image-wrapper">
<img src="images/twitch_stream_ocr/original_stream_image.jpg" alt="Original Stream Image" class="image-full-width"><br>
</p>
<h3>Find yellow health bar</h3>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">_find_our_champion</span><span class="p">(</span><span class="n">image</span><span class="p">,</span> <span class="n">search_step</span><span class="o">=</span><span class="mi">2</span><span class="p">,</span> <span class="n">draw_on_image</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
<span class="c1"># should find the yellow bar underneath our hero and then </span>
<span class="c1"># we can use the top left of that to center our search area</span>
<span class="n">data</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">asarray</span><span class="p">(</span><span class="n">image</span><span class="p">)</span>
<span class="c1"># remove any alpha channel in the data so we just have (r, g, b)</span>
<span class="n">data</span> <span class="o">=</span> <span class="n">data</span><span class="p">[:,</span> <span class="p">:,</span> <span class="p">:</span><span class="mi">3</span><span class="p">]</span>
<span class="n">width</span><span class="p">,</span> <span class="n">height</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">data</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="n">character_name_coords</span> <span class="o">=</span> <span class="kc">None</span>
<span class="c1"># Cut off the bottom few hundred pixels, not needed</span>
<span class="n">height</span> <span class="o">-=</span> <span class="mi">200</span>
<span class="k">for</span> <span class="n">y</span> <span class="ow">in</span> <span class="n">xrange</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="n">height</span><span class="p">):</span>
<span class="n">hits_this_row</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">xrange</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="n">width</span><span class="p">,</span> <span class="n">search_step</span><span class="p">):</span>
<span class="n">r</span><span class="p">,</span> <span class="n">g</span><span class="p">,</span> <span class="n">b</span> <span class="o">=</span> <span class="n">data</span><span class="p">[</span><span class="n">y</span><span class="p">][</span><span class="n">x</span><span class="p">]</span>
<span class="k">if</span> <span class="n">r</span> <span class="o">></span> <span class="mi">200</span> <span class="ow">and</span> <span class="mi">160</span> <span class="o"><</span> <span class="n">g</span> <span class="o"><</span> <span class="mi">230</span> <span class="ow">and</span> <span class="mi">30</span> <span class="o"><</span> <span class="n">b</span> <span class="o"><</span> <span class="mi">70</span><span class="p">:</span>
<span class="c1"># really yellow</span>
<span class="n">hits_this_row</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="k">if</span> <span class="n">hits_this_row</span> <span class="o">></span> <span class="mi">20</span><span class="p">:</span>
<span class="c1"># Top left and bottom right</span>
<span class="n">character_name_coords</span> <span class="o">=</span> <span class="p">(</span><span class="n">x</span> <span class="o">-</span> <span class="mi">120</span><span class="p">,</span> <span class="n">y</span> <span class="o">-</span> <span class="mi">35</span><span class="p">,</span> <span class="n">x</span> <span class="o">+</span> <span class="mi">100</span><span class="p">,</span> <span class="n">y</span> <span class="o">-</span> <span class="mi">8</span><span class="p">)</span>
<span class="k">if</span> <span class="n">draw_on_image</span><span class="p">:</span>
<span class="n">red</span> <span class="o">=</span> <span class="p">(</span><span class="mi">255</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
<span class="n">draw</span> <span class="o">=</span> <span class="n">ImageDraw</span><span class="o">.</span><span class="n">Draw</span><span class="p">(</span><span class="n">image</span><span class="p">)</span>
<span class="n">draw</span><span class="o">.</span><span class="n">rectangle</span><span class="p">(</span><span class="n">character_name_coords</span><span class="p">,</span> <span class="n">fill</span><span class="o">=</span><span class="n">red</span><span class="p">)</span>
<span class="k">break</span>
<span class="k">if</span> <span class="n">character_name_coords</span><span class="p">:</span>
<span class="k">break</span>
<span class="k">return</span> <span class="n">character_name_coords</span>
</code></pre></div>
<p style="text-align: center;" class="image-wrapper">
<img src="images/twitch_stream_ocr/character_name_image_1.png" alt="Character name image" class="image-full-width"><br>
</p>
<h3>Process the image</h3>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">_ocr_name_box</span><span class="p">(</span><span class="n">name_box_image</span><span class="p">):</span>
<span class="c1"># 0 means load in grayscale</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">gray</span> <span class="o">=</span> <span class="n">cv2</span><span class="o">.</span><span class="n">cvtColor</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">(</span><span class="n">name_box_image</span><span class="p">),</span> <span class="mi">0</span><span class="p">)</span>
<span class="k">except</span> <span class="ne">TypeError</span><span class="p">:</span>
<span class="n">gray</span> <span class="o">=</span> <span class="n">cv2</span><span class="o">.</span><span class="n">cvtColor</span><span class="p">(</span><span class="n">name_box_image</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
<span class="n">ret</span><span class="p">,</span> <span class="n">gray</span> <span class="o">=</span> <span class="n">cv2</span><span class="o">.</span><span class="n">threshold</span><span class="p">(</span><span class="n">gray</span><span class="p">,</span> <span class="mi">160</span><span class="p">,</span> <span class="mi">255</span><span class="p">,</span> <span class="n">cv2</span><span class="o">.</span><span class="n">THRESH_BINARY</span><span class="p">)</span>
<span class="c1"># ocr code below</span>
</code></pre></div>
<p style="text-align: center;" class="image-wrapper">
<img src="images/twitch_stream_ocr/character_name_image_processed_1.png" alt="Character name image processed" class="image-full-width"><br>
</p>
<h3>OCR the final processed image</h3>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">_ocr_name_box</span><span class="p">(</span><span class="n">name_box_image</span><span class="p">):</span>
<span class="c1"># processing code above</span>
<span class="n">gray_image</span> <span class="o">=</span> <span class="n">Image</span><span class="o">.</span><span class="n">fromarray</span><span class="p">(</span><span class="n">gray</span><span class="p">)</span>
<span class="k">return</span> <span class="n">pytesseract</span><span class="o">.</span><span class="n">image_to_string</span><span class="p">(</span><span class="n">gray_image</span><span class="p">)</span>
</code></pre></div>
<p>OCR results: Krepo</p>
<p>So, it worked in this particular case... this was one of the few that did work.</p>
<h2>The results</h2>
<p><strong>NOT GOOD!</strong></p>
<p>The idea was super fun to pursue but it doesn't perform very well for a few reasons:</p>
<ul>
<li>Streamers can disable showing character name</li>
<li>The "yellow health bar" specific to your character can be disabled</li>
<li>What region are players on?</li>
<li><a href="https://pypi.python.org/pypi/pytesseract/">PyTesseract</a> is pretty cool but <a href="https://github.com/tesseract-ocr">Tesseract</a> is pretty cumbersome to use. Hard to train, seems kind of outdated but definitely decent for being free!</li>
<li>The OCR results were pretty bad although I'm sure with mechanical turk + some training we could get it in a decent state, but still.. names could not be shown at all!</li>
</ul>
<p>So... it was a fun experiment but not very practical!</p>
<p>You can find all the source <a href="https://github.com/ckcollab/twitch-experiments">here</a>.</p>Transitioning from helpless to helpful2015-06-09T15:10:00+02:002015-06-09T15:10:00+02:00Eric Carmichaeltag:None,2015-06-09:/transitioning-from-helpless-to-helpful.html<p>Four years ago I had no idea what I was doing in Web Development. Asking dumb questions in IRC all day was how I learned, scrounging through tutorials trying to make sense of <i>anything</i>. Now I'm the one answering the dumb questions!</p><p>Four years ago I had no idea what I was doing in Web Development. Asking dumb questions in IRC all day was how I
learned. Scrounging through tutorials trying to make sense of <i>anything</i>. Now I'm the one answering the dumb
questions!</p>
<p>I felt like writing this because I've just realized that I now hang out in <a href="irc://irc.freenode.net/django">#django</a>
and actually spend most of my time giving back instead of taking. To me that's a huge achievement that deserves some
attention, at least.</p>
<p style="text-align: center;" class="image-wrapper">
<img src="images/im_helping.jpg" alt="I'm helping!"><br>
</p>
<p>It's not uncommon that I leave the IRC window up on my 3rd monitor and glance at it occasionally. More and more I'm
seeing people ask questions that I have no problem answering... what the hell! Without even realizing it I've
transitioned from an entry-level-nobody to kinda-somebody-worth-listening-to.</p>
<h2>Being a newb</h2>
<p>When I was just getting started I vividly remember being helped by <code>FunkyBob</code> multiple times... I had no idea why he was
helping me. Maybe now I'm finally starting to understand why he would helped me, but a few years ago I had no clue. I've
bought pizza/beer for a couple people and such, but that's not the <em>right</em> way to give back.</p>
<p>The real way to say thanks would be to, myself, become able to give back to the community. </p>
<h2>The transition</h2>
<p>I'm not sure when the switch flipped, but my best guess is recently with changes at work I am seeking out more and more
ways to be proud of what I do. Instead of just getting the work done, I take pride in taking my time implementing a
better solution. Not something perfect, but something I'm proud of.</p>
<p>I've also made horrendous fuck ups. Huge security holes, missed details, and poorly designed features. Though these seem
like awful things, this where my growth seems to be happening. By being forced to confront these problems without the
option to give up/change course: I am learning a fuck load.</p>
<p>It's regular to get stuck for hours on something simple like forgetting to pass <code>*args, **kwargs</code> to a <code>super</code> method,
but in those painful hours I grow and stretch my brain around the problem. When I finally understand <em>just how stupid
I am</em> with the small mistake, it's like taking a load off my shoulders.</p>
<p>Sometimes it feels like I have no idea how I'll get to the finish line on a feature. That's getting more rare. My
confidence is growing that my weak points are shrinking. In the way I communicate especially.</p>
<h2>Good for you!</h2>
<p>I know this is kinda lame congratulating myself... but I'm finally getting happier doing web development, damnit! Our
job is filled with negativity, set backs, and pressure--so why do we do it? Of course money is there, but that won't keep
me around. </p>
<p>What will keep me around is this:</p>
<div class="highlight"><pre><span></span><code>rpkilby: thanks for the help
xcyclist: Yes, that did it. Thanks.
shutupsquare: Hey Thanks, I never even thought to have the items have a fk to the person, perfect, thanks guys.
</code></pre></div>Top 5 gaming documentaries of all time2015-05-11T15:10:00+02:002015-05-11T15:10:00+02:00Eric Carmichaeltag:None,2015-05-11:/top-5-gaming-documentaries-of-all-time.html<p>The best of gaming competitiveness, attitude, culture and raw skill. To me these are the documentaries I'll look back at like my dad looks back at Superbowl 1. <p style="text-align: center;" class="image-wrapper"><img src="images/top_5_gaming_docs/brood_war.jpg" alt="StarCraft: Brood War"></p></p><p style="text-align: center;" class="image-wrapper">
<img src="images/top_5_gaming_docs/brood_war.jpg" alt="StarCraft: Brood War"><br>
</p>
<p>Here's the scenario: I am 15 years old playing <a href="http://us.blizzard.com/en-us/games/sc/">StarCraft: Brood War</a> against someone named "crapball." Ha! What a funny
name, should be a quick game... no, he beat me! </p>
<p>I begged him for a rematch and thankfully he oblidged. He beat me again. Getting a little upset... I beg him a second time for a rematch, but this
time I won't make such silly mistakes.</p>
<p>He crushed me. We were playing on Lost Temple and he cheesed me by building a pylon where I would put my supply
depot on the ramp. Shit now I can't wall off... what!? Now he's building cannons... that son of a bitch!</p>
<p>Something about how he was playing was beautiful,
I almost beat him every game but he had such clever solutions to problems I thought were insurmountable. I checked the
replays and he was doing hundreds of actions a minute compared to my dozens.</p>
<p>Something about losing to "crapball" pushed me to keep trying and to find my own style, to be as aggressive as he was
and to make people react to me instead of being the reactionary one. To me, there's as much beauty in these struggles as
with any physical sport. It takes a tremendous amount of determination to succeed at anything, but to be the best
requires personal sacrifice as well.</p>
<h2>#1</h2>
<iframe
width="640"
height="360"
class="youtube"
src="https://www.youtube-nocookie.com/embed/NJztfsXKcPQ"
frameborder="0"
allowfullscreen>
</iframe>
<p class="centered-label">
<i>"My Life of Starcraft"</i>
</p>
<h3>My Life of Starcraft - Day[9] Daily #100</h3>
<p><a href="http://day9.tv/">Sean "Day[9]" Plott</a> walks us through his best and worst experiences in his competitive StarCraft days.
It's tear filled and makes me feel old, I remember seeing "Day9" in the World Cyber Games and rooting against him!
Very powerful stuff in this video blog.</p>
<p>Nowadays Sean is doing his daily video blogs, some awesome Magic The Gathering games with celebrities called Spell Slingers,
and always promoting a positive attitude.</p>
<h2>#2</h2>
<iframe
width="640"
height="360"
class="youtube"
src="https://www.youtube-nocookie.com/embed/xMJZ-_bJKdI"
frameborder="0"
allowfullscreen>
</iframe>
<p class="centered-label">
<i>"The King of Kong: Fistful of Quarters"</i>
</p>
<h3>King of Kong</h3>
<p><a href="http://www.imdb.com/title/tt0923752/">King of Kong</a> is the passionate story of Steve Wiebe trying to crush the arrogant and
doofy Billy Mitchell at the classic arcade game Donkey Kong. This shows better than most docs all of the work, emotional pain,
and disappointment we've all faced. Whether it's dealing with our own problems or having a bully knock us down. </p>
<h2>#3</h2>
<iframe
width="640"
height="360"
class="youtube"
src="https://www.youtube-nocookie.com/embed/jX9hbbA-WP4"
frameborder="0"
allowfullscreen>
</iframe>
<p class="centered-label">
<i>"The Smash Brothers Documentary"</i>
</p>
<h3>The Smash Brothers Documentary</h3>
<p><a href="https://www.youtube.com/watch?v=jX9hbbA-WP4">The Smash Brothers Documentary</a> details all of the top players throughout the
history of Smash Bros. It's pretty interesting watching people start the scene, become dominant, get toppled by newcomers
they destroyed earlier, and then come back for dramatic close wins.</p>
<h2>#4</h2>
<iframe
width="640"
height="360"
class="youtube"
src="https://www.youtube-nocookie.com/embed/xyjS0Xdzrlc"
frameborder="0"
allowfullscreen>
</iframe>
<p class="centered-label">
<i>"Word Wars"</i>
</p>
<h3>Word Wars</h3>
<p><a href="http://www.imdb.com/title/tt0390632/">Word Wars</a> is a movie about scrabble, which I am <em>kind of</em> stretching to fit with the rest
of these, but the theme is the same: hard work leads to incredible things. No matter what that thing is, I can respect
hard work and tremendous sacrifice.</p>
<h2>#5</h2>
<iframe
width="640"
height="360"
class="youtube"
src="https://www.youtube-nocookie.com/embed/UjZYMI1zB9s"
frameborder="0"
allowfullscreen>
</iframe>
<p class="centered-label">
<i>"Free to Play: The Movie (US)"</i>
</p>
<h3>Free to Play: The Movie (US)</h3>
<p><a href="https://www.youtube.com/watch?v=UjZYMI1zB9s&t=1244">Free to Play</a> a very professional documentary about the recent developments in the
Dota 2 competitive scene. Millions of dollars are on the line!</p>Debug running process on Linux2014-08-28T15:10:00+02:002014-08-28T15:10:00+02:00Eric Carmichaeltag:None,2014-08-28:/debug-running-process-on-linux.html<p>I am working on putting out a serious fire now. Gettin' into that weird head space where you grow as a developer,
the place where all hair is pulled out, desks are slammed, "WHAAT!?" is screamed every 45 minutes, but you
can't really force yourself to take a break.</p>
<p>I …</p><p>I am working on putting out a serious fire now. Gettin' into that weird head space where you grow as a developer,
the place where all hair is pulled out, desks are slammed, "WHAAT!?" is screamed every 45 minutes, but you
can't really force yourself to take a break.</p>
<p>I need to make sure this worker is running properly and the <code>stdout</code> output is all I need to get to the juicy debug info.</p>
<p>First I'll get the process ID</p>
<div class="highlight"><pre><span></span><code>> ps -ef | grep worker
1000 2778 2308 0 Aug28 ? 00:00:03 /home/supertastic/worker.py
</code></pre></div>
<p>The process id in this case is <code>2778</code>, now let's use this snippet to watch on <code>-p 2778</code></p>
<div class="highlight"><pre><span></span><code><span class="p">></span><span class="w"> </span><span class="nx">sudo</span><span class="w"> </span><span class="nx">strace</span><span class="w"> </span><span class="o">-</span><span class="nx">f</span><span class="w"> </span><span class="o">-</span><span class="nx">e</span><span class="w"> </span><span class="nx">trace</span><span class="p">=</span><span class="nx">write</span><span class="w"> </span><span class="o">-</span><span class="nx">e</span><span class="w"> </span><span class="nx">verbose</span><span class="p">=</span><span class="nx">none</span><span class="w"> </span><span class="o">-</span><span class="nx">e</span><span class="w"> </span><span class="nx">write</span><span class="p">=</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="w"> </span><span class="o">-</span><span class="nx">q</span><span class="w"> </span><span class="o">-</span><span class="nx">p</span><span class="w"> </span><span class="mi">2778</span><span class="w"> </span><span class="o">-</span><span class="nx">o</span><span class="w"> </span><span class="s">"| grep '^ |' | cut -c11-60 | sed -e 's/ //g' | xxd -r -p"</span>
<span class="mi">2014</span><span class="o">-</span><span class="mi">08</span><span class="o">-</span><span class="mi">29</span><span class="w"> </span><span class="mi">02</span><span class="p">:</span><span class="mi">14</span><span class="p">:</span><span class="mi">09</span><span class="p">,</span><span class="mi">459</span><span class="w"> </span><span class="nx">DEBUG</span><span class="w"> </span><span class="nx">Received</span><span class="w"> </span><span class="nx">message</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="s">"task_type"</span><span class="p">:</span><span class="w"> </span><span class="s">"evaluate_submission"</span><span class="p">}</span>
<span class="mi">2014</span><span class="o">-</span><span class="mi">08</span><span class="o">-</span><span class="mi">29</span><span class="w"> </span><span class="mi">02</span><span class="p">:</span><span class="mi">14</span><span class="p">:</span><span class="mi">09</span><span class="p">,</span><span class="mi">460</span><span class="w"> </span><span class="nx">INFO</span><span class="w"> </span><span class="nx">Running</span><span class="w"> </span><span class="nx">task</span><span class="p">:</span><span class="w"> </span><span class="nx">id</span><span class="p">=</span><span class="mi">10131</span><span class="w"> </span><span class="nx">task_type</span><span class="p">=</span><span class="nx">evaluate_submission</span>
<span class="mi">2014</span><span class="o">-</span><span class="mi">08</span><span class="o">-</span><span class="mi">29</span><span class="w"> </span><span class="mi">02</span><span class="p">:</span><span class="mi">14</span><span class="p">:</span><span class="mi">09</span><span class="p">,</span><span class="mi">461</span><span class="w"> </span><span class="nx">DEBUG</span><span class="w"> </span><span class="nx">evaluate_submission_task</span><span class="w"> </span><span class="nx">begins</span><span class="w"> </span><span class="p">(</span><span class="nx">job_id</span><span class="p">=</span><span class="mi">10131</span><span class="p">)</span>
<span class="mi">2014</span><span class="o">-</span><span class="mi">08</span><span class="o">-</span><span class="mi">29</span><span class="w"> </span><span class="mi">02</span><span class="p">:</span><span class="mi">14</span><span class="p">:</span><span class="mi">09</span><span class="p">,</span><span class="mi">462</span><span class="w"> </span><span class="nx">DEBUG</span><span class="w"> </span><span class="nx">evaluate_submission_task</span><span class="w"> </span><span class="nx">submission_id</span><span class="p">=</span><span class="mi">9481</span><span class="w"> </span><span class="p">(</span><span class="nx">job_id</span><span class="p">=</span><span class="mi">10131</span><span class="p">)</span>
</code></pre></div>
<p>Thanks to <a href="http://stackoverflow.com/questions/249703/how-can-a-process-intercept-stdout-and-stderr-of-another-process-on-linux">Lari Hotari</a> for giving everyone a neat solution.</p>Best Pycon 2014 Videos2014-05-19T15:10:00+02:002014-05-19T15:10:00+02:00Eric Carmichaeltag:None,2014-05-19:/best-pycon-2014-videos.html<p>I have been binge watching Pycon 2014 videos and I thought I might as well make that productive somehow, so here's a list of the best videos (in my <em>very humble</em> opinion)!</p><p>I have been binge watching Pycon 2014 videos and I thought I might as well make that productive somehow, so here's a list of
the best videos (in my <em>very humble</em> opinion)!</p>
<p>I haven't watched every single Pycon video so please be sure to recommend your favorites in the comments!</p>
<h2>Python Macgyvering</h2>
<p><a href="https://twitter.com/dabeaz">David Beazley</a> slays it. As an expert witness he writes his own versions of simple things and comes up with
interesting ways to reverse engineering/research a huge codebase.</p>
<iframe width="640" height="360" class="youtube" src="https://www.youtube-nocookie.com/embed/RZ4Sn-Y7AP8?rel=0" frameborder="0" allowfullscreen></iframe>
<p class="centered-label">
<i>"Discovering Python"</i>
</p>
<h2>Imposter Syndrome</h2>
<p><a href="https://twitter.com/juliepagano">Julie Pagano</a> reminds us to maybe not be so hard on ourselves. This one is especially
important to me, because I am starting a new job soon and there's a large part of me that feels like I may not fit in.</p>
<p>That is <strong>bull shit!</strong></p>
<p>I don't know why we get these strange fears. Everyone has that awkward first day but I'm sure at my new job the people
are looking out for my best interests. They want me to be happy and feel welcomed, so I just need to think positive
and do my best—no reason to worry!</p>
<iframe width="640" height="360" class="youtube" src="https://www.youtube-nocookie.com/embed/1i8ylq4j_EY?rel=0" frameborder="0" allowfullscreen></iframe>
<p class="centered-label">
<i>"It's Dangerous to Go Alone: Battling the Invisible Monsters in Tech - PyCon 2014"</i>
</p>
<h2>Helicopter Badassery</h2>
<p><a href="https://github.com/njl">Ned Jackson Lovely</a> flies an awesome <a href="http://www.amazon.com/Syma-S107G-Channel-Helicopter-Gyro/dp/B00F4WMAI4/ref=lp_166591011_1_1?s=toys-and-games&ie=UTF8&qid=1400608454&sr=1-1">little helicopter</a>
around with python! Imagine flying this into your family and friends via iPython, what joy!</p>
<p>This also goes into quite a bit of arduino stuff which is something I am trying to get into, pretty interesting.</p>
<iframe width="640" height="360" class="youtube" src="https://www.youtube-nocookie.com/embed/jJ1vjr1dDqw?rel=0" frameborder="0" allowfullscreen></iframe>
<p class="centered-label">
<i>"Cheap Helicopters In My Living Room"</i>
</p>
<h2>Memcached ins-and-outs</h2>
<p><a href="https://twitter.com/gardaud">Guillaume Ardaud</a> goes into some great specific about memcached, a lot of stuff I didn't know--like you pronounce
memcached "memcache dee"!</p>
<p>I think this guy is a pretty good speaker to model yourself after. He was confident and obviously practiced a lot beforehand,
really enjoyed this presentation.</p>
<iframe width="640" height="360" class="youtube" src="https://www.youtube-nocookie.com/embed/R8Xmeynf1T4?rel=0" frameborder="0" allowfullscreen></iframe>
<p class="centered-label">
<i>"Cache me if you can: memcached, caching patterns and best practices"</i>
</p>
<h2>Machine learning for personalized Hacker News</h2>
<p><a href="https://twitter.com/nedjl">Ned Jackson Lovely</a> did an awesome presentation on machine learning/sciki. It was very beginner
friendly. My favorite part was probably the flow chart "if you have less than 50 samples, get more samples."</p>
<iframe width="640" height="360" class="youtube" src="https://www.youtube-nocookie.com/embed/O7IezJT9uSI?rel=0" frameborder="0" allowfullscreen></iframe>
<p class="centered-label">
<i>"Enough Machine Learning to Make Hacker News Readable Again"</i>
</p>
<h2>Computer Science education in the US</h2>
<p><a href="https://twitter.com/jessicamckellar">Jessica McKellar</a> talks about how we could get more kids involved with computer science.
What I thought stood out about her presentation was concrete ideas/details, like:</p>
<ul>
<li>Make Computer Science count as a Math/Sci credit instead of being an elective -- encourages teachers and students to take it more seriously</li>
<li>Switch from Java to Python—don't throw people into the deep end of OOP, etc.</li>
<li>Call your local school board/legislature and talk about the above simple steps.</li>
</ul>
<iframe width="640" height="360" class="youtube" src="https://www.youtube-nocookie.com/embed/4QOoAw6Su7M?rel=0" frameborder="0" allowfullscreen></iframe>
<p class="centered-label">
<i>"Keynote"</i>
</p>Chin Up motivation tracker2014-05-04T15:10:00+02:002014-05-04T15:10:00+02:00Eric Carmichaeltag:None,2014-05-04:/chin-up-motivation-tracker.html<p><a href="https://github.com/ckcollab/chin-up">Chin Up</a> is my personal motivation tracker. Every day I rate from 1 to 10 how I am
feeling in certain metrics like strength, flexibility, relationships and happiness.</p>
<p class="image-wrapper">
<img src="images/chinup/home_view.png" alt="Home View of Chin Up" class="image-full-width">
</p>
<p>There are two types of metrics:</p>
<ul>
<li><strong>Daily</strong> - Measured every day</li>
<li><strong>Monthly</strong> - Measured on the first of every month</li>
</ul>
<p>It is meant to …</p><p><a href="https://github.com/ckcollab/chin-up">Chin Up</a> is my personal motivation tracker. Every day I rate from 1 to 10 how I am
feeling in certain metrics like strength, flexibility, relationships and happiness.</p>
<p class="image-wrapper">
<img src="images/chinup/home_view.png" alt="Home View of Chin Up" class="image-full-width">
</p>
<p>There are two types of metrics:</p>
<ul>
<li><strong>Daily</strong> - Measured every day</li>
<li><strong>Monthly</strong> - Measured on the first of every month</li>
</ul>
<p>It is meant to be hosted locally/somewhere free, like <a href="http://heroku.com">Heroku</a>.</p>
<p style="text-align: center;" class="image-wrapper">
<img src="images/chinup/input.png" alt="Input data every day to Chin Up" class="image-full-width">
</p>
<h2>What I track</h2>
<p>If you want the same setup I have, go to the <code>admin</code> screen and add these items.</p>
<h3>Daily checklist (binary=True)</h3>
<ul>
<li>Vitamins</li>
<li>Coffee</li>
<li>20 pushups for 3 sets</li>
<li>20 squats for 3 sets</li>
<li>8 pull ups for 3 sets</li>
<li>(a lot more, this depends entirely on what you do/want to do every day)</li>
</ul>
<h3>Daily tracking</h3>
<ul>
<li>Happy<ul>
<li>1 being seeing my mom with tubes coming out of her face and head</li>
<li>10 being happiest I've ever been, nothing feels impossible</li>
</ul>
</li>
<li>Motivated<ul>
<li>1 being ultra depressed not even eating</li>
<li>10 being I did everything I needed to and more</li>
</ul>
</li>
<li>Flexibility<ul>
<li>1 being no stretching in chair all day no walking around</li>
<li>10 being I did every stretch possible and did the splits</li>
</ul>
</li>
<li>Strength<ul>
<li>1 being I feel as soft as an 8 year old girl</li>
<li>10 being I squatted and deadlifted the day before--now I can't move</li>
</ul>
</li>
<li>Endurance<ul>
<li>1 being I got winded getting out of my chair</li>
<li>10 being I jump roped for 10 minutes and ran a mile without breathing hard</li>
</ul>
</li>
<li>Relationship<ul>
<li>1 being I got into a huge argument and feel terrible</li>
<li>10 being we made dinner together and wrote each other poetry</li>
</ul>
</li>
</ul>
<h3>Monthly tracking</h3>
<ul>
<li>Life goals<ul>
<li>1 being no progress towards goals and very depressed</li>
<li>10 being I got a world record deadlift</li>
</ul>
</li>
<li>Power<ul>
<li>1 being no physical activity</li>
<li>10 being I ran and worked out every opportunity I had</li>
</ul>
</li>
<li>Nature<ul>
<li>1 being no outdoor activity</li>
<li>10 being I spent the entire time camping and fishing</li>
</ul>
</li>
</ul>
<h2>Why?</h2>
<p>To become the best I can be: I need motivation, I need to get my chin up.</p>
<h2>How do you measure?</h2>
<p>Every metric that I believe significantly effects my motivation and health will be measured on a scale from 1 to 10.
If there is something significant to note, I'll put that down as well. However, the main goal is to motivate myself through
numbers and statistics.</p>
<p><strong>1 being</strong> the worst, least effort I could possibly put in and <strong>10 being</strong> the best, most effort I could possibly put in.</p>
<h2>Ease of use</h2>
<p style="text-align: center;" class="image-wrapper">
<img src="images/chinup/pin_passcode.png" alt="django-pin-passcode screen">
</p>
<p>To make it easier to use, I made a simple app and threw it on pypi called <a href="https://github.com/ckcollab/django-pin-passcode">django-pin-passcode</a>.
It logs you in to the user you define after you enter a certain passcode composed of 0-9 and #.</p>
<h2>Stats</h2>
<p style="text-align: center;" class="image-wrapper">
<img src="images/chinup/stats.png" alt="highchart stats" class="image-full-width">
</p>
<ul>
<li><strong>Month to month</strong> - Average for the entire month</li>
<li><strong>Week to week</strong> - Average for the entire week</li>
<li><strong>Day of the week</strong> - Average for that day of the week for the last year</li>
<li><strong>7 day</strong> - Last 7 day simple</li>
<li><strong>30 day</strong> - Last 30 day simple</li>
</ul>
<h2>Correlations</h2>
<p style="text-align: center;" class="image-wrapper">
<img src="images/chinup/correlations.png" alt="Correlations" class="image-full-width">
</p>
<p>Thanks to numpy I added really quick correlations between metrics, it's pretty ugly but it will tell you what metrics
might positively/negatively correlate to each other.</p>Python Interview Tests2014-05-01T15:10:00+02:002014-05-01T15:10:00+02:00Eric Carmichaeltag:None,2014-05-01:/python-interview-tests.html<p>I have been applying to a lot of jobs lately and I've done quite a few "prove you know what's up" tests and projects. I think it might be a good exercise to go over them in my head and talk about how everyone could win from these tests. <br><br>Note: this write up is meant for <strong>remote</strong> development work!</p><p>I have been applying to a lot of jobs lately and I've done quite a few "prove you know what's up" tests and projects. I
think it might be a good exercise to go over them in my head and talk about how everyone could win from these tests.</p>
<p>Note: this write up is meant for <strong>remote</strong> development work!</p>
<h2>When are employment tests good?</h2>
<p>I think tests for entry level/junior positions are necessary to prove:</p>
<ul>
<li>Understanding of fundamentals (list, tuple, etc.)</li>
<li>Experience with testing (nose, pywebtest, selenium)</li>
<li>Experience with an ORM/SQL</li>
<li>They are a self-starter and motivated</li>
</ul>
<h2>When are employment tests not as helpful?</h2>
<p>If the position you are trying to fill is for an expert or senior level position that applicant should already have ample code samples and
open source contributions to go through. I don't think it's necessary to check for fundamentals in this situation. It'd
be more important to me that the person is a good fit personality wise for the team.</p>
<p>For a senior position it might be more valuable to give them a problem and ask how they would organize a team to solve it?
Not sure, I'm still a Junior!</p>
<p style="text-align: center;" class="image-wrapper">
<img src="images/interview_tests/twoheads.png" alt="Two heads are better than one"><br>
<i><small><a href="http://www.jisc.ac.uk/blog/identifying-resources-for-students-by-students-03-may-2013">Photo by JISC</a></small></i>
</p>
<h2>What are useful takeaways?</h2>
<p>A good employment test should give the interviewee an opportunity to showcase their skillset.</p>
<p>For example, if testing someone for a Django/Python environment:</p>
<ul>
<li>They understand the WSGI-way of building web apps, django views vs class based views, mixins, etc.</li>
<li>They should be able to write unit tests and functional tests with selenium</li>
<li>They should be able to do queries with the ORM and build models</li>
<li>Given very specific instructions, they should be able to finish the test in a reasonable amount of time.</li>
<li>Give opportunities to showcase something interesting</li>
</ul>
<h3>A good experience I had</h3>
<p>In one test I made up a set of models and as a bonus the employers asked me to write up a strategy to test the models.</p>
<p>Not only does this give me an opportunity to strut-my-stuff, but it makes the problem just a little more interesting.
I started to open up my brain and think of ways to test the code which made me think more clearly about the overall
architecture—and it was fun!</p>
<h3>A bad experience I had</h3>
<p>If your interviewee is completely baffled by the test because it's so specific and they could solve the problem using their
own tools: that is a bad experience for everyone. Valuable time is soaked up during the test Googling how to use library
X and framework Y.</p>
<p>Instead, the applicant should be encouraged to use whatever strategy they see fit to solve the problem
and then they should be tasked with explaining <em>why</em> they chose that route. Often times when you meet someonew new,
they have a few tricks up their sleeve you may have never seen before!</p>
<h3>Another bad experience I had</h3>
<p>The test was super vague, as if the person who wrote the test didn't care much about the end result. It bothered me
that there was a one paragraph explanation and so much openness. No real time limit, no concrete goals. Also, when I asked
for more details it took almost a week to get a response. The lack of response made me lose interest in that position.</p>
<p style="text-align: center;" class="image-wrapper">
<img src="images/interview_tests/coder.jpg" alt="Coder" class="image-full-width"><br>
<i><small><a href="http://commons.wikimedia.org/wiki/User:Matthew_(WMF)">Photo by Matthew Roth</a></small></i>
</p>
<h2>What are less useful takeaways?</h2>
<ul>
<li>They know how to do some obscure binary arithmetic.</li>
<li>Large chunks of the test not entirely related with the position's description, like applying for a Python position but
being tested on ASP.</li>
<li>Solving a problem within very particular restrictions.</li>
</ul>
<h2>When testing an employee for remote work</h2>
<h3>Do</h3>
<ul>
<li>Be super organized and descriptive.</li>
<li>Leave the interviewee alone, only a few questions should be asked. If you are too available it might get the interviewee
out of the creative headspace and into the follow-the-leader headspace.</li>
<li>Test internally first, if no one else can pass your test then it may be too obscure.</li>
<li>Take as many parts of setup out of the equation as you can: git vs mercurial, database fixtures. Allow the interviewee
some latitude to solve problems.</li>
</ul>
<h3>Don't</h3>
<ul>
<li>Change/improve instructions during testing, it breaks focus and uses up valuable nervous energy.</li>
<li>Put off answering interviewee questions.</li>
<li>Put short time limits, give them a way to prove speed but at the same time architect something nice.</li>
</ul>Coeur d Alene Web Developer2014-04-28T15:10:00+02:002014-04-28T15:10:00+02:00Eric Carmichaeltag:None,2014-04-28:/coeur-d-alene-web-developer.html<p>I thought I'd take a moment to talk about what I do and where I do it, here it goes!</p>
<h2>Why web development?</h2>
<p>I've always loved solving problems. In high school one of my <em>hobbies</em>, if you can believe it, was helping out the teachers
by making tools for grading …</p><p>I thought I'd take a moment to talk about what I do and where I do it, here it goes!</p>
<h2>Why web development?</h2>
<p>I've always loved solving problems. In high school one of my <em>hobbies</em>, if you can believe it, was helping out the teachers
by making tools for grading, quizzes and studying.</p>
<p>In college my tastes drifted away from Computer Science. I switched to Education and my plan in life
was to become a teacher. Everything changed when my mother was in a terrible motorcycle accident. She came out of a coma with
a 1% chance to survive, but she suffered massive brain damage.</p>
<p>I dropped out of school and moved home to help, but I needed some way to make a living with a flexible schedule so I could
stay with my mum. I picked back up web development and here I am today about three years later.</p>
<h2>Why North Idaho?</h2>
<p style="text-align: center;" class="image-wrapper">
<img src="images/cda_web_dev/killarney_lake.jpg" alt="Killarney Lake" class="image-full-width"><br>
<i><small>Killarney Lake</small></i>
</p>
<p>It's beautiful here! I want to raise my family in this great community with safe schools and lots of outdoor activities
to do. Recently I have gotten into Kayak Fishing and North Idaho is the place to do it.</p>
<p>Also there is so much around here for great little one day vacations like the <a href="http://www.davenporthotelcollection.com/">Davenport</a>
in Spokane with dinner at <a href="http://www.mainsushi.com/">Sushi.com</a>. Or, if you're into French food, one of my other favorite
restaurants is in Post Falls: <a href="http://fleur-de-sel.weebly.com/">Fleur de Sel</a>—forgive them for their website!</p>
<h2>#represent</h2>
<p>That's how I ended up in web development and why I love CDA!</p>Ignore herokuapp.com domain in nginx2014-04-23T01:50:00+02:002014-04-23T01:50:00+02:00Eric Carmichaeltag:None,2014-04-23:/ignore-herokuappcom-domain-in-nginx.html<p>I was having some trouble with duplicate content from my blog <code>ericcarmichael.com</code> being replicated on <code>ericcarmichael.herokuapp.com</code>. <br><br> I added this little bit to my <a href="https://github.com/ckcollab/heroku-buildpack-pelican">Pelican buildout fork</a> and now requests to <code>myapp.herokuapp.com</code> domain are denied by nginx!</p><p>I was having some trouble with duplicate content from my blog <code>ericcarmichael.com</code> being replicated on <code>ericcarmichael.herokuapp.com</code>.</p>
<p>I added this little bit to my <a href="https://github.com/ckcollab/heroku-buildpack-pelican">Pelican buildout fork</a> and now
requests to <code>myapp.herokuapp.com</code> domain are denied by nginx!</p>
<div class="highlight"><pre><span></span><code><span class="x">http {</span>
<span class="x"> server {</span>
<span class="x"> listen </span><span class="cp"><%=</span><span class="w"> </span><span class="no">ENV</span><span class="o">[</span><span class="s2">"PORT"</span><span class="o">]</span><span class="w"> </span><span class="cp">%></span><span class="x"> default;</span>
<span class="x"> rewrite ^ $scheme://myapp.com$request_uri? permanent;</span>
<span class="x"> }</span>
<span class="x"> server {</span>
<span class="x"> listen </span><span class="cp"><%=</span><span class="w"> </span><span class="no">ENV</span><span class="o">[</span><span class="s2">"PORT"</span><span class="o">]</span><span class="w"> </span><span class="cp">%></span><span class="x">;</span>
<span class="x"> server_name myapp.com www.myapp.com;</span>
<span class="x"> }</span>
<span class="x">}</span>
</code></pre></div>
<p>The first server block catches all and denies them, the second only accepts requests from where we want! Yay!</p>Ionic vs Bootstrap2014-04-22T15:10:00+02:002014-04-22T15:10:00+02:00Eric Carmichaeltag:None,2014-04-22:/ionic-vs-bootstrap.html<p>I am writing this as a tool to help me learn more about <a href="http://www.ionicframework.com/">Ionic</a> and <a href="http://getbootstrap.com">Bootstrap</a> to decide which, if either, library to use for my new hybrid web app. My app has to look good and function well on desktop, tablet and mobile.</p><p>I come from the before times. The times where PHP was it, many were switching from <em>tables</em> to <em>css</em>, Bush was president and you had to get help from some weird phpBB forum post where the signatures took up half the screen—and
you LIKED IT!</p>
<p style="text-align: center;" class="image-wrapper">
<img src="images/ionic_vs_bootstrap/land_before_time.jpg" alt="Land Before Time"><br>
<i><small>I need a more ominous before picture...</small></i>
</p>
<p>If that's how I think of the times before <a href="http://getbootstrap.com/">Bootstrap</a>, how will I think of Bootstrap 5 years from now?</p>
<h2>Why write this?</h2>
<p>I am writing this as a tool to help me learn more about <a href="http://ionicframework.com/">Ionic</a> and Bootstrap to decide which, if either, library to use
for my new hybrid web app. My app has to look good and function well on desktop, tablet and mobile.</p>
<p>I haven't used Ionic at all, yet, but I have used Bootstrap for over a dozen projects. I love bootstrap, but there are some
things that definitely bug me and I'd love to learn about some alternatives.</p>
<p style="text-align: center;" class="image-wrapper">
<img src="images/ionic_vs_bootstrap/bootstrap.png" alt="Bootstrap" class="image-full-width"><br>
</p>
<h2>Pain points of Bootstrap</h2>
<h3>Grid system</h3>
<p>Columns. Rows. Grids.</p>
<p>It's only getting more complex with <code>.col-md-12</code>, <code>.col-xs-12</code> and such. These things aren't hard to learn, but I had just
gotten the hang of doing the whole <code>.container</code> <code>.row</code> <code>.span</code> dance. Oh, don't forget <code>.row-fluid</code> was awesome, too.</p>
<div class="highlight"><pre><span></span><code><span class="p"><</span><span class="nt">div</span> <span class="na">class</span><span class="o">=</span><span class="s">"container"</span><span class="p">></span>
<span class="p"><</span><span class="nt">div</span> <span class="na">class</span><span class="o">=</span><span class="s">"row"</span><span class="p">></span>
<span class="p"><</span><span class="nt">div</span> <span class="na">class</span><span class="o">=</span><span class="s">"span12"</span><span class="p">></span>
<span class="p"></</span><span class="nt">div</span><span class="p">></span>
<span class="p"></</span><span class="nt">div</span><span class="p">></span>
<span class="p"></</span><span class="nt">div</span><span class="p">></span>
</code></pre></div>
<h3>Requires jQuery</h3>
<p>Unforunately Bootstrap requires jQuery, but there is an <a href="https://github.com/tagawa/bootstrap-without-jquery">awesome repo</a>
that has a version of Bootstrap without the jQuery requirement!</p>
<h3>Everything looks the same</h3>
<p>This is a criticism a lot of people have that I don't entirely share, however I thought I'd mention it anyway. Many sites
are using basic bootstrap stuff without tweaking it much—doesn't really bother me, I think it's an important tool for
developers who aren't designers.</p>
<h2>Things Bootstrap does right</h2>
<h3>Great documentation</h3>
<p>Boy did they nail it. I feel like Bootstrap had the best documentation of any library a couple years ago, and it still
may hold that title. It's a snap to find what you are looking for, the explanations are easy to understand and there are
examples for nearly everything.</p>
<h3>Super easy to use, proven</h3>
<p>Bootstrap is getting more and more mature, people know how to use it and appreciate it. There are lots of tried-and-true
plugins, examples, tutorials, videos, etc. for Bootstrap.</p>
<h3>Tons of nice features</h3>
<p>Typeahead, image sliders, on and on. So many useful little features, so easy to use and expand. It's nice to be able to
rely on a library <em>most</em> of the time!</p>
<p style="text-align: center;" class="image-wrapper">
<img src="images/ionic_vs_bootstrap/ionic.png" alt="Ionic framework"><br>
</p>
<h2>Ionic's iconic infamous items (pros)</h2>
<h3>Grid system, 'yall</h3>
<p>Shit is written with <a href="https://developer.mozilla.org/en-US/docs/Web/Guide/CSS/Flexible_boxes">FLEXBOX</a>! I have been waiting for some kind of support for this forever, a good friend of mine
<a href="http://levithomason.com">Levi Thomason</a> tried to explain it to my glazed over eyes but I think I caught the jist. Flexbox solves a ton of
problems that are normally solved with floats, tricks, and wizardry:</p>
<ol>
<li>Vertical align</li>
<li>Things fill in remaining space easily</li>
<li>Elements stay where they should (footer stuck to bottom of screen)</li>
<li>etc. etc.</li>
</ol>
<h3>AngularJS instead of jQuery</h3>
<p>Ionic is built on top of AngularJS for all of the interactions, that is awesome. AngularJS makes writing front end
stuff so much less painful and so much easier to test. Way back when I couldn't imagine writing tests for jQuery stuff,
how would I emulate so many DOM elements?</p>
<p>AngularJS helps you write unit-testable code by abstracting away the DOM, instead you play around in a <code>$scope</code>
where all data is stored.</p>
<h2>Ionic's inconceivable integrations (cons)</h2>
<h3>Kind of confusing docs/dist</h3>
<p>Maybe it's because I am so used to how Bootstrap has everything laid out, but it just feels a little wonky and lacking
sometimes. For example, how the hell do I use that awesome list of animations? How do I use gestures? I had to look at
the source code to figure out what I was doing wrong.</p>
<h3>Purely mobile</h3>
<p>Ionic is built only for mobile, which is a big problem for me because I want to design a hybrid mobile/desktop app.</p>
<p>However, I feel like I just need to "do it" and try. Every other UI framework I've used on mobile has performed terribly,
but Ionic has been doing pretty good so far.</p>
<h3>Colors/styles</h3>
<p>I'm not a huge fan of the colors/styles available now, but I am sure that will get better as the project matures. Do we
really need like 10 colors for buttons?</p>
<h2>Conclusion</h2>
<p>Ionic seems to only support mobile, although it says it is a Hybrid framework. I feel like Bootstrap is much better at
doing desktop. This comparison seems more apples to orange than I first suspected, these frameworks are solving totally
different things.</p>
<p>I am going to give Ionic a shot for my next app even though it only seems to support mobile, I have already started and
it's not going terribly, here's what I have so far for my workout tracking app:</p>
<p style="text-align: center;" class="image-wrapper">
<img src="images/ionic_vs_bootstrap/workout_app1.png" alt="Personal Record workout app"><br>
<i><small>Workout entry screen. This took a few hours and looks terrible but,<br> I'm happy with the performance so far!</small></i>
</p>
<h2>All jokes aside</h2>
<p>I am a shoot from the hip kind of guy, I like to do things fast—if I messed up on any details about Bootstrap or
Ionic, please correct me!</p>
<p>Is something easier than I am making it seem? Am I doing something wrong? Let me know! I plan to update this as I learn
more.</p>Polished2014-04-18T15:10:00+02:002014-04-18T15:10:00+02:00Eric Carmichaeltag:None,2014-04-18:/polished.html<iframe width="640" height="360" class="youtube" src="https://www.youtube-nocookie.com/embed/Yi5fHkGqe38?rel=0" frameborder="0" allowfullscreen></iframe>
<p class="centered-label">Polished resume page video</p>
<p><br>The goal of <a href="http://github.com/ckcollab/polished">polished</a> is to show all of the meticulous tweaks that go into a website. My resume is a good example, dozens of hours of work and tweaking to come up with this pretty basic final product. Showing that blood, sweat and hilarious tears in between should be pretty entertaining. Watch pages undulate, stretch, break, grow, and shrink into place.</p><p>The goal of <a href="http://github.com/ckcollab/polished">polished</a> is to show all of the meticulous tweaks that go into a website. My resume
is a good example, dozens of hours of work and tweaking to come up with this pretty basic final product. Showing that
blood, sweat and hilarious tears in between should be pretty entertaining. Watch pages undulate, stretch, break,
grow, and shrink into place.</p>
<h2>How does it work?</h2>
<p>Once you've installed polished, it works like this:</p>
<ol>
<li>Fires up selected backend (for example, PelicanBackend if you use the <a href="https://github.com/getpelican/pelican">Pelican</a> blog site generator)</li>
<li>Get the history of your git repo</li>
<li>Iterate through that history, preparing each page and finally screen cap it</li>
<li>If after reviewing the images you find bugs, you can go in and <code>@polish</code> out the kinks so it's a nice smooth video</li>
</ol>
<h2>Examples</h2>
<iframe width="640" height="360" class="youtube" src="https://www.youtube-nocookie.com/embed/J4sBUXP7zoo?rel=0" frameborder="0" allowfullscreen></iframe>
<p class="centered-label">
Resume page video without polishing
</p>
<iframe width="640" height="360" class="youtube" src="https://www.youtube-nocookie.com/embed/Yi5fHkGqe38?rel=0" frameborder="0" allowfullscreen></iframe>
<p class="centered-label">
Polished resume page video
</p>
<p>And I had to "polish" these videos to get them just right, fix bad links for some commits:</p>
<div class="highlight"><pre><span></span><code><span class="kn">from</span> <span class="nn">selenium.webdriver.common.by</span> <span class="kn">import</span> <span class="n">By</span>
<span class="kn">from</span> <span class="nn">selenium.webdriver.support.ui</span> <span class="kn">import</span> <span class="n">WebDriverWait</span>
<span class="kn">from</span> <span class="nn">selenium.webdriver.support</span> <span class="kn">import</span> <span class="n">expected_conditions</span> <span class="k">as</span> <span class="n">EC</span>
<span class="kn">from</span> <span class="nn">polished.backends</span> <span class="kn">import</span> <span class="n">PelicanBackend</span>
<span class="kn">from</span> <span class="nn">polished.decorators</span> <span class="kn">import</span> <span class="n">polish</span>
<span class="k">class</span> <span class="nc">EricPelicanBackend</span><span class="p">(</span><span class="n">PelicanBackend</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">_patch_image_srcs</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">wait</span> <span class="o">=</span> <span class="n">WebDriverWait</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">DRIVER</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span>
<span class="n">element</span> <span class="o">=</span> <span class="n">wait</span><span class="o">.</span><span class="n">until</span><span class="p">(</span><span class="n">EC</span><span class="o">.</span><span class="n">visibility_of_element_located</span><span class="p">((</span><span class="n">By</span><span class="o">.</span><span class="n">TAG_NAME</span><span class="p">,</span> <span class="s1">'img'</span><span class="p">)))</span>
<span class="bp">self</span><span class="o">.</span><span class="n">DRIVER</span><span class="o">.</span><span class="n">execute_script</span><span class="p">(</span><span class="s2">"""</span>
<span class="s2"> var img_array = document.getElementsByTagName('img');</span>
<span class="s2"> for(var i=0; i<img_array.length; i++) {</span>
<span class="s2"> var href_replaced = img_array[i].getAttribute('src').replace(/^\/images/, "../images");</span>
<span class="s2"> img_array[i].setAttribute("src", href_replaced);</span>
<span class="s2"> }</span>
<span class="s2"> """</span><span class="p">)</span>
<span class="nd">@polish</span><span class="p">(</span><span class="n">urls</span><span class="o">=</span><span class="p">[</span><span class="s2">"output/pages/about.html"</span><span class="p">],</span> <span class="n">commit_indexes</span><span class="o">=</span><span class="nb">range</span><span class="p">(</span><span class="mi">112</span><span class="p">,</span> <span class="mi">135</span><span class="p">))</span>
<span class="k">def</span> <span class="nf">fix_image_links_on_about_me_page</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_patch_image_srcs</span><span class="p">()</span>
<span class="nd">@polish</span><span class="p">(</span><span class="n">urls</span><span class="o">=</span><span class="p">[</span><span class="s2">"output/pages/resume.html"</span><span class="p">],</span> <span class="n">commit_indexes</span><span class="o">=</span><span class="nb">range</span><span class="p">(</span><span class="mi">68</span><span class="p">,</span><span class="mi">134</span><span class="p">))</span>
<span class="k">def</span> <span class="nf">fix_resume_page_broken_images</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_patch_image_srcs</span><span class="p">()</span>
</code></pre></div>
<h2>Installation</h2>
<p>Requires:</p>
<ul>
<li>Mac</li>
<li><a href="http://nodejs.org/">NodeJS</a></li>
<li><a href="http://phantomjs.org/">PhantomJS</a></li>
<li><a href="http://www.ffmpeg.org/">ffmpeg</a></li>
</ul>
<p>Then the usual</p>
<div class="highlight"><pre><span></span><code>><span class="w"> </span>pip<span class="w"> </span>install<span class="w"> </span>polished
</code></pre></div>
<p>For more detailed instructions please check out the <a href="https://github.com/ckcollab/polished">repo readme</a>.</p>
<h2>Usage</h2>
<div class="highlight"><pre><span></span><code>> polished
</code></pre></div>
<p>The default behavior is to capture <code>"index.html"</code> each commit</p>
<div class="highlight"><pre><span></span><code>> polished output/index.html
</code></pre></div>
<p>Local file</p>
<div class="highlight"><pre><span></span><code>> polished http://localhost:8000/
</code></pre></div>
<p>Local server</p>
<p>By default the files are saved to <code>polished_output/<commit count>.<sha>.polished.png</code> and <code>polished_output/output.mp4</code></p>Writing my first python package2014-04-14T13:00:00+02:002014-04-14T13:00:00+02:00Eric Carmichaeltag:None,2014-04-14:/writing-my-first-python-package.html<p>I've been looking make a little python package to launch on <a href="https://pypi.python.org/">pypi</a> for quite some time. With all of my recent <a href="http://getpelican.com">Pelican</a> blog tweaking, I am having about 50 urges per second to improve the development a little bit.</p><p>I've been looking make a little python package to launch on <a href="https://pypi.python.org/">pypi</a> for quite some time. With
all of my recent <a href="http://getpelican.com">Pelican</a> blog tweaking, I am having about 50 urges per second to improve the
development a little bit.</p>
<h2>The problem</h2>
<p>One of the things that keeps irking me while writing posts was constantly having to double check that I put in the
correct value for each link in markdown</p>
<div class="highlight"><pre><span></span><code>So there I was, at [Some Restaurant I can't Remember]() downtown and I totally saw Becky macking on Jonathon!
</code></pre></div>
<p>Right as I am getting into the juicy gossip I normally would blog about, now I have worries:</p>
<ol>
<li>I will forget to fix that link</li>
<li>I will think about "did I fix that link?" even if I already did</li>
<li>I. will. forget. to. fix. that. LINK!</li>
<li>Years later I will wake up in a cold sweat wondering, "Did I fix that link?"</li>
</ol>
<h2>The attempted solution</h2>
<p>My first idea was to make some kind of hook in the Pelican system that re-wrote <code>[text](url)</code> links where the <code>url</code> was
empty with the first google search result of the <code>text</code> description.</p>
<p>That idea sucks for so many reasons:</p>
<ul>
<li>It could completely destroy posts</li>
<li>It modifies the repository</li>
<li>Did I fix that link?</li>
</ul>
<h2>The real solution: <a href="https://github.com/ckcollab/existence">Existence</a></h2>
<p>After playing around with a few different ideas I finally thought: screw this. I was thinking too hard, all of the
solutions were too convoluted. When I stepped away from the computer and thought for a while I realized
the problem I was <em>really</em> trying to fix: broken links!</p>
<p>After some searching I didn't find a nice simple python module that ran through static html files, tried links, and spit
out which ones were bad.</p>
<h2>Writing the module</h2>
<p>I want this to run quickly or I'll never use it, so my plan is to use <code>requests</code> asychronously after scanning all of the
files for broken links. However, after trying my damndest I couldn't get <code>grequests</code> to work right... I was having weird
errors in the background I couldn't debug easily.</p>
<p>Then I saw <a href="http://stackoverflow.com/a/14369828/2197389">this</a> nice little example, here's my version:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">async_check_url</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">file_name</span><span class="p">,</span> <span class="n">line_number</span><span class="p">):</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">urllib2</span><span class="o">.</span><span class="n">urlopen</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
<span class="k">except</span> <span class="n">urllib2</span><span class="o">.</span><span class="n">URLError</span><span class="p">:</span>
<span class="n">BROKEN_URLS</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="n">url</span><span class="p">,</span> <span class="n">file_name</span><span class="p">,</span> <span class="n">line_number</span><span class="p">))</span>
<span class="k">def</span> <span class="nf">check_urls</span><span class="p">(</span><span class="n">urls</span><span class="p">):</span>
<span class="w"> </span><span class="sd">'''</span>
<span class="sd"> expected format of urls is list of tuples (url, file name, source line) i.e. ("google.com", "index.html", 32)</span>
<span class="sd"> '''</span>
<span class="n">threads</span> <span class="o">=</span> <span class="nb">list</span><span class="p">()</span>
<span class="k">for</span> <span class="n">u</span> <span class="ow">in</span> <span class="n">urls</span><span class="p">:</span>
<span class="n">t</span> <span class="o">=</span> <span class="n">Thread</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="n">async_check_url</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">(</span><span class="n">u</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">u</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">u</span><span class="p">[</span><span class="mi">2</span><span class="p">]))</span>
<span class="n">t</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
<span class="n">threads</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">t</span><span class="p">)</span>
<span class="k">for</span> <span class="n">thread</span> <span class="ow">in</span> <span class="n">threads</span><span class="p">:</span>
<span class="n">thread</span><span class="o">.</span><span class="n">join</span><span class="p">()</span>
</code></pre></div>
<h2>Getting it on pypi</h2>
<h3>1. sign up on pypi.com</h3>
<h3>2. create <code>.pypirc</code> in home directory with login info</h3>
<div class="highlight"><pre><span></span><code>[distutils]
index-servers = pypi
[pypi]
username:ckcollab
password:hunter2
</code></pre></div>
<h3>3. fill out setup.py</h3>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">os</span>
<span class="kn">from</span> <span class="nn">setuptools</span> <span class="kn">import</span> <span class="n">setup</span>
<span class="k">try</span><span class="p">:</span>
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s1">'README.md'</span><span class="p">)</span> <span class="k">as</span> <span class="n">readme</span><span class="p">:</span>
<span class="n">long_description</span> <span class="o">=</span> <span class="n">readme</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
<span class="k">except</span> <span class="ne">IOError</span><span class="p">,</span> <span class="ne">ImportError</span><span class="p">:</span>
<span class="n">long_description</span> <span class="o">=</span> <span class="s1">''</span>
<span class="n">setup</span><span class="p">(</span>
<span class="n">install_requires</span> <span class="o">=</span> <span class="p">[</span>
<span class="s2">"lxml>=3.3.4"</span><span class="p">,</span>
<span class="s2">"cssselect>=0.9.1"</span>
<span class="p">],</span>
<span class="n">name</span><span class="o">=</span><span class="s2">"existence"</span><span class="p">,</span>
<span class="n">py_modules</span><span class="o">=</span><span class="p">[</span><span class="s2">"existence"</span><span class="p">],</span>
<span class="n">version</span><span class="o">=</span><span class="s2">"0.0.8"</span><span class="p">,</span>
<span class="n">author</span><span class="o">=</span><span class="s2">"Eric Carmichael"</span><span class="p">,</span>
<span class="n">author_email</span><span class="o">=</span><span class="s2">"eric@ckcollab.com"</span><span class="p">,</span>
<span class="n">description</span><span class="o">=</span><span class="s2">"Checks static .html files for bad links"</span><span class="p">,</span>
<span class="n">long_description</span><span class="o">=</span><span class="n">long_description</span><span class="p">,</span>
<span class="n">license</span><span class="o">=</span><span class="s2">"MIT"</span><span class="p">,</span>
<span class="n">keywords</span><span class="o">=</span><span class="s2">"link checker"</span><span class="p">,</span>
<span class="n">url</span><span class="o">=</span><span class="s2">"https://github.com/ckcollab/existence"</span><span class="p">,</span>
<span class="n">classifiers</span><span class="o">=</span><span class="p">[</span>
<span class="s1">'Intended Audience :: Developers'</span><span class="p">,</span>
<span class="s1">'Natural Language :: English'</span><span class="p">,</span>
<span class="s1">'License :: OSI Approved :: MIT License'</span><span class="p">,</span>
<span class="s1">'Operating System :: OS Independent'</span><span class="p">,</span>
<span class="s1">'Programming Language :: Python'</span><span class="p">,</span>
<span class="s1">'Programming Language :: Python :: 2'</span><span class="p">,</span>
<span class="s1">'Programming Language :: Python :: 2.6'</span><span class="p">,</span>
<span class="s1">'Programming Language :: Python :: 2.7'</span><span class="p">,</span>
<span class="s1">'Topic :: Software Development :: Libraries :: Python Modules'</span><span class="p">,</span>
<span class="p">],</span>
<span class="p">)</span>
</code></pre></div>
<h3>4. submit</h3>
<div class="highlight"><pre><span></span><code>> python setup.py register
> python setup.py sdist upload
</code></pre></div>
<h2>Plugging it into <a href="http://www.fabfile.org/">fabric</a></h2>
<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">os</span>
<span class="kn">from</span> <span class="nn">existence</span> <span class="kn">import</span> <span class="n">get_bad_urls</span>
<span class="kn">from</span> <span class="nn">fabric.api</span> <span class="kn">import</span> <span class="o">*</span>
<span class="c1"># Local path configuration (can be absolute or relative to fabfile)</span>
<span class="n">env</span><span class="o">.</span><span class="n">deploy_path</span> <span class="o">=</span> <span class="s1">'output'</span>
<span class="n">DEPLOY_PATH</span> <span class="o">=</span> <span class="n">env</span><span class="o">.</span><span class="n">deploy_path</span>
<span class="k">def</span> <span class="nf">clean</span><span class="p">():</span>
<span class="k">if</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">isdir</span><span class="p">(</span><span class="n">DEPLOY_PATH</span><span class="p">):</span>
<span class="n">local</span><span class="p">(</span><span class="s1">'rm -rf </span><span class="si">{deploy_path}</span><span class="s1">'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="o">**</span><span class="n">env</span><span class="p">))</span>
<span class="n">local</span><span class="p">(</span><span class="s1">'mkdir </span><span class="si">{deploy_path}</span><span class="s1">'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="o">**</span><span class="n">env</span><span class="p">))</span>
<span class="k">def</span> <span class="nf">deploy</span><span class="p">():</span>
<span class="n">clean</span><span class="p">()</span>
<span class="n">local</span><span class="p">(</span><span class="s2">"make html"</span><span class="p">)</span>
<span class="nb">print</span> <span class="s2">"Checking URLs"</span>
<span class="n">bad_urls</span> <span class="o">=</span> <span class="n">get_bad_urls</span><span class="p">(</span><span class="n">DEPLOY_PATH</span><span class="p">)</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">bad_urls</span><span class="p">:</span>
<span class="nb">print</span> <span class="s2">"URL's are looking good"</span>
<span class="n">local</span><span class="p">(</span><span class="s2">"git push"</span><span class="p">)</span>
<span class="n">local</span><span class="p">(</span><span class="s2">"git push heroku master"</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">for</span> <span class="n">url</span> <span class="ow">in</span> <span class="n">bad_urls</span><span class="p">:</span>
<span class="nb">print</span> <span class="s2">"Broken link found in file </span><span class="si">%s</span><span class="s2"> on line </span><span class="si">%s</span><span class="s2"> linking to </span><span class="si">%s</span><span class="s2">"</span> <span class="o">%</span> <span class="p">(</span><span class="n">url</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">url</span><span class="p">[</span><span class="mi">2</span><span class="p">],</span> <span class="n">url</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
</code></pre></div>
<p>Now I just run</p>
<div class="highlight"><pre><span></span><code>> fab deploy
</code></pre></div>
<p>and I will never ever, ever have to worry about broken links in my blog! Thanks <a href="https://github.com/ckcollab/existence">existence</a>!</p>Starting a blog with Pelican2014-04-12T00:30:00+02:002014-04-12T00:30:00+02:00Eric Carmichaeltag:None,2014-04-12:/starting-a-blog-with-pelican.html<p>How I started and deploy my blog with <a href="https://github.com/getpelican/pelican">Pelican</a> blog system and <a href="http://heroku.com/">heroku</a>!</p><p>Deciding what static site generator to use was tricky. The first option was <a href="http://jekyllrb.com/">Jekyll</a> which is ruby
and I'm looking for something python. Next I looked at <a href="https://github.com/lakshmivyas/hyde">Hyde</a> but that had the extra
requirement of <a href="https://www.djangoproject.com/">Django</a>. I feel like all I need something super simple that has the bare
minimum to poop out content from markdown.</p>
<p>Then I finally found <a href="https://github.com/getpelican/pelican">Pelican</a> and it seems to satisfy all of my needs: python, ultra simple integration with <a href="http://heroku.com/">heroku</a> via <a href="https://github.com/getpelican/heroku-buildpack-pelican">this buildout</a>!</p>
<h2>Development</h2>
<h3>Getting started</h3>
<div class="highlight"><pre><span></span><code>><span class="w"> </span>pelican-quickstart
</code></pre></div>
<p>That will take you through a wizard to create a project structure like:</p>
<div class="highlight"><pre><span></span><code>yourproject/
├── content
│ └── (pages)
├── output
├── develop_server.sh
├── fabfile.py
├── Makefile
├── pelicanconf.py # Main settings file
└── publishconf.py # Settings to use when ready to publish
</code></pre></div>
<h3>Testing locally</h3>
<div class="highlight"><pre><span></span><code>><span class="w"> </span>make<span class="w"> </span>regenerate
</code></pre></div>
<p>Any time you make a change to a piece of content the site is rebuilt.</p>
<h2>Theming</h2>
<h3>Theme folder structure</h3>
<p>This is what I ended up with for a rough rough draft, from examples in <a href="https://github.com/getpelican/pelican-themes">pelican-themes</a> repo</p>
<div class="highlight"><pre><span></span><code>mytheme/
├── static/
│ └── css/
│ └── fonts/
│ └── images/
└── templates/
└── includes/
│ └── _article.html
└── article.html
└── base.html
└── index.html
└── page.html
</code></pre></div>
<h3>Adding tags</h3>
<p>Added this to the bottom of my <code>_article.html</code></p>
<div class="highlight"><pre><span></span><code><span class="cp">{%</span> <span class="k">if</span> <span class="nv">article.tags</span> <span class="cp">%}</span>
<span class="x"> <div class="tags"></span>
<span class="x"> <i class="fa fa-tags"></i></span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">for</span> <span class="nv">tag</span> <span class="k">in</span> <span class="nv">article.tags</span> <span class="cp">%}</span>
<span class="x"> <a href="</span><span class="cp">{{</span> <span class="nv">SITEURL</span> <span class="cp">}}</span><span class="x">/</span><span class="cp">{{</span> <span class="nv">tag.url</span> <span class="cp">}}</span><span class="x">"></span><span class="cp">{{</span> <span class="nv">tag</span> <span class="cp">}}</span><span class="x"></a></span><span class="cp">{%</span> <span class="k">if</span> <span class="k">not</span> <span class="nb">loop</span><span class="nv">.last</span> <span class="cp">%}</span><span class="x">, </span><span class="cp">{%</span> <span class="k">endif</span> <span class="cp">%}</span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">endfor</span> <span class="cp">%}</span>
<span class="x"> </div></span>
<span class="cp">{%</span> <span class="k">endif</span> <span class="cp">%}</span>
</code></pre></div>
<h3>Adding page links</h3>
<p>Added this to my menu, the "blog" is just a link back to index, so I added that manually</p>
<div class="highlight"><pre><span></span><code><span class="x"><div class="menu"></span>
<span class="x"> <ul></span>
<span class="x"> <li><a href="</span><span class="cp">{{</span> <span class="nv">SITEURL</span> <span class="cp">}}</span><span class="x">/index.html">blog</a></li></span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">for</span> <span class="nv">page</span> <span class="k">in</span> <span class="nv">pages</span> <span class="cp">%}</span>
<span class="x"> <li><a href="</span><span class="cp">{{</span> <span class="nv">SITEURL</span> <span class="cp">}}</span><span class="x">/</span><span class="cp">{{</span> <span class="nv">page.url</span> <span class="cp">}}</span><span class="x">"></span><span class="cp">{{</span> <span class="nv">page.title</span><span class="o">|</span><span class="nf">lower</span> <span class="cp">}}</span><span class="x"></a></li></span>
<span class="x"> </span><span class="cp">{%</span> <span class="k">endfor</span> <span class="cp">%}</span>
<span class="x"> </ul></span>
<span class="x"></div></span>
</code></pre></div>
<h2>Deploying</h2>
<h3>Easy with buildout</h3>
<div class="highlight"><pre><span></span><code>> git push heroku master
</code></pre></div>
<h3>Compressing CSS/JS</h3>
<p>Used the assets plugin from <a href="https://github.com/getpelican/pelican-plugins">pelican-plugins</a> which requires the <a href="https://github.com/miracle2k/webassets">webassets</a> python module.</p>
<div class="highlight"><pre><span></span><code><span class="cp">{%</span> <span class="k">assets</span> <span class="nv">filters</span><span class="o">=</span><span class="s2">"cssmin"</span><span class="o">,</span> <span class="nv">output</span><span class="o">=</span><span class="s2">"css/packed.min.css"</span><span class="o">,</span> <span class="s2">"css/bootstrap.min.css"</span><span class="o">,</span> <span class="s2">"css/pygment-solarized.css"</span><span class="o">,</span> <span class="s2">"css/main.css"</span> <span class="cp">%}</span>
<span class="x"> <link rel="stylesheet" href="</span><span class="cp">{{</span> <span class="nv">SITEURL</span> <span class="cp">}}</span><span class="x">/</span><span class="cp">{{</span> <span class="nv">ASSET_URL</span> <span class="cp">}}</span><span class="x">"></span>
<span class="cp">{%</span> <span class="k">endassets</span> <span class="cp">%}</span>
</code></pre></div>
<h3>Configuration</h3>
<p>Created an environment variable <code>PELICAN_SITE_URL</code> for my virtualenv and on heroku, that way I can set it locally and test with <code>make regenerate</code> easily.</p>
<h3>Adding disqus</h3>
<p>Signed up on <a href="http://disqus.com">disqus</a> and got the embed code, then added the <code>disqus_identifier</code>, <code>disqus_title</code> and <code>disqus_url</code> extra configuration variables.</p>
<p>To the bottom of <code>article.html</code> I added:</p>
<div class="highlight"><pre><span></span><code><span class="nt"><div</span><span class="w"> </span><span class="na">class=</span><span class="s">"disqus"</span><span class="nt">></span>
<span class="w"> </span><span class="nt"><div</span><span class="w"> </span><span class="na">id=</span><span class="s">"disqus_thread"</span><span class="nt">></div></span>
<span class="w"> </span><span class="nt"><script</span><span class="w"> </span><span class="na">type=</span><span class="s">"text/javascript"</span><span class="nt">></span>
<span class="w"> </span>var<span class="w"> </span>disqus_shortname<span class="w"> </span>=<span class="w"> </span>'ericcarmichaelsnerdery';
<span class="w"> </span>var<span class="w"> </span>disqus_identifier<span class="w"> </span>=<span class="w"> </span>'<span class="cp">{{</span> <span class="nv">article.url</span> <span class="cp">}}</span>';
<span class="w"> </span>var<span class="w"> </span>disqus_title<span class="w"> </span>=<span class="w"> </span>'<span class="cp">{{</span> <span class="nv">article.title</span> <span class="cp">}}</span>';
<span class="w"> </span>var<span class="w"> </span>disqus_url<span class="w"> </span>=<span class="w"> </span>'<span class="cp">{{</span> <span class="nv">SITEURL</span> <span class="cp">}}</span>/<span class="cp">{{</span> <span class="nv">article.url</span> <span class="cp">}}</span>';
<span class="w"> </span>(function()<span class="w"> </span>{
<span class="w"> </span>var<span class="w"> </span>dsq<span class="w"> </span>=<span class="w"> </span>document.createElement('script');
<span class="w"> </span>dsq.type<span class="w"> </span>=<span class="w"> </span>'text/javascript';
<span class="w"> </span>dsq.async<span class="w"> </span>=<span class="w"> </span>true;
<span class="w"> </span>dsq.src<span class="w"> </span>=<span class="w"> </span>'//'<span class="w"> </span>+<span class="w"> </span>disqus_shortname<span class="w"> </span>+<span class="w"> </span>'.disqus.com/embed.js';
<span class="w"> </span>(document.getElementsByTagName('head')[0]<span class="w"> </span>||<span class="w"> </span>document.getElementsByTagName('body')[0]).appendChild(dsq);
<span class="w"> </span>})();
<span class="w"> </span><span class="nt"></script></span>
<span class="w"> </span><span class="nt"><noscript></span>Please<span class="w"> </span>enable<span class="w"> </span>JavaScript<span class="w"> </span>to<span class="w"> </span>view<span class="w"> </span>the<span class="w"> </span><span class="nt"><a</span><span class="w"> </span><span class="na">href=</span><span class="s">"http://disqus.com/?ref_noscript"</span><span class="nt">></span>comments<span class="w"> </span>powered<span class="w"> </span>by<span class="w"> </span>Disqus.<span class="nt"></a></noscript></span>
<span class="w"> </span><span class="nt"><a</span><span class="w"> </span><span class="na">href=</span><span class="s">"http://disqus.com"</span><span class="w"> </span><span class="na">class=</span><span class="s">"dsq-brlink"</span><span class="nt">></span>comments<span class="w"> </span>powered<span class="w"> </span>by<span class="w"> </span><span class="nt"><span</span><span class="w"> </span><span class="na">class=</span><span class="s">"logo-disqus"</span><span class="nt">></span>Disqus<span class="nt"></span></a></span>
<span class="nt"></div></span>
</code></pre></div>
<p>Pretty dissappointed in disqus, can't style it very easily and I feel it clashes with the style of the blog quite a bit.</p>
<h2>Conclusion</h2>
<p>Man. What a joy! Everything made sense, except for a couple hiccups with webassets that ended up being my fault, typical!</p>
<p>I'd recommend <a href="https://github.com/getpelican/pelican">Pelican</a> to anyone looking to start a simple blog, it was a joy to work with thanks to all of the hard work
that went into <code>pelican-quickstart</code> and the <a href="https://github.com/getpelican/heroku-buildpack-pelican">custom heroku buildout</a>!</p>