As mentioned in a previous post, I moved to hexo partly because I wanted to make image galleries work.

As it turns out, the landscape theme has already laid the groundwork by including fancybox so you can conveniently add something like this to your front matter:

1
2
3
4

photos:
- /images/IMG_20170125_160347.jpg
- /images/IMG_20170125_082923.jpg
- /images/IMG_20170125_082915.jpg

and place the relevant photos in /source/images/. Now if you click on the first image on the top of the post you will get a default fancybox gallery. However, this does not allow you to add annotations or captions.

What would be preferable would be a gallery with thumbnails and captions:

This should looks something like this:

1
2
3

{% fancybox "/images/IMG_20170125_160347.jpg" "Studio Bell" %} 
{% fancybox "/images/IMG_20170125_082923.jpg" "Downtown Calgary" %}
{% fancybox "/images/IMG_20170125_082915.jpg" "The Rocky mountains" %}

What I also wanted to achieve was to separate the images into multiple galleries, be it on the same page or on the home page. Using the fancybox tag as above leads to the unexpected result that all images on the page will end up in a single gallery. This is a side effect of the theme trying to ensure that the galleries from individual posts are in separate galleries on the home page. It does this in themes/landscape/js/script.js by iterating over all images in each article and setting their rel link to a unique id for each article, overwriting any pre-existing rel links in the process. The fix looks like this:

1 Copy the themes/landscape/scripts/fancybox.js to create a new tag. Let’s call it fancybox2

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

var rUrl = /((([A-Za-z]{3,9}:(?:\/\/)?)(?:[-;:&=\+\$,\w]+@)?[A-Za-z0-9.-]+|(?:www.|[-;:&=\+\$,\w]+@)[A-Za-z0-9.-]+)((?:\/[\+~%\/.\w-_]*)?\??(?:[-\+=&;%@.\w_]*)#?(?:[.\!\/\\w]*))?)/;
/**
* Fancybox tag
*
* Syntax:
*   {% fancybox2 galleryname /path/to/image [/path/to/thumbnail] [title] %}
*/
hexo.extend.tag.register('rpgallery', function(args){
  var reltag = args.shift(),
    original = args.shift(),
    thumbnail = '';
  if (args.length && rUrl.test(args[0])){
    thumbnail = args.shift();
  }
  var title = args.join(' ');
  return '<a class="fancybox" href="' + original + '" title="' + title + '" rel="' + reltag + '">' +
    '<img src="' + (thumbnail || original) + '" alt="' + title + '">' +
    '</a>' + 
    (title ? '<span class="caption">' + title + '</span>' : '');
});

2 Modify themes/landscape/source/js/script.js to append to the existing rel link instead of overwriting it.

1
2
3
4
5
6
7
8
9
10
11
12

diff --git a/themes/landscape/source/js/script.js b/themes/landscape/source/js/script.js
index 1e58767..58d9fa8 100644
--- a/themes/landscape/source/js/script.js
+++ b/themes/landscape/source/js/script.js
@@ -98,7 +98,7 @@
     });
 
     $(this).find('.fancybox').each(function(){
-      $(this).attr('rel', 'article' + i);
+      $(this).attr('rel', $(this).attr('rel') + '-article-' + i);
     });
   });

And voila, we can have another separate gallery …

I’ve been meaning to clean up my blog for a while and was intrigued by the ability to serve complete websites from github. Sure, these have to be static websites but for what I do with my blog I don’t see that as much of a limitation. As a secondary benefit I was hoping to finally get my pictures back online and the VCS via generator to static web format seemed good for that.

Which static site generator to use?

The obvious initial answer was Jekyll because then github could take care of compiling and serving and I theoretically like the file layout. Practically, however, I was too late to that game and with Jekyll 3 things are so broken that you might be able to use the minimal theme but anything beyond that is likely only going to work if you do the generation on your end and check in the results. So what are the alternatives?

Apparently this is a topic of some importance with its own site dedicated to tracking this. As of this writing the favorits are jekyll, hugo, and hexo. With jekyll out of the running, hexo seemed easier to adopt.

Framework vs. Theme

Maybe the hardest bit to understand is where the framework ends and the themes begin. This appears to be a problem with all of the static site generators, presumably because they are trying to be “flexible” instead of “opinionated”.

Fixing up the landscape theme

Hexo 3 ships with the “landscape theme” which has a good set of starting functionality and works well on mobile too. Sadly there are a few things that you will need to modify in the theme right away: the banner image and the navbar links. Once that’s done you should be good for a while.

To change the banner image:

1
2
3
4
5

# Save the original banner in case you want to use it later
mv themes/landscape/source/css/images/banner.jpg themes/landscape/source/css/images/banner.jpg.off
# Add your own banner
cp MY-NEW-IMAGE                                  themes/landscape/source/css/images/banner.jpg

To modify the navbar, e.g. to add an about section, create a static page

1
2

hexo new page about
subl source/about/index.md

You will notice that this cdreatd a subfolder called /about/ with an index file in it. You may subsequently edit the index file and maybe even change the title in the front matter, e.g. for proper uppercasing to “About”. However, it is the directory that will be referenced going forward. You now need to add the name of this directory to the theme configuration, in my case the landscape theme:

1
2

# Save the original config
[ -s themes/landscape/_config.yml ] || cp themes/landscape/_config.yml themes/landscape/_config.yml.orig

then edit the file to point at your about page. Note that the key “About” will be the text in your navbar while the value “about” is the name of the directory referenced to find the relevant page:

1
2
3

menu:
  ...
  About: about

Adding a favicon

Simply add an image in source/favicon.png and hexo will do its magic to create a favicon.

Making RSS atom feed work

The landscape theme has an RSS icon but when you click on it nothing happens. To resolve this simply include the relevant generator which will then automagically create the missing atom.xml file:

1

https://github.com/hexojs/hexo-generator-feed

Creating a sitemap

Just like for the atom feed, there is a hexo generator that will create a sitemap:

1

npm install hexo-generator-sitemap --save

However, this only creates a sitemap file in /sitemap.xml. While I expect that some crawlers will find this explicitly, it is probably a good idea to reference the sitemap using robots.txt, maybe like so:

User-agent: *
Disallow: /tags/
Disallow: /categories/
Disallow: /archives/

Sitemap: https://artiverse.github.io/sitemap.xml

This particular configuration will also exclude crawling of the autogenerated locations which matches what the sitemap creates.

In the previous post I discussed how bad it is for proxies to retry. In that post I mentioned offhandedly that the proxy retrying was not only going to make your app slower but also more expensive. This is a first look at that problem.

For your convenience I have created another visualization on GitHub and you can play with it here.

Imagine you have a perfect system and it has constant response time no matter how heavily you load it and it will never drop anything from its queue. If we hit the system with more requests than it can handle it will keep processing requests at that same speed but the responses will be ever more delayed because the requests are stuck in a request queue.

No retries

In the no retry case, as we slowly load up the system the total number of requests in the queue eventually exceeds the speed with which they can be processed (about 1500s with default settings). At this point both the queue length and the response time start increasing. Eventually the response time exceeds the timeout on the client (about 1800s with default settings). From the client’s perspective at this point all requests start failing. Note that this situation persists well past the point where the load has dropped below what the system can handle (about 2200s with default settings) because there are still so many – effectively dead – requests stuck in the queue. Only when the queue size drops significantly does the response time drop back below the 15s timeout (about 2350s with default settings) and requests start succeeding again.

All this is bad, but it is expected. One could argue that in this model there is only a problem if we are running very close to the limit of the ability of the system to handle load and that at that point some failures are expected.

With retries

Of course when the request fails it is likely the client will retry. In the simplest case we add a single retry. Everything remain the same until the first timeouts. At this point the number of requests on the system increases significantly because for every request more than 15s old a new request is added to the queue. This can be observed in the steep increase of the overall length of the request queue (bold red line). At the same time the average response time for requests also increases (orange line) because there are now even more (still dead) requests in the queue that need to be processed – and dropped by the client) before recovery can happen.

The thin red lines show the individual contributions of the original requests and the retries. As expected with a single retry these each contribute about half of the overall queued requests. If you look very carfully you will see that the contribution of the original requests initially almost follows the no-retry case but then with the increase of the retries suddenly increases to about the same level as the retries. This may initially seem counterintuitive but can easily be understood if you consider that original requests and retry requests are indistinguishable in the queue and the back end will process them as they come in. In other words, for every retry the back end will not process a first request and as a result more and more original requests will remain in the queue for longer.

To be continued …

I will leave you to ponder the implications of all this while I go off building a couple more models – specifically (1) a model where the system is running with a perfectly fine amount of headroom but gets hit by a requests spike – be it Black Friday or a network outage – and (2) a more realistic model where the response time is not constant with load to give you a more visceral sense of how much headroom a system really needs.

You live in the cloud. Your app lives in the cloud. Mostly. You’ve decided to add access controls via a simple proxy. Your service is supposed to have “100%” uptime, so of course the proxy has to have “100%” uptime.

So far so good – except that the back end only has 99.9% uptime and your friendly ops people have set up alarms that check service uptime via your proxy. Since you don’t want to get dinged you figure you’ll retry. No alarms, no problem. Right?

Truth is you’ve just made your app slower. Probably a lot slower. And more expensive. And less stable.

Huh?

Look at the data

Have a look at this picture. This is a real test for a proxy that retries after 15s.

Let’s focus on the orange data. You’ll have to trust me when I say there are orange dots under the green dots. What you see is that the retry works really well: typical response time is about 2s and if that fails we get responses after about 17s (15+2) and if that fails we get responses after about 32s (215+2) and if that fails we get responses after about 47s (315+2). This is great! The proxy works!

Does it though? What should the client do? Should it wait for 50s? Or should it retry retry 25 times after 2s in the hopes that a single call will take the expected 2s? ? 10 times after 5s to account for some spread? Exponential backoff?

Based on the orange lines the client should absolutely retry every 3-5s. Of course that will kill your proxy and back end because under load each of the “timed out” calls will still go through the full proxy/back-end retry cycle. You just made the client DoS your service.

In practice the blue data is more realistic. Under load there is a time spread in service response times. Some calls may take up to 15s and response times generally increase with server load. This is why best practice would suggest exponential backoff. Using exponential backoff may improve the situation because you will eventually wait longer than the first proxy retry but even now the early calls in the backoff sequence are “timing out” on the client but the abandoned client calls still put a load on the backend through the proxy retry pattern so now you are using the proxy to DoS your service. Better but not good.

Finally, in both of the above cases you client contains retry code. Now, why would you have retry code in your proxy?

Slower? I don’t believe you!

Ok. Just for you I have created this cool little toy on GitHub which allows you to walk through this step by step. Let’s say your server takes at least 2s to respond and at most 6s. Let’s model this as a gaussian because they are pretty:

The blue line shows instantaneous probability that your request will be served at this time. The green line is the integrated probability, meaning that your request will be served by this time. Basically at 6s it is all but guaranteed that you received a reply.

So far so good. Now let’s have a look at the red line and what happens if we retry. If we retry early then we give up on any chance of the old request being fulfilled and start the wait again at the beginning. What this shows very nicely is that for any retry before you are guaranteed completion at 6s your performance will get worse.

How’s that different from the client doing the retry? Admittedly it isn’t. Except the client now has to wait until it’s guaranteed that the proxy would return!

A fried alerted me to one of the hackaday CTF games: microcorruption.com. I think every programmer, nay anyone using a computer, should play this!

Well, ok so this spoke to my obsessive nature and I really wish something this cool had existed when I was young, poor, and had the spare time to engage in some real hacking. Not that any of us did. Ever.

I admit it speaks to obsessive personalities and you probably won’t make it past the first 5 or so levels unless you have that obsessive streak, but even if you don’t: it’s a GAME. It has LEVELS. It TRAINS you and levels get progressively harder. And it has this cool hall of fame that shows you how much better you did than everyone else. And if you finish the tutorial and the first level you are already in the top 50% :)

And there’s lots of people playing it so you can get help if you ask. In fact it’s so popular that people are creating mods / tools / plugins to play this game:

• Add comments to assembly code (JS snippet)
• Add an extra info box for malloc lists (chrome plugin) … this is a shameless plug

As you may know I row. Well, mostly I scull, but for this year’s end-of-year event in Victoria, BC, the Head of the Gorge race, we decided to dress up and row an 8. So far that’s been a fairly good experience … we have costumes, we have airline tickets, we have hotel rooms … and even the boat works well enough for a bunch of people just thrown together.

Well, I hit a snag today when I made the suggestion to match our blades to our costumes. And then spent an inordinate amount of time today dealing with the insanity that is club politics. In the process we were discussing spinning off a separate club, just for the purpose of that race, so we could break the “club colors” rule: 2013 rowing canada racing rules, section 6.5. This was the proposed photo for the club president:

Of course reading rules is a dangerous thing … apparently (according to section 6.2 of the same rules) if all rowers are from the same club the can wear whatever they want as long as everyone is wearing the same thing … if rowers are from different clubs they must wear singlets. Who write these things? More on this after races …