Rust is Hard

I’m not a Rust expert by any means, but I’ve spent enough time with the language that I can write reasonable code with a bit of effort (and a lot of DuckDuckGo). However, I’ve found that I often hesitate to use it for my personal projects.

Rust is a large language. The sheer number of features and their complexity introduce so much cognitive overhead while writing code that I can’t always muster up the mental energy to use it for things I’m simply tinkering around with after work. It’s just not possible for me to keep the whole thing in my head.

If I was writing Rust for my day job, it would be a different matter. Using it to build production software every day would let me internalize it to the point that it would become mostly automatic. It would let me cement difficult concepts in my brain so that I wouldn’t have to go looking for explanations every five minutes (lifetimes, anyone?)

Sadly I only get to use Rust for one-off side-projects, which means I spend maybe two hours a week using it. This gives me barely enough time to get my code into a working state, let alone dive into things like the Rustonomicon or even macros.

I love Rust, so I’m going to continue using it despite my issues with it. I’ll just have to lower my expectations of the level of proficiency I can hope to attain with the language.

PS: somebody please give me a Rust job 😭

Emacs Sparks Joy

Much like how some people enjoy tinkering with motorcycles, electronics, or craft projects, I enjoy tinkering with software. If my computing environment stays the same for too long, I start getting restless. I crave constant change.

Emacs is a tinkerer’s dream, an infinite sandbox that can be molded into something entirely different each day. I can dive into the manual and discover new features, try different combinations of packages to see what’s most comfortable for me, glue together packages to make them do things they was never intended to do, write snippets of ELisp that help me get my work done faster, and so much more.

In this way Emacs is not only a tool that lets me do my job, but also a creative outlet that provides endless hours of entertainment and joy. I consider it one of the best pieces of software ever written.

Of course, there are other reasons for using Emacs — longevity, efficiency, ubiquity — but these features can be found in many other tools both free and commercial. Only Emacs is good at being Emacs.

Cooking (North) Indian Food

While health officials have said that COVID-19 doesn’t spread through food, especially if you heat it properly before consuming, I haven’t felt comfortable ordering food from restaurants since the government announced the lockdown in March. This means Ankush and I have been cooking all our own meals for the last two months.

I’ve cooked for myself in some capacity since I moved to Bangalore seven years ago, but this is the first time I’ve been forced to cook nearly every single meal myself. In the process, I’ve learned a few things. They may be familiar to folks who have been cooking for years, but each one of these tips made a huge difference to the quality of my cooking:

  • Ghee can make everything taste like it was cooked by the spirit of a dead Mughal chef, but it can also overpower all the other flavors in your food. I used to use a ton of ghee in all my cooking, but I now prefer using a neutral oil for most dishes.
  • Most North Indian recipes call for a 1:2 ratio of onions to tomatoes in the curry, but I like to use a bit of extra tomato to add some tang to the gravy. If I don’t have a small tomato lying around, I add a tablespoon of packaged tomato puree.
  • Using ginger-garlic paste, whether homemade or packaged, always results in tastier gravies compared to using chopped ginger and garlic.
  • The slower you brown your onions, the better your gravies end up tasting. About twenty minutes on a medium flame works great, but I’ve cooked them for even longer with good results.
  • You want to put either jeera powder or dhania powder in your gravies, but usually not both.
  • You want to put either jeera seeds or mustard seeds into your tadka, but usually not both.
  • Hing is a very potent spice, use it with care.
  • It can take a really long time to cook peas. I’ve boiling them in water for a few minutes before putting them into gravies, because sometimes they end up staying raw even when all the other ingredients in a dish are cooked through.
  • If you’re cooking chana or rajma in a pressure cooker, don’t forget to add salt. If you pressure cook them without salt, the resulting dish will taste bland even if you later add enough salt to the gravy.
  • If you don’t use up your rice fast enough, you’ll sometimes see small black bugs appear on the grains after a few weeks. These are rice weevils, and their eggs are invariably present on your rice even if you buy the really fancy brands. You can get rid of these eggs by putting the rice in the freezer for a few days before using it.
  • Common grains in descending order of cooking times are: white chana > rajma > toor daal > moong daal and basmati rice > masoor daal.

GoAccess

CloudFlare recently informed me that this website was getting thousands of hits every day, which is an unusual occurrence. I pulled up Google Analytics to figure out where all that traffic was coming from, only to be informed that I was getting three to four hundred daily visitors at most.

This felt a bit suspicious, so I dug into my Nginx logs to see if something was up. I pulled up the logs in Vim, but it was too hard to make sense of any of the raw data by reading it line-by-line. I needed something that could help me visualize my logs. I asked around and found a little tool called GoAccess.

GoAccess is an open-source log analyzer that can help you visualize your server logs in the terminal or use them to produce an HTML report. I installed it from the Ubuntu repositories and ran it:

goaccess /var/log/nginx/access.log

I was greeted with a dialog listing a bunch of popular log formats, asking me which one my file conformed to. After searching the Web for a bit, I figured out that Nginx uses something called the NCSA Combined Log Format for its messages. I picked that in the dialog and was on my way.

After a few minutes of looking at the aggregated data, I felt that analyzing just one file wasn’t telling me the entire story. I wondered if I could analyze all the logs produced by Nginx in the last month at once. After searching the Web a bit more, I found that I could use zcat to unzip the older logs and print them to stdout, and then pipe that into GoAccess. So I did this:

zcat -f /var/log/nginx/access.log* | goaccess --log-format=COMBINED

Turns out someone was trying to exploit my website by by sending malicious inputs to WordPress’s xmlrpc.php. From the access patterns, it looked more like a drive-by automated attack than a human trying to break in. Since I didn’t need any of the features enabled by the WordPress API, I blocked access to xmlrpc.php entirely:

location = /xmlrpc.php {
    deny all;
    access_log off;
    log_not_found off;
}

The downside to this is that I can’t post to my website from the WordPress app on my iPhone. But that’s not something I do often, so losing that feature is not a big deal.

After this incident, I also removed Google Analytics from my website. I’m fundamentally opposed to business models that are based on surveillance-based advertising, which is why I try to stay away from Google products as much as possible. My server logs give me enough data to judge how well my posts are doing, and with GoAccess I now have a reasonable way of querying and visualizing that data. That’s pretty much all I need.

E-Commerce Case Study: Building Faster Listing Pages on abof.com (Part 3)

This case study was first published on the Alaris Prime blog on January 4, 2017. You can read the original case study here.

Part 1 of this case study was a general overview of how the Alaris Prime team rebuilt abof.com to load almost instantly even on flaky mobile connections, and part 2 was an account of how we got to grips with the often confusing React ecosystem. If you haven’t checked out the first two parts yet, you should do so now.

In this final part of our case study, I’ll discuss a few specific issues we ran up against through the course of the project, and how we tackled them.

Keeping Track of Scroll Position in an Infinite Scrolling Grid

abof’s product listing page is an infinitely scrolling grid of product images that loads 12 items per “page”. When a user visits a listing URL, our CDN responds with a pre-rendered HTML page with an initial set of 12 products already loaded. Another 12 products are loaded asynchronously the moment our JavaScript bundle loads and React takes over the page. From this point on, a new set of products is loaded whenever the user scrolls to the last loaded page.

abof.com product listing page
Endless.

A recent post on the Google Developers blog talks about the challenges inherent in implementing an efficient infinite scrolling list in the browser. The post is recommended reading, and I won’t repeat the information it already covers in this case study. Instead, I’ll talk about how we use a URL to keep track of the user’s position within our infinite scrolling grid without slowing the browser down.

Two common causes of jank on pages that use infinite scrolling are:

  1. Event listeners on the document’s scroll event.
  2. Repeatedly querying the DOM from those event listeners.

With a little bit of work, we can avoid listening on the scroll event altogether, as well as keep DOM queries to a minimum.

Listing URLs on abof look something like this:

https://abof.com/women/clothing/dresses?page=xxx

That page=xxx bit at the end keeps track of the user’s position within the grid, and changes as she scrolls from page to page.

Every 12th product in the grid has a data-page-end property attached to its DOM representation that indicates that the product appears at the end of a certain page. For example, the product card at the end of the 4th page (i.e, the 48th product in the grid) is marked up as follows:

<div itemscope="" itemtype="http://schema.org/Product" class="product-card product-card--data-marker" data-page-end="4" data-product-id="205675">
    <!-- product details here -->
</div>

We call these elements page markers, and we keep track of them in an array called activePageMarkers inside our ProductGridContainer component. Whenever a new set of products is loaded, any page markers inside that set are appended to this array.

These page markers are references to actual DOM nodes within the document. Along with these references, we also keep track of their positions on the page, as well as their dimensions. This way, we don’t have to query the DOM for this information repeatedly as the user interacts with the page. We only recalculate it when the user triggers an event that is likely to invalidate our existing data (e.g, resizing the page or rotating the device).

Finally, we use requestIdleCallback() to fire a function called syncPageLocation() whenever the browser is idle, throttle it so it fires at most once every 500ms, and make sure it doesn’t fire if the user hasn’t scrolled the page for a while.

syncPageLocation() uses the browser’s scroll offset and the position data stored in activePageMarkers to find the page marker closest to the bottom of the page. It then extracts the value of data-page-end from that element, and uses history.replaceState() to change the page=xxx bit in the URL to reflect the value stored in data-page-end.

This machinery allows the user to share the URL of the listing page over IM, email, or social media with the confidence that anybody who follows it will see the same set of products that were on her screen a moment ago. Moreover, it allows users to move back and forward between product detail pages and listing pages without losing her position in the grid.

Analytics with Google Tag Manager and Redux Middleware

Analytics on abof.com are powered by Google Tag Manager hooked up to a number of third-party analytics providers.

On each page, the analytics team at abof wanted to capture a number of custom events tied to specific user interactions. We wanted to do this in a way that none of our components had to be made aware of analytics or GTM.

We started out by making a list of all the custom events that the analytics team wanted to capture. For example, they wanted to capture a bunch of data about the current page whenever the user changed its sort option from the default value of “Popularity” to one of the other available options (“Just In”, “Discount — High to Low”, “Price — Low to High”, “Price — High to Low”).

Then, we mapped each interaction to one or more of our React components. The components mapped to each user interaction would emit a Redux action containing all the data we needed to capture about that interaction. For example, the SortDropdown component would emit an action called SORT_OPTION_CHANGED every time the user changed the sort option on a page. This action looked something like this:

{
  name: 'SORT_OPTION_CHANGED',
  payload: {
      from: 1,
      to: 4
  }
}

In the payload object, the from field kept track of the sort option before the user changed it, and the to field kept track of the new sort option.

Of course, our components were not aware of all the data required by a analytics event. For example, the SortDropdown didn’t know whether the user was logged in, what her IP address was, or even the current page URL. We didn’t want our components to be analytics-aware, so we only had them capture the data that they actually had access to. We filled in the missing bits using a Redux middleware called gtm.

The gtm middleware looked at each Redux action that we were interested in, created one (or, in some cases, more than one) analytics events for each action, filled in any missing information that the events required, and pushed them into GTM’s dataLayer array.

This architecture allowed our components to be oblivious of GTM while still allowing analytics data to be collected at a very granular level.

Caching Pre-Rendered Pages for Logged-in Users

Once our universal React app renders a product listing page on the server, abof’s CDN caches it for 10 minutes. This not only shaves a few hundred milliseconds off our load time, but also helps keep abof’s server bills down.

This optimization is straightforward to apply to requests that come from customers who are not logged into their abof accounts. Any given listing page will look identical to all of these anonymous users, which means we can serve them whatever the CDN has cached.

However, we can’t blindly serve a cached page from the CDN to a user who is logged into abof. A customer who is logged in sees a few extra bits of information on each listing page:

  1. Her username, with a link to her profile, on the top right corner of the page (on mobile, this appears in the hamburger menu).
  2. A dropdown listing all the items she’s added to her cart.
  3. If she’s added an item to her favorites, the tiny heart icon on the top right of each product image is filled-in.

abof.com top navigation (logged in)
We display the user’s name and bag contents when she’s logged in.

Since this information varies from user to user, caching the page is not an option for logged-in users. On the other hand, letting our universal application deal with every request that comes from a logged-in user means it now has to handle a load it was never designed for.

We work around this problem by serving the same cached listing pages from our CDN to every single user — logged in or not — and having JavaScript fill in the missing information after page load.

This is what a typical page load looks like:

  1. User makes a request to a listing page.
  2. The CDN serves up a static HTML page that doesn’t contain any user-specific information (i.e, no cart, no favorites, no username). At this point the user can start interacting with the page.
  3. Our JavaScript bundle loads, and React takes over.
  4. Our root component makes a request to a REST endpoint that returns user information.
  5. If the endpoint returns valid information, our app knows the user is logged in. At this point, it makes requests for cart items, favorites, and whichever other bits of information are required to customize the page for this specific user.
  6. If the endpoint doesn’t return valid information, our app knows the user is not logged in. It doesn’t need to do anything special to handle this case.

This architecture is not perfect. On slower connections, the user sees page elements move around and change as we fetch the extra information needed to assemble the page. However, it lets us eke out that last bit of performance from an already fast webpage.

Final Words

In this final part of our case study, I talked about three specific problems we faced while rebuilding the listing pages on abof.com:

  1. Keeping track of page URLs as users scroll through abof’s infinite scrolling grid of products.
  2. Using Google Tag Manager and Redux middleware to collect granular analytics without impacting page performance.
  3. How caching works in an application that uses universal rendering.

In case you missed the first two parts, you can read them here: part 1, and part 2.

Published
Categorized as Articles

E-Commerce Case Study: Building Faster Listing Pages on abof.com (Part 2)

This case study was first published on the Alaris Prime blog on October 6, 2016. You can read the original case study here.

If you haven’t read the first part of this case study, I suggest you go check it out before diving into the second part. It’s a quick read that explains in detail our motivations for the technology choices we made while building the new abof.com.

Done? Great! In this second part, I’ll talk about my and my team’s impressions of the React ecosystem, our opinions on build tooling, and our approach to performance testing.

Learning React

When we started working with abof, all of us were primarily Angular 1.x developers. We had used the framework to build several complex applications, which meant we had the ability to ship quality Angular code rapidly.

However, with a stable release of Angular 2 right around the corner, starting a new Angular 1.x project would have been irresponsible on our part.
My experience with building a small application using Angular 2 a few weeks prior to the start of the abof project had left me with mixed feelings. I personally enjoyed working with the framework and found it a welcome improvement over Angular 1.x, but I had to admit that the number of concepts a newcomer to the framework must wrap her head around just to build a functioning TODO list application with Angular 2 was unjustified.

Besides, as I mentioned in part 1, there were other issues with Angular 2 that made it a no-go for us (mainly the large payload size, and the lack of support for universal rendering).

With some apprehensiveness, we began the process of learning React — and what looked to us like a glut of supporting libraries that were apparently absolutely required to produce a working application. There were a number of tools and libraries that we either didn’t understand the purpose of, or didn’t know if we needed. Redux, Radium, Immutable.js, MobX, Relay, Falcor, Flow, Babel, Webpack, just to name a few.

Despite this fractured and confusing landscape, learning React turned out to be easier (and way more fun!) than we anticipated.

We found that there is only one thing that is absolutely required to build React applications: React. Learning it takes a few hours, and — besides the official docs — there are plenty of tutorials on the web that can accelerate and supplement the learning process. I’m partial to the tutorials on egghead.io.

After we wrapped our heads around React, we built a small prototype that pulled product data from the abof.com REST API and rendered it as a grid. No fancy JavaScript preprocessors, no supporting libraries, just plain old ES5 and React. Over the course of a week we added a few more features to this prototype (routing, pagination, infinite scrolling), but it was mostly an experiment that never made it to production.

Having built this throwaway prototype, we were in a position to take a deeper look at the React ecosystem and understand what problems each of the popular libraries was designed to solve. For example, after having spent a day scratching our heads over how to elegantly share state between components, we had a better appreciation of the problems Redux solves.

This exercise let us choose a subset of the libraries that were relevant to us from the plethora of libraries available to us.
In the end, the structure of our application was very similar to what a tool like create-react-app would produce, and our list of dependencies was no different from what any standard React application written in 2016 would use. However, by taking a YAGNI approach to building abof, we were able to understand what purpose each of our libraries served at a deeper level. Most importantly, it kept us from getting overwhelmed with new tools and concepts right at the beginning of the project.

Build Tooling

We wrote most of our build system from scratch, adding tools and features as we went. This often caused us pain — for example, adding support for isomorphic rendering after most of the application was already written and functional cost us a few days of development time. We had to rewrite parts of our codebase to make sure they ran correctly on Node.

Our build system did nothing out of the ordinary, but knowing it inside-out gave us the confidence to jump into our Webpack and Babel configurations and tweak things to our heart’s desire. It also helped us automate our release process to a point where building and deploying a new version of the website was a single command.

Would I recommend that every team assemble their build tooling in this piecemeal manner? No. As much as we learned from this exercise, starting with one of the hundreds of available React boilerplates on GitHub and carefully studying its source code would have been a more productive exercise and given us an equal amount of confidence in our tooling.

If you’re starting a new React project now, don’t even think twice about using create-react-app.

Measuring and Optimizing Page Load Performance

Our primary source of insight while measuring page load performance was using the website on real devices connected to real mobile networks. A number of tools exist to simulate different network conditions and spit out numbers, but we found that seeing what our users see on flaky connections and devices was valuable while optimizing our page load times.

WebPageTest and PageSpeed Insights are great for giving you hard numbers to target while building or optimizing your application, and for pinpointing exact areas of your application that need work. However, only by testing on real devices will you know which optimizations directly enhance your users’ experience and which ones shave a few seconds off your loading time without affecting perceived performance in any way.

Our second source of performance metrics was the Chrome developer tools. Even while developing locally, we tested the website with a throttled connection. This pushed us to keep the number of API requests and payload sizes small. We set ourselves a page size quota, which was 150kb of data minified and gzipped. That sounds generous, but we got away with it because we were serving a pre-rendered page to the user.

Our final source of performance metrics was WebPageTest. We ran both WebPageTest and PageSpeed Insights after deploying to our staging server to surface issues we might have overlooked. There are far too many things that can go wrong while building a web application, and automated testing tools serve as — for lack of a better term — interactive checklists that will help you ensure you comply with all the best practices. If it hadn’t been for WebPageTest, we would have never realized that the cache headers on our product images were all wrong, or that we could compress them more aggressively.

Measuring Application Performance

Just like page load performance, our primary source of insight while measuring application performance was using the application on a real device. We had access to a number of low-end mobile phones running Android and Windows Phone, and we would periodically (usually after a staging deploy) test the website on these to make sure abof performed acceptably.

Final Words

This part of the case study was a mostly subjective look at the React landscape. In the last and final part, I’ll talk about some specific issues we ran into while building abof and how we tackled them.

Updates

Read part 1 and part 3 of this article series.

Published
Categorized as Articles

E-Commerce Case Study: Building Faster Listing Pages on abof.com (Part 1)

This case study was first published on the Alaris Prime blog on June 8, 2016. You can read the original case study here.

abof.com (pronounced ae-boff dot com) is an online fashion store that’s part of Aditya Birla Group’s e-commerce strategy. Earlier this year, the company brought in the Alaris Prime team along with Ciju from ActiveSphere Technologies for a complete rewrite of the product listing page on abof.com. After we delivered the rewrite, the load times for the page on 3G connections went from ~20 seconds to ~7 seconds, and bounce rates decreased by over 40%. These improvements have encouraged abof to invest a significant chunk of their technology resources into web performance, in particular the React and Redux ecosystem.

In this three-part case study, I will talk about abof’s motivations for the rewrite, the technology choices our team made to meet abof’s business goals, our rationale for the choices we made, and our experiences with React, Redux, and the ecosystem that has emerged around these libraries.

Motivation

Most Indians access the web using mobile phones, and this fact is reflected in the analytics data collected on abof.com: at the time we started working with abof, more than 60% of the website’s traffic came from mobile users.

While the legacy version of the website adequately served the needs of desktop users, it had three issues on mobile:

  1. For first-time visitors on 3G connections, a first paint of the product listing page could take over 20 seconds. Other pages on the website had similar first paint characteristics.
  2. JavaScript performance on most pages was poor, even on high-end mobile devices.
  3. Since the website was initially built for desktop and only later adapted to smaller screens, the mobile user experience was sub-optimal.

These performance issues were the root cause of low conversion rates on mobile, with as many as 50% of new visitors dropping off after the first page load.

In line with industry practices, abof’s mobile strategy was to provide users with a minimally useful experience on the mobile web while pushing them to install the company’s native Android/iOS apps, which would unlock the full shopping experience. However, relying purely on native apps to drive mobile sales has been a losing proposition in the developing world for a while now for a number of reasons:

  1. Retention rates for native apps have been historically poor. Unless an application serves a very specific need, chances are users will uninstall it within about a week of trying it out.
  2. The on-boarding flow for a native application has a huge amount of friction: the user has to visit a website, click through to the application’s page on the App Store/Google Play, and wait for the application to download and install before she can use it. Unless there’s a compelling reason for her to install an application, the user will not jump through the hoops.
  3. App fatigue has set in. Nobody wants to install yet another app.
  4. Users are wary about installing new applications because, more often than not, they slow down their devices, take up memory, and drain battery life. This is especially true in countries like India where most handsets on the market are vastly underpowered.
  5. Users are sick of notification spam.

Industry heavyweights Flipkart and Myntra — among others — have tried an app-only strategy, only to re-launch their mobile websites, allegedly in the face of dropping conversion rates.

Meanwhile, the introduction of new browser APIs — most notably the ServiceWorker API — has enabled mobile webapps to provide the same level of engagement to users as their native counterparts. The webapps of today are performant, run offline, can be launched from the users’ home screens in a chromeless window, and can engage users using push notifications even after they’ve navigated away.

The product team at abof has always understood the importance of a good user experience across all platforms, including the mobile web. However, these recent shifts in the mobile landscape in India pushed abof to not just bring the mobile web UX on abof.com up to par with native apps, but to make the web the centerpiece of their mobile strategy. Enter Alaris Prime.

Goals

The analytics team at abof identified three areas of the website where the largest percentage of mobile users would drop off: the product listing page, the login/signup page, and the checkout process.

Out of these three areas, the primary point of entry into the website for first-time users is the product listing page. This is where the sales funnel starts. It made sense, then, to begin with a rewrite of the listing page and immediately put it into production in order to measure the impact of improved performance on conversion rates.

Our primary goals while building the new listing page were:

  1. Minimize time to first paint on 3G connections.
  2. Improve JavaScript performance across all device classes.
  3. Improve user interactions on smaller screens.

To achieve these goals, we followed a few tried-and-tested guidelines:

  1. Keep the payload size and number of resources delivered to the browser as small as possible.
  2. Minimize network calls made from the client.
  3. Use third-party libraries only if absolutely necessary. When using a library, make sure its performance impact is well understood.
  4. Test performance on real devices. We usually kept a stack of cheap Android and Windows Phone handsets on our desks while developing, and we’d take some time out daily to test our new code on each of them.
  5. To ensure the page shows up as quickly as possible, pre-render the initial HTML on the server and deliver it to the user.
  6. Lazy load resources whenever possible.

Technology Stack

At the time we started our engagement with abof, all of us were Angular developers. We had put numerous Angular 1.x applications of varying complexity into production, and we had been evaluating Angular 2 for new projects. While we loved the direction Angular 2 was headed in, the framework had a few issues that rendered it impractical for our purposes:

  1. A Hello World application written with Angular 2 weighed in at over 150 kilobytes. This was completely unacceptable for an application that had to be delivered over flaky 2G and 3G networks.
  2. The Angular team had promised support for server-side (or isomorphic) rendering of webapps, but it was not clear when it would land.
  3. The tooling around the framework was not very mature.

Besides Angular 2, we also evaluated Vue.js and Riot.js. Both of these are powerful libraries with tiny footprints, great performance, and support for isomorphic rendering, but the communities around them aren’t as large as the ones around React and Angular. Basing our rewrite on top of one of these libraries would have made maintenance and hiring harder for abof.

In the end, we settled on a battle-tested technology stack built around React and Redux:

  1. We used React as our view library. It’s small (about 56 kilobytes with everything included), easy to learn, blazing fast, well-supported by a wonderful community, and has great tooling built around it. Its performance characteristics on different devices and browsers are well understood and it has great support for isomorphic rendering.
  2. We used Redux to manage application state. It brings together a small number of composable ideas to elegantly tackle a hairy problem.
  3. Webpack was the workhorse that had the primary function of slurping up our ES6 code, turning it into ES5 code, analyzing and digesting it, and spitting out an optimized 150 kilobyte JavaScript bundle that we could deliver to our users. Besides that , it helped us generate separate builds for the browser and node from the same codebase, combine and inline our SVG icons, modify aspects of the build depending on environment variables, and a variety of other niceties that I’ll discuss in upcoming articles.
  4. Gulp helped us automate pretty much everything except ordering chai (soon!).
  5. Koa rendered our application on the server-side.
  6. CSS frameworks are convenient for quick prototypes, but they do more harm than good on production projects. All our styles were written by hand using SASS, with some help from Bourbon.

Performance Comparisons

What follows is a visual comparison of the legacy abof.com product listing page with the rewrite our team delivered on a 3G connection:

abof.com performance comparison
New React-powered product listing page vs the legacy product listing page. Click to view video.

That’s over twice as fast! Here’s a video comparing the new abof.com product listing page with listing pages on some of the most popular e-commerce websites in India:

Final Words

In the next two parts of this case study I will go into more details about our experiences with React and Redux, our thoughts on the current state of front-end tooling, the methodologies we used for gauging and improving both perceived and actual performance of the new abof.com, and how we used SASS and PostCSS to successfully prevent CSS-induced hair loss through the course of the project.

Updates

Read part 2 and part 3 of this article series.

Published
Categorized as Articles

Seven Languages in Seven Weeks, Week 2: Io

Designed by Steve Dekorte, Io is a small, embeddable programming language that borrows its prototype-based object model from Self, its purely object-oriented nature from Smalltalk, and its homoiconicity from Lisp (although, unlike Lisp, it doesn’t use s-expressions to represent programs). The language is such a mind-expanding experience that I have now spent way more than a week playing with it.

Syntax

Io’s syntax takes only a few minutes to learn. In short, everything in Io is a message that is passed to a receiver:

Io> receiver message

A message can accept arguments:

Io> receiver message(param1, param2, ...)

And finally, a message without a receiver is sent to the top-level object called Object:

Io> writeln("this message is sent to Object")

That’s it. Any other syntax you see is sugar that gets translated into this simple form.

The receiver can choose whether it wants to evaluate a message or not, which allows you to do so selectively in order to implement domain-specific languages. For example, Io has an if conditional like any other language:

Io> if (a > 10, "more than 10", a = a + 10)

A simple re-implementation of if would look something like this:

Io> myIf := method(
  call evalArgAt(0) ifTrue(call evalArgAt(1)) ifFalse(call evalArgAt(2))
)

And here’s how you’d use it:

Io> a := 10
Io> myIf(a == 10, "a is 10" println, "a is not 10" println)
a is 10
Io> a = 11
Io> myIf(a == 10, "a is 10" println, "a is not 10" println)
a is not 10

Prototypes

Io has a prototype-based object system, which it borrows from Self. After learning how Io deals with objects, I started to investigate JavaScript’s object model in greater depth. As a result, I walked away with a much better understanding of OOP in JavaScript.

In a language with a prototype-based object system, new objects are created using existing objects as templates. For example, in the next block of code, Animal is a clone of the top-level Object. It contains all the slots (or properties) of Object.

Io> Animal := Object clone

We can use the := operator to add a new slot to Animal:

Io> Animal talk := method(writeln("This animal can't talk."))

Cat is a clone of Animal. It gets all the slots of Object, as well as the talk slot defined on Animal.

Io> Cat := Animal clone
Io> meep := Cat clone
Io> meep talk
This animal can't talk.

However, it can have its own talk slot too.

Io> Cat talk := method(writeln("Meow!"))
Io> meep := Cat clone
Io> meep talk
Meow!

Likewise, Cow is a clone of Animal, but it doesn’t have its own talk slot. It always uses the talk slot from Animal.

Io> Cow := Animal clone
Io> daisy := Cow clone
Io> daisy talk
This animal can't talk.

The equivalent code in JavaScript is:

// In a file called animals.js

'use strict';

function Animal() { }

Animal.prototype.talk = function() {
  console.log("This animal can't talk.");
};

function Cat() {
  Animal.call(this);
}

Cat.prototype = Object.create(Animal.prototype);
Cat.prototype.constructor = Cat;

Cat.prototype.talk = function() {
  console.log("Meow!");
}

function Cow() {
  Animal.call(this);
}

Cow.prototype = Object.create(Animal.prototype);
Cow.prototype.constructor = Cow;

const meep = new Cat();
meep.talk(); // Prints "Meow!"

const daisy = new Cow();
daisy.talk(); // Prints "This animal can't talk."

Domain-specific Languages

Like Ruby, Io lets you build powerful DSLs. However, Io’s DSLs are far more powerful on account of its homoiconicity, and they can go as far as changing the very syntax of the language. In this regard, Io is similar to Lisp and its descendants.

Here’s an example straight from Steve’s book. Creating and using a map (a collection of key-value pairs) in Io looks something like this:

Io> map := Map clone
Io> map atPut("foo", "bar")
Io> map atPut("baz", "quux")
Io> map at("foo")
==> bar

Let’s add JavaScript-esque object literal syntax to the language, which will enable you type the following into the Io interpreter and get a built-in Map object:

{
  "foo": "bar",
  "baz": "quux"
}

First, we add a new assignment operator, represented by the colon (:), to Io’s operator table:

Io> OperatorTable addAssignOperator(":", "atPutValue")

Now whenever Io encounters a colon, it will translate it to the message atPutValue, with the item on the left of the colon as the first argument, and the item on the right as the second argument. So, the following code:

Io> "foo": "bar"

Is translated to:

Io> atPutValue("\"foo\"", "\"bar\"")

Notice the extra quotes around “foo” and “bar”. This is because Io treats all values passed to the assignment operator as strings.

Next, we define a new slot called atPutValue on the built-in Map:

Io> Map atPutValue := method(
  self atPut(
    call evalArgAt(0) asMutable removePrefix("\"") removeSuffix("\""),
    call evalArgAt(1) asMutable removePrefix("\"") removeSuffix("\"")
  )
)

This method removes the extra quotes from around its arguments, and passes them on to the built-in atPut method defined on Map.

Finally, we define a new slot called curlyBracket on the top level Object. Io will call the method stored in this slot every time it encounters a pair of curly brackets.

Io> curlyBrackets := method()

Inside this method, we create a new Map:

Io> curlyBrackets := method(
  m := Map clone
)

Next, we take each argument passed to curlyBrackets and send it to our new Map for evaluation. In the end, we return the Map:

Io> curlyBrackets := method(
  m := Map clone
  call message arguments foreach (arg,
    m doMessage(arg)
  )
  m
)

Now the following syntax will produce a new Map:

Io> { "foo": "bar", "baz": "quux" }

First, Io parses each key-value pair inside the curly braces. Since we’ve defined “:” to be an assignment operator that is equivalent to the message atPutValue, each key-value pair gets parsed into that message.

Next, all items within the curly braces are collected into a list and passed to the curlyBrackets method on Object. In the end, the JavaScript-esque syntax above gets parsed into this method call:

Io> curlyBraces(
  list(
    atPutValue("\"foo\"", "\"bar\""),
    atPutValue("\"baz\"", "\"quux\"")
  )
)

Finally, our definition of curlyBraces creates and returns a new dictionary for us.

Closing Thoughts

While the Io website has a tutorial, guide, and language reference,  it’s hard to find any additional information about the language on the Web. There seems to be very little activity on the GitHub repository, mailing list, or the subreddit. All the Io-related blog posts I found study were notes written by people working their way through Bruce Tate’s book.

For all practical purposes, Io is abandonware.

Regardless of its current status, Io’s simplicity, elegance, and extensibility puts it in the same league as Lisp and Smalltalk. Even if you don’t end up using the language in a project, learning it will make you a better programmer.

I plan to come back to Io in the future, when I have some more time to tinker with language implementations. For now, it’s on to the next language!

Published
Categorized as Articles

Seven Languages in Seven Weeks, Week 1: Ruby

In an attempt to get back into programming language theory and implementation—a hobby I’ve neglected since I started working full-time—I recently started reading Bruce Tate’s Seven Languages in Seven Weeks. These are my notes and observations from my first week of study.

In week 1, Bruce introduces Ruby, drawing attention to its dynamic nature, expressive syntax, and metaprogramming capabilities. Together, these properties make it a suitable language for building natural, English-like APIs.

I didn’t think I could learn anything from Ruby that years of writing Python and JavaScript hadn’t already taught me. However, after three days of studying the language, I was pleasantly surprised to be proven wrong. There’s a lot to learn from Ruby about designing languages for humans first and machines second.

Syntax

Most of my experience with dynamic languages has been with Python and JavaScript, both of which are conservative with syntax sugar. This is a design decision I have always appreciated but, after acquainting myself with some Ruby libraries, I feel there’s an argument to be made in favor of liberally adding syntax sugar to make expressing common programming idioms more convenient.

The downside of all the sugar is that Ruby’s syntax comes with many surprises. For example, I can define a method that accepts a block as its final argument like so:

def myFun n, &block
  # do something nice
end

However, I can’t define a method that accepts two blocks as arguments using the same syntax, since the ampersand is a special bit of syntax reserved for defining methods that accept a block as their final argument. So, this is an error:

def myFun &block1, &block2
  # do something nice
end

There are a number of ways of calling a method, which can seem overwhelming at first. For example, consider the following method:

def myFunction param, &block
  # do something nice
end

Both these ways of calling myFunction are correct:

myFunction 1, { |n| puts n }
myFunction(1) { |n| puts n }

But the following is a syntax error:

myFunction(1, { |n| puts n })

If, however, we change the definition of myFunction so that it does not accept a block as its second argument:

def myFunction param1, param2
  # do something nice
end

Then, it can be called in the following ways:

myFunction 1, 2 
myFunction(1, 2)

But the following is a syntax error:

myFunction(1) 2

Things can get quite murky when we start talking about defining and calling methods, especially when we throw the ampersand operator into the mix. You win some you lose some.

REPL

For certain classes of programs, I like to use a REPL for interactive development. Ruby comes with irb, which is passable but not great. It helps while debugging and exploring language concepts, but it’s not powerful enough to let you interactively build and test programs.

I’d like to shout out ipython here, which is the best non-Lisp REPL I’ve used in my career.

Introspection

Both Python and JavaScript have great introspection capabilities, but they never feel as natural and as they do in Ruby.

For example, I wanted to check if there was a way to convert a Ruby hash into a bunch of key-value pairs. I knew that, in Ruby, the names of all methods that convert one data type into another conventionally have the prefix “to_”. Armed with this knowledge, I only had to write this bit of code to list all methods on a hash that converted it into a different object:

{ :foo => :bar }.methods.select { |m| m.to_s.start_with? "to_" }

Just for comparison, the equivalent in JavaScript would be:

const isMethod = obj => typeof obj === 'function';

const listAllMethods = obj => {
  const ownMethods = Object.getOwnPropertyNames(obj).filter(p => isMethod(obj[p]));
  const proto = Object.getPrototypeOf(obj);
  
  if (proto) {
    return [].concat(ownMethods, listAllMethods(proto));
  } else {
    return ownMethods;
  }
};

const s = {
  foo: 'bar'
};

listAllMethods(s).filter(m => m.startsWith('to'));

 

Metaprogramming

Metaprogramming is Ruby’s strong suit, and there isn’t much I can say about it that hasn’t already been said. Rails’ ActiveRecord is perhaps the best example of how metaprogramming can help build clean, natural-looking APIs.

My opinion of metaprogramming has always been that it’s great for people building libraries and frameworks, but not so great for people building applications. Too much magic can make code unpredictable and hard to debug.

However, I haven’t spent enough time with Ruby to know for sure how the liberal use of metaprogramming affects code clarity and maintainability in a large codebase. Most programming languages make it circuitous to do any kind of metaprogramming, and the APIs are usually an afterthought. In Ruby, the metaprogramming APIs are so well-designed and natural that—used in moderation—metaprogramming might actually enhance code clarity without any negative affect on maintainability.

As an example of the power of metaprogramming, Bruce presents a class that returns the Arabic equivalent of a Roman numeral whenever it’s accessed as a static property. I.e,

irb(main):001:0> Roman.X
=> 10
irb(main):002:0> Roman.XC
=> 90
irb(main):003:0> Roman.XII
=> 12

In Ruby, the Roman class looks something like this:

class Roman
  def self.method_missing name, *args
    roman = name.to_s
    roman.gsub!("IV", "IIII")
    roman.gsub!("IX", "VIIII")
    roman.gsub!("XL", "XXXX")
    roman.gsub!("XC", "LXXXX")

    (roman.count("I") +  
     roman.count("V") * 5 +
     roman.count("X") * 10 + 
     roman.count("L") * 50 +
     roman.count("C") * 100)
  end
end

I attempted to translate this into JavaScript using ES6 Proxies:

'use strict';

String.prototype.count = function(rx) {
    return (this.match(new RegExp(rx, 'g')) || []).length;
};

const Roman = new Proxy({}, {
    get: (_, property) => {
        let roman = property.slice();
        roman = roman.replace(/IV/g, 'IIII');
        roman = roman.replace(/IX/g, 'VIIII');
        roman = roman.replace(/XL/g, 'XXXX');
        roman = roman.replace(/XC/g, 'LXXXX');

        return (
            roman.count('I') +
            roman.count('V') * 5 +
            roman.count('X') * 10 +
            roman.count('L') * 50 +
            roman.count('C') * 100
        );
    }
});

Not too bad, huh?

Published
Categorized as Articles

WordPress is Maximum Cool

If you dig into my post history on this blog, you’ll find I’ve written a lot about blogging platforms.

When I started writing a blog, back in the day before memes and Snapchat, I got myself an account on WordPress.com because that’s what you did in those times. Well, okay, you could also set up shop on Blogger or LiveJournal, but I was one of those people who wanted all the software to be GPL and all the content to be CC-BY-SA. My obsession with open source and open culture, coupled with my belief that nu-metal was a valid art form, ensured that I didn’t have too many friends growing up.

By the time I got to high-school, I realized that software wasn’t magic and, if you were smart, you could build your own. When I graduated from blogging about high-school drama to blogging about open-source mailing-list drama, I built myself a little blogging tool using Python and Django. It was a great learning experience, but a few months into it I realized that maintaining your own blogging software is as boring as sitting through a J Cole album. Frustrated, I moved all my blog posts to a self-hosted WordPress install. You live and learn, right?

Just as I was starting to enjoy actually writing an actual blog that actual real people actually read, the Internet told me I was a schmuck for using WordPress. WordPress was built with PHP, and  PHP was for uncool dads who wear New Balance and cargo shorts. If I wanted to be cool, I had to use something called Jekyll, which was written in Ruby. Writing Ruby makes you literally Miles Davis, or so I was told. I wanted to be Miles Davis, so I moved all my old posts from WordPress to Jekyll. I even wrote a custom theme for my blog, and made it responsive because some guy named Steve Jobs put a web browser in a phone and suddenly 1280×800 wasn’t the only game in town. Steve made many contributions to humanity, but even he couldn’t make New Balance cool.

After I got the hang of Jekyll, things started looking up for me. I lost a lot of weight, fell in love, and learned how to properly iron my shirts. A designer friend told me she liked the colors on my blog. Macklemore admitted Kendrick got robbed. I wrote quite a bit and life was perfect, but then Medium came along and everyone I met on the street was like, “Bro have you checked out Medium yet?”

I went home and checked out Medium, and discovered it was a cross between a GIF gallery, an emoji keyboard, and a stock photo website. The combination was compelling enough, and I immediately got myself an account. Right about this time, having to rely on a bunch of build tooling to post to my Jekyll blog was starting to frustrate me. It meant that I couldn’t publish my posts from anything but my work computer.

One Friday night I drank too much whiskey by myself and migrated all my Jekyll posts to Medium. Jekyll was still cool, but I’d been told that Medium was cooler and I’ve always strived to be maximum cool.

Medium was great for writing, and even better for getting more eyeballs on my posts, but within a few months I started to notice a decline in the number of people reaching out to me after reading something on my blog. When I published a useful post on my self-hosted WordPress or Jekyll blog, people usually stuck around for long enough to click on my about page. From there, they ended up contacting me either with the intention of hiring me, or just to thank me for something I’d written. This behavior was reflected in my analytics data.

The reason for the decline in communication after I moved to Medium was that the platform doesn’t give you space to talk about yourself. You can enter a short bio on your profile page, and a description for your publication if you create one, but there really is no way on Medium to maintain a regularly updated about page, or a page listing your public talks, or one listing your work. All of this content has to be hosted on an external service, at which point there’s little reason to use Medium in the first place, at least in my opinion. I want a single tool that I can use to centralize my online presence, and unfortunately Medium is not it. This is not Medium’s fault. The platform is just designed for a different use case.

In the six months I spent writing exclusively on Medium, nobody reached out to me over email. I got quite a few comments on my posts, but the conversations never went beyond technical discussions.

WordPress is maximum cool because it gives me total control over my online identity. Even a managed blog on WordPress.com is leagues ahead of anything Medium has to offer in terms of customization and having a corner of the Web to myself, to do with as I please.

They may not sound like groundbreaking features, but the ability to change some CSS, add some text or a few links to a sidebar, or create a few pages on your blog talking about yourself and your work goes a long way when it comes to having an identity of your own on the Web.

I’ve considered trying out other self-hosted blogging software – Ghost being the one that excites me most – but the theme and plugin ecosystem around WordPress is so large that everything I want to do with the software is usually a Google search away.

In summary:

  • If you want to write a blog, don’t start by writing your own blogging software.
  • Jekyll is great, but I like being able to write from my iPhone and home computer.
  • Medium is too limiting in terms of how I portray myself on the Web.
  • Ghost doesn’t have the kind of community and plugin ecosystem that WordPress has.
  • Therefore, WordPress is maximum cool.

From my perspective, WordPress is a solved problem. PHP is fast enough, hosting is cheap, there are plugins for everything, customization is a cinch, and Santa Claus is real. After I’ve set everything up as I like it, I don’t really have to think about the software anymore and I can focus on writing.

I’ve moved all my writing back to a self-hosted WordPress and I intend to keep it here.

Published
Categorized as Articles