testing-node-apps-with-docker-compose (and some Soul)

Contents of this post

  • Purpose: the reason for this blog post.
  • Scenario: what this example of using docker-compose can be useful for.
  • Prerequisites: basic setup to be able to run the code contained in this post.
  • Code example: an actual step-by-step guide on how you can setup your test environment to run with docker-compose.
  • Improvements: a couple of ideas on how to expand this technique.

Purpose

As I am sure the audience of this post knows to some extent, Docker is a technology that has grown to become popular over the last few years, allowing developers to deploy pieces of software by packaging them into standardized containers, in a number of various ecosystems (Apache Mesos, Amazon Web Services and many more).

So we can use Docker for our deployment needs, awesome. But let’s pay attention to a key word I used above. Docker grants isolation. And what do we like to perform on our application in isolation? Yeah, you guessed right – testing!

Specifically, with this post, I aim to dig deeper into how to use docker-compose (a specific Docker-based tool that enables creation of multi-container Docker applications) to build and run a Node.js application connected to MongoDB, to test their interaction and the interaction of the app with the external world, all inside containers running on your machine. All isolated and testable thanks to the usage of containers that we can spin up, hit with tests, and clean up with little effort.

[Read More]

falcor-postman

At OpenTable, we have an engineering culture that empowers us to research, experiment and learn.

In an effort to foster innovation and to try new ideas, Chris Cartlidge, Nick Balestra, Tom Martin and myself started to work on a side project nicknamed big-mars. Our project is a mobile-first, responsive web application that uses Falcor by Netflix.

Falcor, a JavaScript library for efficient data fetching, is an implementation of the Backend for Frontend (BFF) or the API Gateway pattern.

One powerful concept that Falcor has is its query model, in which you access your data as if it was a single JSON model in memory. You can navigate your data structure the way you would navigate a JSON object; either with a “dot notation” (e.g. restaurant.address.city) or with an “array notation” (e.g. restaurant[‘address’][‘city’]).

Really easy and intuitive.

[Read More]

OpenComponents - microservices in the front-end world

Many engineers work every day on opentable.com from our offices located in Europe, America, and Asia, pushing changes to production multiple times a day. Usually, this is very hard to achieve, in fact it took years for us to get to this point. I described in a previous article how we dismantled our monolith in favour of a Microsites architecture. Since the publication of that blog post we have been working on something I believe to be quite unique, called OpenComponents.

Another front-end framework?

OpenComponents is a system to facilitate code sharing, reduce dependencies, and easily approach new features and experiments from the back-end to the front-end. To achieve this, it is based on the concept of using services as interfaces - enabling pages to render partial content that is located, executed and deployed independently.

OpenComponents is not another SPA JS framework; it is a set of conventions, patterns and tools to develop and quickly deploy fragments of front-end. In this perspective, it plays nicely with any existing architecture and framework in terms of front-end and back-end. Its purpose is to serve as delivery mechanism for a more modularised end-result in the front-end.

OC is been in production for more than a year at OpenTable and it is fully open-sourced.

Overview

[Read More]

Testing React Components

At OpenTable it’s becoming an increasingly popular trend to use React.
One of the reasons for this is the ability for it to server-side render whilst still
giving us the client side flexibility that we all crave!

We all know to have stable, reliable software you need to have well written tests. Facebook knows this and
provides the handy Test Utilities library to make
our lives easier.

Cool — I hear you all say! But what is the best approach to testing React components?

Well unfortunately this is something that is not very well documented and if not approached in
the correct way can lead to brittle tests.

Therefore I have written this blog post to discuss the different approaches we have available to us.

[Read More]

Puppet-Community

Puppet is an important tool to us at OpenTable; we couldn’t operate as efficiently without it but Puppet is more than a tool or a vendor, it is a community of people trying to help
each other operate increasing complex and sophisticated infrastructures.

The Puppet community and the open source efforts that drive that community have always been important to us which is why we want to take a step further in our efforts and introduce
you to the “Puppet-community” project.

##What is Puppet-community
Puppet-community is a GitHub organisation of like-minded individuals from across the wider Puppet ecosystem and from a diverse set of companies. Its principle aims are to allow the community to synchronise its efforts and to provide a GitHub organisation and Puppet Forge namespace not affiliated with any company.

Its wider aims are to provide a place for module and tool authors to share their code and the burden of maintaining it.

I would like to say that this was our idea, as it’s an excellent one, but actually all credit goes to its founders: Igor Galić, Daniele Sluijters and Spencer Krum

[Read More]

The DNS ABC

Introduction to DNS

Before joining OpenTable I was looking for a software engineer job and I’ve done my fair share of interviews. A question that has popped out a lot, and when I say a lot I mean always, is:

Could you tell me what happens when I type an URL in a web browser on my computer and press enter?

Of course the possible answers could range from “MMMHHH, wellll, I’m not sure where to start…” to a whole book on computer networks.

After a number of attempts to answer briefly and correctly, I’ve concluded that mentioning DNS can make a reasonable start.

Let’s think about it. When we type the address of the resource we want to browse, we use the alphabet, right? With letters and names easily readable and retainable by a human being.

[Read More]

Hapi.js and SIGTERM

When we first stood up our hapi.js APIs, we wrote init scripts to start/stop them. Stopping the server, was simply a case of sending SIGKILL (causing the app to immediately exit).

Whilst this is fine for most cases, if we want our apps to be good Linux citizens, then they should terminate gracefully. Hapi.js has the handy server.stop(...) command (see docs here) which will terminate the server gracefully. It will cause the server to respond to new connections with a 503 (server unavailable), and wait for existing connections to terminate (up to some specified timeout), before stopping the server and allowing the node.js process to exit. Perfect.

This makes our graceful shutdown code really simple:

1
2
3
4
5
process.on('SIGTERM', function(){
server.stop({ timeout: 5 * 1000}, function(){
process.exit(0);
});
});

When we see a SIGTERM, call server.stop(), then once the server has stopped, call process.exit(0). Easy peasy.

[Read More]

Dismantling the monolith - Microsites at Opentable

A couple of years ago we started to break-up the code-base behind our consumer site opentable.com, to smaller units of code, in order to improve our productivity. New teams were created with the goal of splitting up the logic that was powering the back-end and then bring to life new small services. Then, we started working on what we call Microsites.

Microsites

A microsite is a very small set of web-pages, or even a single one, that takes care of handling a very specific part of the system’s domain logic. Examples are the Search Results page or the Restaurant’s Profile page. Every microsite is an independently deployable unit of code, so it is easier to test, to deploy, and in consequence more resilient. Microsites are then all connected by a front-door service that handles the routing.

Not a free ride

When we deployed some microsites to production we immediately discovered a lot of pros:

  • Bi-weekly deployments of the monolith became hundreds of deployments every week.
  • Not anymore a shared codebase for hundreds of engineers. Pull requests accepted, merged, and often deployed on the same day.
  • Teams experimenting and reiterating faster: product was happy.
  • Diversity on tech stacks: teams were finally able to pick their own favourite web-stack, as soon as they were capable of deploying their code and taking care of it in terms of reliability and performance.
  • Robustness: when something was wrong with a microsite, everything else was fine.
[Read More]

A Beginner's guide to REST services

Why this post?

As a junior, I always find it easier to just sit and write code than actually stop to think about the theoretical basis that lie under the applications I work on. REST is one of those terms I heard a lot about, so I decided to try to sum up what it means and how it affects the choices we make everyday as software engineers.

Introduction to REST

REST stands for Representational State Transfer, and it can be defined as an architectural style used to build Web Services that are lightweight, maintainable, and scalable. A service that is designed by REST principles can be called a RESTful service.

It has been described first in 2000 by Roy Fielding, in a dissertation called “Architectural Styles and the Design of Network-based Software Architectures”. The basic idea was to describe the interactions between the components of a distributed system, putting constraints on them and emphasizing the importance of an uniform interface, that is abstracted from the single components.

REST is often applied to the design and development of web services, which is the scenario I’ll try to address in this post.

[Read More]

On Strongly Typed Logging

Logging is a crucial element of monitoring highly available systems. It allows not only to find out about errors but also quickly identify their cause. Logs are often used to generate metrics that help business and engineering make informative decisions on future development directions.

At OpenTable we have a central logging infrastructure, that means all logs are stored in the same shared database (ElasticSearch for us). And everybody can access any logs they want without having very specialized knowledge (thanks Kibana!).

ElasticSearch, though living in a NoSQL world, is not actually a schema-free database. Sure, you do not need to provide schema to it but instead ES will infer schema for you from documents you send to it. This is very similar to type inference you can find in many programming languages. You do not need to specify type of field, but if you later on try to assign inappropriate value to it you will get an exception.

This trait of our database goes all the way to the root of our logging system design. Let me explain why I say that we have ‘strongly typed logs’.

In The Beginning There Was String

Before centralization we just logged a single message along with its importance. In code it looked something like:

1
logger.ERROR(“Kaboom!”)

[Read More]