This is a guest blog post from Patrice Ferrot who works on our Autodesk Localization Services team. Patrice explains how localization services is using Mesos, Docker and Ochopod to improve its operational efficiency.
Localization is no easy task. Before I joined Autodesk 4 years ago, I had very little idea about the complexity of localizing software products, technical documentation, support articles, marketing content or any other localizable content. The challenges go from globalization issues (i.e. the base product is not localization ready) to the evaluation of machine translation (MT) quality and include the permanent goal of always reusing what has been translated already, adding more languages to our offering or making sure that translators productivity is optimal, to just name a few.
The complexity of properly and efficiently localizing any type of content in a multinational software corporate like Autodesk implies that we have a highly complex IT environment. In my team for example, we have 20+ systems to support. It goes from Node.js web services running on AWS to large commercial web-based applications hosted in our own data center. Most of those tools are monitored by a (very) small team of 3 ops guys. As you can imagine, it is not always easy to offer the level of service we would like with that setup. So what can we do to improve? Here are the measures that we are taking:
- Embrace the DevOps software development method in order to improve the quality of our tools/services and to improve operations performance.
- Think resources, not machines: reduces complexity for ops (one type of machine to maintain).
- Automatic configuration: simplified deployment thanks to automatic services discovery and configuration.
- Try to reduce the number of systems: with that a high number of systems, it will be a good exercise to evaluate the ROI of every one of them and possibly retire the ones that add very limited value. This will need to be discussed and negotiated with the business owners of course.
For the rest of this blog post, we will focus on measures 2. and 3.
In a nutshell, we are implementing those two measures by using Mesos (with Marathon), Docker and Ochopod. Mesos and Docker are fairly recent open source solutions that have become very popular in the last couple of years. Here are a very simplified definitions of what they are for those not familiar:
Mesos is basically a cluster manager which abstracts away the notion of individual servers from the developer and enables him/her to program against his/her data center like it is a single pool of resources. Marathon is a popular Mesos framework for long-running applications.
Docker is a container platform that allows to package an application with all of its dependencies.
Ochopod is not as well known as Mesos or Docker but should still be familiar to those reading the Autodesk Cloud Engineering blog. For the others: it is an Autodesk open source project that provides automated container orchestration over Mesos or Kubernetes.
For us in Localization Services, Ochopod was the missing piece for making the move to Mesos and Docker really worth it and even necessary. Thanks to Ochopod’s container-centric orchestration model, we can basically drop our Docker images into our Mesos/Marathon cluster and our applications will automatically configure and cluster themselves at runtime. Even port mappings – a typical challenge when working with Docker containers – is taken care of by Ochopod. How does this work? Every Docker image not only contains the software it needs to run but also Ochopod itself and the logic to configure the software (more details can be found everywhere on this blog). To make it even better, we also use Ochothon which makes it very convenient to deploy new images, scale clusters up/down and monitor running applications.
We are not yet in production with that setup. We still need to finalize our dev environment and do the necessary validation before we go live, but all looks very promising so far.
Our brand new system for managing all of the translation units processed in Localization Services is all setup to work with Ochopod. The main application is front-ended by a dedicated load-balancer which is automatically re-configured as we add/remove instances of the main application. We have decided to not leverage Ochopod with Cassandra (where the translation units are stored) for the time being because of limited flexibility in the way Cassandra ports can be configured and also because of possible complications for managing data with containers.
We have created an Ochopod-managed reverse proxy that will automatically reconfigure itself whenever a new service gets deployed into our cluster. E.g. say tomorrow we have a new service XYZ, all we need to do is to deploy the xyz-frontend Docker (with Ochopod) image into our cluster and the reverse proxy will automatically reconfigure itself to forward requests to say https://<hostname>/api/xyz/... to the xyz-frontend node(s). We still need to validate that we will want to do this in production, but this is an interesting option.
Next step: we will very shortly start integrating our (still under development) new MT infrastructure. It will be a great step forward for us. MT typically requires a lot of processing power: today, we have around 50 servers in production just for that. Leveraging Mesos will allow us to optimize resources utilization and Ochopod will enable us to easily and quickly scale our MT infrastructure up or down depending on the workload (we have no plan to automate the scaling up or down for the time being though, but being able to do it by running one single command in Ochothon will still be a game changer).
You can find below the diagram of what we are currently working on. Then we have many more tools and services where we will want to leverage Ochopod & co.
We are very excited about what we can accomplish thanks to technologies like Mesos, Docker and Ochopod and we are confident that we are on the right track to keep up with the always more aggressive business objectives while improving our level of service.