Posts Tagged ‘kumina’

KumiNews 2016: The latest and the future at a glance

Monday, January 2nd, 2017

KumiNews: The latest and the future at a glance

Kumina can look back on a successful year. A lot happened at Kumina these past 12 months. We carried out challenging new projects and started new collaborations with amazing organisations. Behind the scenes, we worked hard on optimising and extending our services. We also welcomed new colleagues at our team.

Tim’s vision on the recent and future progression and expansion of our services:

In 2017, Kumina will be celebrating its 10th anniversary. A lot has changed since Bart, Kees and I decided to merge our companies in 2007. And we expect a lot more changes in the coming years as well, as the industry is evolving faster and faster.

Where in 2007 Kumina’s main business was providing “old-style” system administration, usually on hardware provided by the customers, we’ve since moved to our own hosting offering for most of our clients. Even when clients do not make use of the Virtual Private Servers we offer, a lot of them rent hardware with us directly, using either of the two datacenters Kumina is using herself. A lot of our business still revolves around maintaining the Virtual Private Servers and associated services.

Although we noticed the rise in people asking for ‘cloud’, we always refrained from calling our offering ‘cloud’, as it is not as flexible as the public cloud environments provided by the large players in the field.

This was never much of a problem, as most customers did not require this type of flexibility. Having a reliable platform is more important and in those starting years, everyone looked a bit askance at those big clouds. Most of our customers preferred a solid partner that they could actually communicate with.

The public cloud is picking up speed with our customers as well. The ease with which one can set up an environment on a public cloud and break it down again offers a whole lot of options to our customers. Scaling is becoming an issue more often, especially when a web application has to deal with a sudden peak in concurrent users. The public cloud works great for that. Where those public cloud companies lack in personal contact, it makes up for it with the range of possibilities the platforms offer. The pay-as-you-go constructions help a lot here as well.

At Kumina, we notice that a lot of customers are expressing more and more interest in those possibilities. Although we’ve been offering our services on AWS as well for quite a while (as we are still a maintenance company, not a hosting company), until recently most of our customer’s systems ended up on either their own hardware or our hosting cluster. This is changing, however, as our customers are interested in the scaling options and quick replication that a cloud environment provides. And we’re happy to provide.

We wouldn’t be Kumina if we didn’t have an opinion about the best way of doing that. Since 2016, we’ve been working a lot with containers and Kubernetes and we’re convinced that currently, there’s no better way of working with applications than running them containerized within a Kubernetes cluster. This provides a lot of additional possibilities to our customers, which in turn allows them to do more in less time. We love to support them in this.

So for the immediate future, we expect an uptick in the number of Kubernetes setups we administer, which includes more than “just” maintaining Kubernetes of course. We add monitoring and metrics collection via Prometheus, ElasticSearch for log aggregation and lots more. We even provide a full development stack, from a Gitlab instance to an automatically deploying Jenkins.

2017 is promising to become a very interesting year regarding all the new possibilities. Are you wondering if we can be of meaning to your organisation? We always offer a free consult and advice by phone, so don’t hesitate to contact us!

Meet our new colleagues

Last year, we welcomed two new colleagues to our team: Ed Schouten and Bart Vercoulen.

In 2011 during his studies, Ed worked as part-time employee at Kumina. Five years later, after working at Google, we welcomed him back to our team. In his combined function as system administration and software developer, he mainly focusses on process optimisation. He is currently working on replacing our monitoring system. With this new feature-rich system, we can get more insights and also share these with our customers. This enables us to solve potential alerts faster and gives us the ability to get the most performance out of our systems. With his experience with large systems, algorithms, developer techniques and extended knowledge on the tools we use, like Kubernetes, Prometheus and Cassandra, he is great addition to our team.

We also welcomed our new colleague Bart, who started as our part-time junior developer. He supports the team and our internal processes by developing tools that we need internally. Thanks to his commitment we were able to take great strides towards augmenting the coverage of our monitoring and trending the past year. With his Icinga checks and Prometheus collectors we are now able to detect potential problems even faster. In the near future he will work on making our customers set-ups more comprehensible. We hope we are able to offer this new service to all our customers in the year to come.

In 2017 Niek Geerts will start as our new system administrator, who will be introduced at some point in the future. We continue assuring and improving the quality of our services. For example, we just started the process to obtain ISO 27001 certification. We will keep you informed!

Optimisation and Innovation: Open Source Releases and sponsoring
Since the foundation of Kumina we have worked almost exclusively with Open Source software. Past year, we gladly contributed to the open source community by releasing several open source software improvements and initiatives. We also decided to sponsor a promising new open source project by the company Nuxi.

Open Source Releases
Once in a while we face a challenge, without there being a solution that lives up to our quality standards. This is also the case with our current project to reimplement our monitoring to be based on Prometheus. In some cases we want to be able monitor applications that cannot yet interface with Prometheus, which is why we’ve designed these components ourselves. Curious about these and other open source releases from last year? Have a look at our Business Github page or click around on this blog.

Cooperation and sponsoring Nuxi / CloudABI
In 2014, our colleague Ed started an open source project named CloudABI. CloudABI is a framework which allows software developers to build applications that are strongly sandboxed. Sandboxing massively reduces the impact of security problems. With the use of CloudABI it is also possible to test and manage software in a better way. Kumina decided to help Ed with this promising project by sponsoring him.

Kumina sponsoring CloudABI: practical sandboxing for UNIX

Friday, October 14th, 2016

Ed Schouten: “Almost exactly two years ago I started working on a project called CloudABI. In a nutshell, CloudABI is a UNIX-like programming environment for Linux and the BSDs that allows you to easily design sandboxed applications. It accomplishes this by making strong use of capability-based security, inspired by the University of Cambridge’s Capsicum. Compared to traditional UNIX applications, CloudABI applications are better resistent against security vulnerabilities, easier to test and easier to maintain. CloudABI is available as Open Source Software, free of charge. Feel free to watch my talk at 32C3 if you’re interested in all of the nitty-gritty details.

Some time ago I decided to visit the folks at Kumina, as I used to work there until early 2012. That’s why you’ll see my name next to some of the older posts on this blog. During my visit, Tim made me an offer I simply couldn’t refuse: a job at Kumina that allows me to spend a significant amount of time every week to continue the development of CloudABI. As you can see, I’ve accepted the offer. As of last month, I’m a member of the team once again!

What brings me joy is that this step makes the development of CloudABI sustainable. Over the last couple of weeks I’ve already managed to implement at least one large new feature: support for 32-bit hardware architectures. The CloudABI Development Blog now has an article describing the work that was needed to realise this.

At Kumina my job consists of a mixture between systems administration and software development. There are various pieces of software that we’re developing in-house. One of my tasks is to release some of these as Open Source Software, so stay tuned for my next posts!”

Buckler: Authentication and authorization for Kibana, for free!

Thursday, September 29th, 2016

 

At Kumina, we make heavy use of the ELK stack: Elasticsearch, Logstash and Kibana. All of our servers have their logs collected by Logstash and stored in Elasticsearch, so we can easily access them through Kibana. As of recently we started providing direct access to our Kibana instance to our customers, so that they can perform analysis on the data themselves. This brings us to an interesting problem: Elasticsearch – and in effect Kibana – does not implement any authentication and authorization mechanisms. This means that by default customers would be able to view each other’s data.

Support for access controls is instead offered by a commercial product by Elastic, called Shield. Though Shield certainly looks like an interesting product, it looks far too advanced and costly for the problem we tried to solve at Kumina: simply having partitioned access to the data for several customers. This is why we commissioned the development of a new piece of software called Buckler. Buckler is a light-weight proxy for Kibana, written in Python (Django). It allows you to restrict access in Kibana by adding password authentication. When logged in, a user is only allowed to access indices specified for that user in Buckler’s configuration file.


Free alternative to Shield

Today we’re glad to announce that we’re releasing Buckler as open source software licensed under the Apache License, version 2.0. The Git repository containing sources and documentation can be found on our company’s Git Hub page. In addition to the proxy itself, we’re also releasing a Vagrant environment that allows you to easily test and experiment with Buckler. Right now Buckler only works in combination with Kibana 4.1, as that’ s the version in use at Kumina. There is a fair chance we’re going to extend Buckler over time to support newer versions of Kibana, such as 4.3 and 5.x.

Enjoy!

buckler_logo

awssyncer: an automatic syncer for Amazon S3 that makes use of inotify

Friday, September 16th, 2016

# awssyncer: Continuous syncing of local files into Amazon AWS S3.

At Kumina, we’re strong users of the Amazon AWS cloud computing platform. We’ve been using EC2 instances for quite some time and are currently working on expanding this by making use of Kubernetes.

While setting this up, we’ve noticed that we sometimes want to run jobs for which we want to keep track of small amounts of local state (i.e., files on disk). In this case we’ve decided that we want to store this data in S3, but do want to have it efficiently available through the local file system. The advantage of using S3 for this purpose is that it’s globally replicated, unlike EBS.

For this purpose we’ve developed a new utility called awssyncer, which is as of now available on GitHub! awssyncer is a utility written in C++ that uses Linux’s inotify to keep track of local modifications to a directory on disk. The purpose of this utility is to use these inotify events to determine which files need to be synced back into S3. This utility thus provides continuous one-way sychronisation from local disk to S3. A simple container startup script is used to sync files from S3 to local disk on startup.

Though we realise that this utility is fairly specific to our situation at hand, we do invite all of you to give it a try. Feel free to get in touch in case you have any questions or discover any bugs!