This was the fourth edition of cloudconf, taken place in Turin, Italy. It was an interesting conference to attend whether you are a developer, an ops or a startup entrepreneur.

Morning keynotes

The first speaker is an Atlassian ambassador, Sven Peters. He did a good job catching everybody’s attention well. First because he’s from Atlassian, but and the slides and the transition were well done and the topic is about some magical recipes to make super productive development teams.

Rise of the machine : automate your dev, by Sven Peters, Atlassian

To sum things up, the automation concerns everything in IT, from tests, deployment, code-review to even standups. He introduced us to what he called ‘bots’. For instance, Agent Charlie can download, install and configure all the tools you need at first when you join a new project.
He highlighted the workflow that developers should do which can be illustrated this way:

However, this is what is usually done:

Many developers recognized that kind of situation and made a big laugh.
That’s why Sven insisted on communication to keep the team updated.
Besides, he enumerated best practices for developers:

• Using hooks to check the build status in order to avoid branching off a bad ref (git).
• Naming someone specific from the team to do the code review.
• Organizing « Hall of shame » and « Hall of fame » and make it visible (it’s part of the gamification of work).
• Using Pagerduty: to ping the right team for failures.
• Writing release notes.
• Optimizing standups by having standup mascot to alert members when time exceeds the 5 minutes.
• Using Cron tab that redirects to an URL with a video or music.
• Having some notes/summary on chat related to standup especially when some of the team work on remote.
• Flaky test detector : detect, rerun and quarantine (disable and issue on it if not exists).

And he ended with, the best quote of the day :

“Building software is a craft, creating customer value is art”.

Coming next was Danilo Poccia, an evangelist technical from Amazon Web Services

Building event driven serverless applications

This talk was about how you can write « serverless » applications using the Amazon AWS Lambda services. The basic idea behind « serverless » is the idea that you (as the developper or entrepreneur) have not to think about infrastructure ; you write your code and you deploy it by sending it to the AWS platform.
Amazon AWS Lambda also enables you to write more events driven software with function as main pieces. Functions can generate events, that will trigger more functions, and so on. Events driven application have been around since more than 20 years, but with Lambda, tt’s like microservices but on the function level. It’s also pretty well integration into the Amazon AWS ecosystem.
Danilo gave us an Hello World example and another use case : a sample event-driven, serverless media sharing application, based on Amazon AWS S3 (storage) and Lambda.

Just after the first break, Alexis Moussine-Pouchkine, from Google, came with an exciting title:

What a great time to be a developer

This talk was awesome and pretty scary at the same time. Alexis started by giving us a brief history tour of the Google of the past 10 years and what element of their internal stack they open-sourced. From colossus to Flumejava, TensorFlow or Kubernetes, it was an impressive list with a concept in center :

Data makes software great

From that, he presented the current data-related Google API, that are pretty impressive (and scary). The Google Vision API is a powerful image API that analyses images and try to detect faces, objects or location from it. He did a demo of it with images from Paris, Turin and other cities and it was pretty accurate !
The gist of his talk was to tell the crowd how cool it is to have so much API across the internet that makes it really cool to be a developer these days. But it was also a demonstration of the power of Google in that world.

The next speaker if the Tech lead of Docker compose, Aanand Prasaad who came with a funny title and a humorous philosophy.

The internet is a computer

The first slides give the tone of the speech, opinionated, idealist and tainted with english humour.

computer are good, most software are bad

Why software are bad you may ask ? Because, most of the time, it is optimized for people who look like us — from the user perspective and from the developer perspective.
How can we fix this, you’re gonna ask then, am I right ? Well, that’s pretty easy, we have to allow more people, that are not like us, to make good software, and software that changes the world. We can do that by lowering the barrier, making it easier to write and deploy software.
Writing software is not that easy, running it locally — deploying on a computer — is not that big of a deal but deploy it to the internet is even more messy. But if you zoom out enough, internet is a computer and should look like it.
And this is where tools like the one from the Docker ecosystem comes into places, and why Aanand is working on the docker/compose project. He gave us a cool demo of how efficient you can be with, for example, the docker toolbox (compose, machine and swarm).
And a wonderful slide to finish :

“We are gatekeepers to power” :°)

Just before the lunch break, Alexander Reelsen, from Elastic gave his colorful talk and the last keynote about:

Awesome logging infrastructure with using the elastic stack

In a gist, Alexander presented how important it is to use, aggregate and visualize logs and how the Elastic stack can help with that : Elasticsearch, Kibana, Logstash and Beats.
He did a good job asking the right questions and explaining the topics you need to consider as well

• It requires expertise.
• You need to take care of access rights.
• How structured are the logs? : unstructured, semi-structured, structured.
• You need to handle timestamps carefully.

Then, there are different steps to make logging useful :

• enrichment
• centralization
• shipping logging data to the right place
• analytics
• visualization
• alerting (automating)

In addition, you have to deal with logging outages and/or peak. Logging is getting even harder with :

• microservices (with several versions)
• serverless system
• cluster/server/process management platforms
• short lived services

The basic lifecycle of a log is the following :

You have to make sure it takes short time moving from steps so you have to well design your logging infrastructure :

• shipper (beats) highly structured tool to extract logging data
• receiver (logstash and/or elasticsearch)
• visualize (kibana)

By using multiple modular tools (that “does one job, but does it well”), it becomes much easier to scale and compose.

Afternoon talks

We attended to a few talks in the afternoon.

The road to Service Oriented Architecture, by Mauro Servienti from Particular Software

Mauro gave us an interesting talk about Service Oriented Architecture by going through a use case : Buying a ticket to a conference or a sport event:

• The different point of view between user “services” and back services (examples with buying a ticket). Sometimes order in important, somewhere it’s not.
• The Transaction boundaries (business vs technical), and how you can make a parallel between SOA boundaries and Domain Driven Design bounded context.
• How messages is the main part of the SOA architecture, they should be atomic — meaning the message must be enough to move the system to a new consistent state — and unique. He gave some example of messaging patterns :
  • request/response – knows each others, tightly coupled
  • Pub/sub – broadcast event (in the past), less coupling

In the end, the conclusion was pretty obvious : in a Service Oriented Architecture, a Service is a autonomous piece of code, that can talk with others pieces through messages.

Stackoverflow behind the scene – how it’s made, by Oded Coster from Stackoverflow

Oded’s talk was almost all about Stackoverflow crazy numbers and how it’s working behind the scene. It was pretty impressive to see how much you can do with not that much infrastructure and not that big of a team.
Stackoverflow serves nearly 100 million unique visitors a month, serving a page in 40ms using 9 web servers and 2 database servers.

Advanced Docker features, by Antonio Murdaca, from Red Hat

Antonio is a Docker core maintainer and is working with containers all day long at Red Hat. His talk was about the latest advanced features that landed in the docker engine v1.10, with a focus on security :

• Logging drivers (demo with journald)
• Volumes and network plugins
• User namespaces
• SELinux integration
• Seccomp integration

In the end, lots of cool demos and ideas on how to strengthen the security of your docker platform and how to perform its integration with external tools (through drivers and plugins).

Coming next, a talk related to Docker, entitled:

Testing at scale with docker, by Emil Stolarsky from Shopify

Emil presented how Shopifiy is scaling their test platform using Docker and gave us some feedback and ideas based on the — sometimes painful — experience they got.
A Continuous Integration platform is basically two components performing:

• Scheduling
• Computing

And the goal for any continuous integration platform is :

• Quick build, to have quick feedback.
• Confidence in the build.

At Shopifiy, they use Buildkite as the scheduler of their continuous integration platform, and some amazon EC2 host with Docker on it for the build. They build their own Continuous integration “compute” components, to enable better parallelization of their build and flaky test detection ; but also mainly to have control on it.
At the end, EmilEmil focused on two interesting things based on his experience:
First, when you should build your own continuous integration platformed:

• build time exceeds 15 min: and all other optimizations exhausted
• monotholic applications
• parallelization limits: If you reached the parallelization limit your provider allows
• spiky workloads: Heavy period of work during a particular time

On the other hand, this is when you should not build your own integration platform:

• build time is less than 10 min: because it’s not a productive use of time
• small application
• problem is not parallelizable
• round the clock team

Conclusion

We had a blast attending this conference:

• We met developers and some “famous” people
• We shared some of our experiences
• We didn’t miss the conference goodies: cool tee-shirt

The italian coffee was a bit too strong but what an enjoyable day that was in Turin !
And finally, it was inspiring Vincent Demeester to present a talk on the cloud conf next year!

This post was written by Houbeb Ben Othmene and Vincent Demeester.